Use the same model for GPU and CPU

[thewh1teagle]

I would like to use olive to create simple single whisper medium int8 model for the app Vibe.
However it seems that olive need separate models per GPU / CPU.
I want to simplify that to single model. Is that possible?
Thanks

[samuel100]

@thewh1teagle Ultimately, an ONNX model will run on the ONNX Runtime. So to answer your question... you can optimize a model for CPU and then inference it on both a GPU and CPU machine.

However, Olive optimizes the model for specific devices (NPU/GPU/CPU) from different hardware vendors (Nvidia, Qualcomm, etc). Therefore, you will likely get better performance by optimizing for the specific device you want to run on.