-
Notifications
You must be signed in to change notification settings - Fork 110
Description
I have a computer vision model (quantised with xint8). In it I have a Resize layer with mode=linear. Apparently this gets moved to the CPU. I see this when executing:
[Vitis AI EP] No. of Operators : CPU 8 NPU 2356 VITIS_EP_CPU 13
[Vitis AI EP] No. of Subgraphs : NPU 2 Actually running on NPU 2
2025-12-25 15:21:24.923046791 [W:onnxruntime:, session_state.cc:1316 VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.
2025-12-25 15:21:24.923067150 [W:onnxruntime:, session_state.cc:1318 VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.
I have set explicitly some of the initial layers to the CPU as well but I'd like to only get one NPU subgraph -- so it would be nice to get the Resize in the middle also into NPU.
Now I have some other Resize layers in my model that use mode=nearest and they execute on the DPU. So. I figured that maybe nearest is better supported and changed the linear Resize layers into nearest -- hoping that it would then run them on the DPU. But then this happens:
[Vitis AI EP] No. of Operators : CPU 713 VITIS_EP_CPU 1664
2025-12-25 15:04:34.142350845 [W:onnxruntime:, session_state.cc:1316 VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.
2025-12-25 15:04:34.142371073 [W:onnxruntime:, session_state.cc:1318 VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.
Apparently now the entire model is moved to run on the CPU and nothing executed on the NPU. Why? This makes no sense. Also. Is there any way to get debug on what layers and why are moved to the CPU? It would really help to get some sort of instrumentation. This "aianalyzer" seems to do nothing:
options = {
'ai_analyzer_visualization': True,
'ai_analyzer_profiling': True,
}
sess = ort.InferenceSession(path, sess_options=opts, providers=['VitisAIExecutionProvider'], provider_options=[options])
I see no "logs" anywhere. It seems not to be working. Or are they created somewhere else, where? How can I help myself?
I'm running ryzen_ai-1.6.1 on Ubuntu 22.04.