[RyzenAI 1.6][Windows] quicktest.py passes but YOLOv8 object_detection sample fails with "cannot find producer" error

[keabcya]

Hi AMD team,

I’m trying out RyzenAI 1.6 on Windows and I noticed that the built-in quicktest.py passes, but the YOLOv8 object detection tutorial sample fails on NPU with a graph error.

Below are the details.

Environment

[QUARK_INFO]: OS and CPU information:
system --- Windows
node --- DESKTOP-ONMJN7M
release --- 11
version --- 10.0.26100
machine --- AMD64
processor --- AMD64 Family 25 Model 117 Stepping 2, AuthenticAMD
[QUARK_INFO]: Tools version information:
python --- 3.12.11
onnx --- 1.18.0
onnxruntime --- 1.23.0.dev20250928
quark.onnx --- 0.10+db671e3+db671e3

Python: 3.12.11 (in conda/venv yolov8m_env)

RyzenAI SDK: 1.6.0

Other relevant packages (partial pip list):

ryzen-ai-lt 1.6.0
ryzenai_onnx_utils 1.6.0
onnx 1.18.0
onnxruntime-vitisai 1.23.0
ultralytics 8.3.155
torch 2.8.0
torchvision 0.23.0

1. quicktest.py works
   (yolov8m_env) C:\Users\POC-900\RyzenAI-SW\tutorial\object_detection>cd "C:\Program Files\RyzenAI\1.6.0\quicktest"

(yolov8m_env) C:\Program Files\RyzenAI\1.6.0\quicktest>python quicktest.py
Setting environment for PHX/HPT
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20251126 02:53:34.774220 12624 register_ssmlp.cpp:124] Registering Custom Operator: com.amd:SSMLP
I20251126 02:53:34.774220 12624 register_matmulnbits.cpp:110] Registering Custom Operator: com.amd:MatMulNBits
I20251126 02:53:34.805635 12624 vitisai_compile_model.cpp:1266] Vitis AI EP Load ONNX Model Success
I20251126 02:53:34.805635 12624 vitisai_compile_model.cpp:1267] Graph Input Node Name/Shape (1)
I20251126 02:53:34.805635 12624 vitisai_compile_model.cpp:1271] input : [-1x3x32x32]
I20251126 02:53:34.805635 12624 vitisai_compile_model.cpp:1277] Graph Output Node Name/Shape (1)
I20251126 02:53:34.805635 12624 vitisai_compile_model.cpp:1281] output : [-1x10]
[Vitis AI EP] No. of Operators : NPU 398 VITIS_EP_CPU 2
[Vitis AI EP] No. of Subgraphs : NPU 1 Actually running on NPU 1
Test Passed

So the basic PHX/HPT quick test seems fine.

2. YOLOv8 object_detection sample fails

Then I follow the tutorial object detection sample:

(yolov8m_env) C:\Users\POC-900\RyzenAI-SW\tutorial\object_detection>python run_inference.py --model_input models\yolov8m_BF16.onnx --input_image test_image.jpg --output_image test_output.jpg --device npu-bf16
Input model : models\yolov8m_BF16.onnx
Input image : test_image.jpg
Output file : test_output.jpg
Running BF16 on NPU
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20251126 03:17:43.366060 6204 register_ssmlp.cpp:124] Registering Custom Operator: com.amd:SSMLP
I20251126 03:17:43.368171 6204 register_matmulnbits.cpp:110] Registering Custom Operator: com.amd:MatMulNBits
I20251126 03:17:43.573102 6204 vitisai_compile_model.cpp:1266] Vitis AI EP Load ONNX Model Success
I20251126 03:17:43.573102 6204 vitisai_compile_model.cpp:1267] Graph Input Node Name/Shape (1)
I20251126 03:17:43.573748 6204 vitisai_compile_model.cpp:1271] images : [1x3x640x640]
I20251126 03:17:43.573748 6204 vitisai_compile_model.cpp:1277] Graph Output Node Name/Shape (4)
I20251126 03:17:43.573748 6204 vitisai_compile_model.cpp:1281] output0 : [1x84x8400]
I20251126 03:17:43.573748 6204 vitisai_compile_model.cpp:1281] onnx::Reshape_905 : [1x144x80x80]
I20251126 03:17:43.573748 6204 vitisai_compile_model.cpp:1281] onnx::Reshape_924 : [1x144x40x40]
I20251126 03:17:43.573748 6204 vitisai_compile_model.cpp:1281] onnx::Reshape_943 : [1x144x20x20]
[Vitis AI EP] No. of Operators : CPU 923 VAIML 2
[Vitis AI EP] No. of Subgraphs : NPU 1 Actually running on NPU 1
check failure: node != nullptrcannot find producer. onnx_node_arg_name=split_with_sizes_24_split_0

So the ONNX model loads and gets partitioned, but then it fails with:

check failure: node != nullptrcannot find producer. onnx_node_arg_name=split_with_sizes_24_split_0

The same environment passes quicktest.py but the YOLOv8 BF16 object detection sample fails consistently with this error.

Questions

1. Is this a known issue with run_inference.py and YOLOv8 BF16 models on RyzenAI 1.6 (Windows)?

2. Does this error suggest there is something wrong with the generated yolov8m_BF16.onnx graph (e.g., missing producer for split_with_sizes_24_split_0), or is it expected to be handled inside the RyzenAI toolchain?

3. Are there recommended ONNX export settings / opset / ultralytics versions for YOLOv8 models to avoid this “cannot find producer” error?

Thanks in advance for your help!

Best regards,
Kevin Lin

[ThomasXilinx]

According to the quicktest log, you are running on a PHX/HPT system. The only models supported on PHX/HPT systems are INT8 CNNs (see: https://ryzenai.docs.amd.com/en/latest/relnotes.html#model-compatibility-table). This is why you are seeing this error when running the BF16 YOLOv8 model. Running the quantized version of YOLOv8 should work for you.

[keabcya]

Hello Thomas,

Thank you for the clarification about PHX/HPT only supporting INT8 CNN models.

Following your suggestion, I converted my FP32 YOLOv8m ONNX model to an INT8 model using quantize_quark.py with the XINT8 config. I also excluded the post-processing subgraphs as shown in the tutorial. The quantization seems to complete successfully, but running run_inference.py with --device npu-int8 still fails with a similar “cannot find producer” error, now on a quantized tensor.

1. Is there any additional constraint on the YOLOv8 “dfl” head or post-processing branch for PHX/HPT NPU that I should be aware of?
2. Is there a recommended export workflow for YOLOv8 on PHX/HPT (e.g., specific Ultralytics version / ONNX export flags / opset) that is known to work with the run_inference.py sample?

Thanks.

Here are the details.
The processor i used is AMD Ryzen 5 PRO 8640U w/ Radeon 760M Graphics (3.50 GHz).

(yolov8m_env) C:\Users\POC-900\RyzenAI-SW\tutorial\object_detection>python run_inference.py --model_input models\yolov8m_XINT8.onnx --input_image test_image.jpg --output_image test_output.jpg --device npu-int8
Input model : models\yolov8m_XINT8.onnx
Input image : test_image.jpg
Output file : test_output.jpg
Running INT8 on NPU
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20251126 05:26:13.260041 8328 register_ssmlp.cpp:124] Registering Custom Operator: com.amd:SSMLP
I20251126 05:26:13.270731 8328 register_matmulnbits.cpp:110] Registering Custom Operator: com.amd:MatMulNBits
I20251126 05:26:13.319059 8328 vitisai_compile_model.cpp:1266] Vitis AI EP Load ONNX Model Success
I20251126 05:26:13.319059 8328 vitisai_compile_model.cpp:1267] Graph Input Node Name/Shape (1)
I20251126 05:26:13.322604 8328 vitisai_compile_model.cpp:1271] images : [1x3x640x640]
I20251126 05:26:13.322604 8328 vitisai_compile_model.cpp:1277] Graph Output Node Name/Shape (4)
I20251126 05:26:13.322604 8328 vitisai_compile_model.cpp:1281] output0 : [1x84x8400]
I20251126 05:26:13.322604 8328 vitisai_compile_model.cpp:1281] onnx::Reshape_905 : [1x144x80x80]
I20251126 05:26:13.322604 8328 vitisai_compile_model.cpp:1281] onnx::Reshape_924 : [1x144x40x40]
I20251126 05:26:13.322604 8328 vitisai_compile_model.cpp:1281] onnx::Reshape_943 : [1x144x20x20]
check failure: node != nullptrcannot find producer. onnx_node_arg_name=/model.22/dfl/Transpose_output_0_QuantizeLinear_Output

(yolov8m_env) C:\Users\POC-900\RyzenAI-SW\tutorial\object_detection>python quantize_quark.py --input_model_path models\yolov8m.onnx --calib_data_path calib_images --output_model_path models\yolov8m_XINT8.onnx --config XINT8 --exclude_subgraphs "[/model.22/Concat_3], [/model.22/Concat_10]"

[QUARK-INFO]: Checking custom ops library ...

[QUARK-INFO]: The CPU version of custom ops library already exists.

[QUARK-INFO]: Checked custom ops library.
[QUARK_INFO]: Time information:
2025-11-26 05:30:17.354245
[QUARK_INFO]: OS and CPU information:
system --- Windows
node --- DESKTOP-ONMJN7M
release --- 11
version --- 10.0.26100
machine --- AMD64
processor --- AMD64 Family 25 Model 117 Stepping 2, AuthenticAMD
[QUARK_INFO]: Tools version information:
python --- 3.12.11
onnx --- 1.18.0
onnxruntime --- 1.23.0.dev20250928
quark.onnx --- 0.10+db671e3+db671e3
[QUARK_INFO]: Quantized Configuration information:
model_input --- models\yolov8m.onnx
model_output --- models\yolov8m_XINT8.onnx
calibration_data_reader --- <main.ImageDataReader object at 0x000002426C62BB30>
calibration_data_path --- None
quant_format --- QDQ
input_nodes --- []
output_nodes --- []
op_types_to_quantize --- []
extra_op_types_to_quantize --- []
per_channel --- False
reduce_range --- False
activation_type --- QUInt8
weight_type --- QInt8
nodes_to_quantize --- []
nodes_to_exclude --- []
subgraphs_to_exclude --- [(['/model.22/Concat_3'], ['/model.22/Concat_10'])]
optimize_model --- True
use_external_data_format --- False
calibrate_method --- PowerOfTwoMethod.MinMSE
execution_providers --- ['CPUExecutionProvider']
enable_npu_cnn --- True
enable_npu_transformer --- False
specific_tensor_precision --- False
debug_mode --- False
convert_fp16_to_fp32 --- False
convert_nchw_to_nhwc --- False
include_cle --- True
include_sq --- False
include_rotation --- False
include_fast_ft --- False
extra_options --- {'ActivationSymmetric': True, 'BF16QDQToCast': True}

[QUARK-INFO]: The input ONNX model can create InferenceSession successfully

[QUARK-INFO]: Obtained calibration data with 3 iters

[QUARK-INFO]: Removed initializers from input

[QUARK-INFO]: The operation types and their corresponding quantities of the input float model is shown in the table below.
┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Op Type ┃ Float Model ┃
┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Conv │ 84 │
│ Sigmoid │ 78 │
│ Mul │ 78 │
│ Slice │ 18 │
│ Add │ 14 │
│ Concat │ 19 │
│ MaxPool │ 3 │
│ Resize │ 2 │
│ Reshape │ 5 │
│ Split │ 1 │
│ Transpose │ 1 │
│ Softmax │ 1 │
│ Sub │ 2 │
│ Div │ 1 │
├──────────────────────┼───────────────────────────┤
│ Quantized model path │ models\yolov8m_XINT8.onnx │
└──────────────────────┴───────────────────────────┘

[QUARK-INFO]: The quantized information for all operation types is shown in the table below.

[QUARK-INFO]: The discrepancy between the operation types in the quantized model and the float model is due to the application of graph optimization.
┏━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┓
┃ Op Type ┃ Activation ┃ Weights ┃ Bias ┃
┡━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━┩
│ Conv │ UINT8(83) │ INT8(83) │ INT8(83) │
│ HardSigmoid │ UINT8(77) │ │ │
│ Mul │ UINT8(77) │ │ │
│ Slice │ UINT8(16) │ │ │
│ Add │ UINT8(12) │ │ │
│ Concat │ UINT8(17) │ │ │
│ MaxPool │ UINT8(3) │ │ │
│ Resize │ UINT8(2) │ │ │
│ Reshape │ UINT8(3) │ │ │
└─────────────┴────────────┴──────────┴──────────┘

(yolov8m_env) C:\Users\POC-900\RyzenAI-SW\tutorial\object_detection>

(yolov8m_env) C:\Users\POC-900\RyzenAI-SW\tutorial\object_detection>python run_inference.py --model_input models\yolov8m_XINT8.onnx --input_image test_image.jpg --output_image test_output.jpg --device npu-int8
Input model : models\yolov8m_XINT8.onnx
Input image : test_image.jpg
Output file : test_output.jpg
Running INT8 on NPU
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20251126 05:31:56.822674 11660 register_ssmlp.cpp:124] Registering Custom Operator: com.amd:SSMLP
I20251126 05:31:56.822674 11660 register_matmulnbits.cpp:110] Registering Custom Operator: com.amd:MatMulNBits
I20251126 05:31:56.870132 11660 vitisai_compile_model.cpp:1266] Vitis AI EP Load ONNX Model Success
I20251126 05:31:56.870132 11660 vitisai_compile_model.cpp:1267] Graph Input Node Name/Shape (1)
I20251126 05:31:56.870132 11660 vitisai_compile_model.cpp:1271] images : [1x3x640x640]
I20251126 05:31:56.870132 11660 vitisai_compile_model.cpp:1277] Graph Output Node Name/Shape (4)
I20251126 05:31:56.870132 11660 vitisai_compile_model.cpp:1281] output0 : [1x84x8400]
I20251126 05:31:56.870132 11660 vitisai_compile_model.cpp:1281] onnx::Reshape_905 : [1x144x80x80]
I20251126 05:31:56.870132 11660 vitisai_compile_model.cpp:1281] onnx::Reshape_924 : [1x144x40x40]
I20251126 05:31:56.870132 11660 vitisai_compile_model.cpp:1281] onnx::Reshape_943 : [1x144x20x20]
check failure: node != nullptrcannot find producer. onnx_node_arg_name=/model.22/dfl/Transpose_output_0_QuantizeLinear_Output

(yolov8m_env) C:\Users\POC-900\RyzenAI-SW\tutorial\object_detection>

[dwithchenna]

@keabcya appreciate the detailed response. Can you make to delete the old modelcachekey folder from the working directory.

Also, can you please confirm the NPU driver version being used. Thanks!

Here is a sample output on PHX device with RAI 1.6.1

Running INT8 Model on NPU
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20251126 12:17:20.524014  3924 register_ssmlp.cpp:124] Registering Custom Operator: com.amd:SSMLP
I20251126 12:17:20.524014  3924 register_matmulnbits.cpp:110] Registering Custom Operator: com.amd:MatMulNBits
I20251126 12:17:20.573239  3924 vitisai_compile_model.cpp:1263] Vitis AI EP Load ONNX Model Success
I20251126 12:17:20.573239  3924 vitisai_compile_model.cpp:1264] Graph Input Node Name/Shape (1)
I20251126 12:17:20.573239  3924 vitisai_compile_model.cpp:1268]          images : [1x3x640x640]
I20251126 12:17:20.573239  3924 vitisai_compile_model.cpp:1274] Graph Output Node Name/Shape (4)
I20251126 12:17:20.573239  3924 vitisai_compile_model.cpp:1278]          output0 : [1x84x8400]
I20251126 12:17:20.573239  3924 vitisai_compile_model.cpp:1278]          onnx::Reshape_905 : [1x144x80x80]
I20251126 12:17:20.586550  3924 vitisai_compile_model.cpp:1278]          onnx::Reshape_924 : [1x144x40x40]
I20251126 12:17:20.586550  3924 vitisai_compile_model.cpp:1278]          onnx::Reshape_943 : [1x144x20x20]
[Vitis AI EP] No. of Operators :   CPU    18    NPU  1237 VITIS_EP_CPU     7
[Vitis AI EP] No. of Subgraphs :   NPU     1 Actually running on NPU      1
2025-11-26 12:17:58.7087450 [W:onnxruntime:, session_state.cc:1316 onnxruntime::VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.
2025-11-26 12:17:58.7174498 [W:onnxruntime:, session_state.cc:1318 onnxruntime::VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.

[keabcya]

Hello @dwithchenna,

Thank you for the helpful suggestion.
After deleting the modelcachekey folder and re-running quantization + inference, the INT8 model now runs successfully on NPU.

1.Could you please provide a bit more information about why removing the modelcachekey folder resolves the issue?
2. May I know if there's any python api that I can get the NPU usage infomation?

From the user side, it appears that the cached subgraph partition or cached compilation artifacts were not being refreshed after regenerating a new ONNX model, which might explain why the runtime was still referencing the old producer nodes. But an official explanation would be very helpful for us to better understand the workflow and avoid similar issues in future deployments.

Note:

• NPU driver version: 32.0.203.314
• I also noticed that Ryzen AI SDK v1.6.1 was released yesterday, and the current environment has been updated to v1.6.1.

Thanks again for your support and guidance!

Best regards,
Kevin Lin

[dwithchenna]

@keabcya thanks for the confirmation, i'm glad that the issue is resolved.

For more details on Managing Compiled Models refer to the model compilation and deployment section in the RyzenAI-SW documentation here: https://ryzenai.docs.amd.com/en/latest/modelrun.html

When trying different experiments with models either use enable_cache_file_io_in_mem, which is enabled by default in RAI 1.6.1 or use unique model cache names. Model cache can save time by saving the compiled model.

I hope this is helpful. Let me know, if you have any other questions.