Integrate at least one vendor SDK for real NPU compilation

[m96-chan]

Problem

All 8 vendor backends (Samsung, Intel, AMD, Qualcomm, Arm-Ethos, CEVA, Rockchip, MediaTek) are pass-through wrappers that delegate to ONNX or TFLite and add a diagnostic message string. For example:

// crates/nxpu-backend-intel/src/lib.rs:30-36
let mut output = OnnxBackend.compile(module, opts)?;
output.diagnostics.push(Diagnostic {
    message: "Load in OpenVINO: ov::Core::read_model(\"output.onnx\")".into(),
});

No vendor SDK is actually invoked. No vendor-specific binary format is produced.

Requirements

Pick at least one vendor and implement actual SDK integration:

Recommended: Arm Ethos-U (via Vela)

• Invoke vela CLI tool on the emitted TFLite model
• Parse Vela output for optimized .tflite with Ethos-U custom ops
• Return the Vela-optimized binary as BackendOutput

Alternative: Intel OpenVINO

• Use OpenVINO C API (via FFI) or invoke mo (Model Optimizer) CLI
• Return the compiled OpenVINO IR (.xml + .bin)

Alternative: Qualcomm QNN

• Invoke qnn-onnx-converter + qnn-net-run CLI tools
• Return the QNN model library (.so)

Impact

Without real vendor integration, the transpiler cannot produce artifacts that run on actual NPU hardware. The entire purpose of the project is blocked.

[m96-chan]

Vendor NPU SDK Research

Research on all 8 vendor SDKs referenced by the existing backend stubs. This covers SDK location, installation, input/output formats, CLI/API surface, licensing, and integration approach for NxPU.

---

1. Arm Ethos-U — Vela Compiler

Item	Detail
SDK	Vela (open-source CLI compiler)
Install	pip install ethos-u-vela
Input	TFLite (int8 quantized)
Output	Optimized TFLite with Ethos-U custom ops
CLI	vela model.tflite --accelerator-config ethos-u55-128
Targets	Ethos-U55, Ethos-U65, Ethos-U85
License	Apache 2.0
Repo	https://review.mlplatform.org/plugins/gitiles/ml/ethos-u/ethos-u-vela
Dynamic shapes	No — static only
Quantization	Required (int8); Vela does not quantize, expects pre-quantized TFLite

NxPU integration approach: Emit TFLite via nxpu-backend-tflite, apply int8 quantization pass, then invoke vela CLI as a subprocess. Return the Vela-optimized .tflite as binary output. Straightforward — Vela is a simple CLI with no SDK dependency.

---

2. Intel — OpenVINO

Item	Detail
SDK	OpenVINO
Install	pip install openvino (Python) or download C/C++ runtime from Intel
Input	ONNX (opset ≤17), TF SavedModel, PaddlePaddle, PyTorch
Output	OpenVINO IR (.xml + .bin) or in-memory ov::CompiledModel
CLI	ovc model.onnx (replaces old mo Model Optimizer)
C API	ov_core_create(), ov_core_read_model(), ov_core_compile_model() — suitable for FFI
Targets	Intel NPU 3720 (Meteor Lake), NPU 4000 (Lunar Lake), CPU, iGPU
License	Apache 2.0
Repo	https://github.com/openvinotoolkit/openvino
Dynamic shapes	NPU: static only (must set fixed shapes before compile); CPU/GPU: dynamic OK
Quantization	NNCF toolkit or POT; FP16/INT8

NxPU integration approach: Emit ONNX via nxpu-backend-onnx, then either (a) invoke ovc CLI to produce IR files, or (b) use OpenVINO C API via FFI (ov_core_t etc.) to compile in-process. C API is well-documented and stable. Return .xml + .bin as multi-file output.

---

3. Qualcomm — QNN SDK

Item	Detail
SDK	Qualcomm AI Engine Direct (QNN)
Install	Download from Qualcomm Developer portal (requires account)
Input	ONNX (opset ≤21), TFLite, PyTorch
Output	QNN model library (.so) or context binary (.bin)
CLI pipeline	qnn-onnx-converter → qnn-model-lib-generator → qnn-context-binary-generator
Targets	Hexagon HTP (NPU), Adreno GPU, Kryo CPU
License	Proprietary (free download with Qualcomm account)
Dynamic shapes	No — static only
Quantization	Required for HTP; converter supports --input_list for calibration

NxPU integration approach: Emit ONNX, then invoke the 3-step QNN CLI pipeline. The SDK is proprietary and requires manual installation, so this backend should detect SDK availability at runtime and fail gracefully if not found. Return .bin context binary as output.

---

4. Samsung — ONE (On-device Neural Engine)

Item	Detail
SDK	Samsung ONE (open-source compiler toolchain)
Install	Build from source (./nncc configure && ./nncc build) or use prebuilt Debian packages from GitHub releases
Input	TFLite, ONNX, Circle
Output	Circle (.circle) → TVN (.tvn, device-optimized)
CLI	onecc — unified compiler driver; sub-tools: one-import-onnx, one-import-tflite, one-optimize, one-quantize, one-codegen
Targets	Samsung Exynos NPU
License	Apache 2.0
Repo	https://github.com/Samsung/ONE
Quantization	one-quantize (post-training quantization, int8/uint8)

NxPU integration approach: Emit ONNX or TFLite, then invoke onecc pipeline: one-import-onnx → one-optimize → one-quantize → one-codegen. Circle is an intermediate format (FlatBuffers-based, similar to TFLite). The onecc driver wraps the sub-tools into a single config-driven invocation.

---

5. MediaTek — NeuroPilot / Neuron SDK

Item	Detail
SDK	Neuron SDK (part of NeuroPilot)
Install	Ships with MediaTek Genio Yocto BSP; neuropilot-sdk-basic-<version> package
Input	TFLite (quantized int8/uint8)
Output	DLA (Deep Learning Archive) — proprietary binary for MDLA/VPU
CLI	ncc-tflite --arch=mdla3.0 model.tflite
Runtime	neuronrt — C/C++ API to load DLA and run inference
Targets	Genio 350/510/700/1200 (Dimensity SoCs with MDLA)
License	Proprietary (bundled with Genio BSP)

NxPU integration approach: Emit TFLite with int8 quantization, then invoke ncc-tflite CLI. SDK is platform-specific (ships with BSP), so backend should check for ncc-tflite in PATH. Return .dla binary as output.

---

6. Rockchip — RKNN-Toolkit2

Item	Detail
SDK	RKNN-Toolkit2
Install	pip install rknn-toolkit2 (Python, x86_64 host) or rknn-toolkit-lite2 (on-device)
Input	ONNX, TFLite, PyTorch, Caffe, Darknet
Output	.rknn (proprietary binary for Rockchip NPU)
API	Python: rknn.config() → rknn.load_onnx() → rknn.build(do_quantization=True) → rknn.export_rknn()
Targets	RK3588, RK3576, RK3566, RK3568 (NPU 1.0/2.0)
License	BSD 3-Clause (toolkit); proprietary (runtime)
Quantization	Built-in asymmetric int8/int16, hybrid quantization

NxPU integration approach: Emit ONNX, then use RKNN-Toolkit2 Python API (invoke via subprocess with a small Python script). No CLI tool — must script the Python API. Return .rknn binary. Alternative: wrap the conversion in a standalone Python helper that NxPU shells out to.

---

7. CEVA — NeuPro Studio / CDNN

Item	Detail
SDK	CEVA NeuPro Studio
Install	IP licensee only — available to companies licensing CEVA NeuPro IP
Input	ONNX, TVM-compatible models
Output	Proprietary binary for NeuPro-M NPU
CLI	CDNN compiler (details not publicly documented)
License	Proprietary (IP license required)
Availability	Not publicly downloadable; contact CEVA sales

NxPU integration approach: This backend cannot be implemented without an IP license from CEVA. The SDK/tools are not publicly available. Recommend keeping as a stub with documentation noting the requirement. If we get access, the approach would be ONNX → CDNN compiler → proprietary binary.

---

8. AMD — Ryzen AI / Vitis AI

Item	Detail
SDK	Ryzen AI Software (v1.7.0, Jan 2026)
Install	Download installer from AMD developer portal (Windows primary; Linux experimental)
Input	ONNX (via ONNX Runtime with Vitis AI EP)
Output	Compiled micro-code for XDNA NPU (in-memory, not a standalone file)
Compilation	Integrated into ONNX Runtime — model compiles at session start, no separate CLI
Quantization	AMD Quark (pip install amd-quark) or Olive; ONNX int8/int16
Targets	AMD XDNA NPU (Ryzen 7040+, Ryzen AI 300, Strix Point)
License	Proprietary (free download)
Repo	https://github.com/amd/RyzenAI-SW

NxPU integration approach: Emit ONNX, then quantize with Quark. The compilation happens inside ONNX Runtime via the Vitis AI EP — there's no standalone compiler CLI that produces a portable binary. This makes traditional "compile and return binary" integration difficult. Options: (a) shell out to a Python script that loads via ONNX Runtime + Vitis AI EP and caches the compiled model, or (b) emit quantized ONNX + a loader script.

---

Summary & Recommendations

Vendor	Feasibility	Public SDK	CLI Available	Recommended Priority
Arm Vela	Easy	Yes (pip)	Yes	1st — simplest integration
Intel OpenVINO	Easy	Yes (pip)	Yes + C API	2nd — great C API for FFI
Samsung ONE	Medium	Yes (GitHub)	Yes (onecc)	3rd
Rockchip RKNN	Medium	Yes (pip)	Python API only	4th
Qualcomm QNN	Medium	Free download	Yes (3-step)	5th
MediaTek	Hard	BSP only	Yes (ncc-tflite)	6th
AMD Ryzen AI	Hard	Free download	No standalone CLI	7th
CEVA	Blocked	IP license only	Unknown	8th — cannot proceed without license

Recommendation: Start with Arm Vela (simplest, pip install, single CLI call) and Intel OpenVINO (stable C API, Apache 2.0, broad adoption).