foundry model list only shows three models as NPU-enabled?

[ElSrJuez]

I was really surprised when I saw there seems to be NO NPU support for AMD NPUs on Microsoft Copilot+ PC compliant spec.

When I finally got a Qualcomm NPU-enabled laptop (the first NPU chip officially supported by MS), the first thing that I did was install Foundry Local. There seem to be only THREE models shown with qnn NPU support? (see below).
I am doing something wrong. It must be?

Where is the authoritative list of all Foundry Local models with their respective silicon support?

foundry model list

Service is Started on http://127.0.0.1:50191/, PID 13384!
Alias Device Task File Size License Model ID
phi-4 CPU chat-completion 10.16 GB MIT Phi-4-generic-cpu
phi-3.5-mini CPU chat-completion 2.53 GB MIT Phi-3.5-mini-instruct-generic-cpu
deepseek-r1-14b NPU chat-completion 7.12 GB MIT deepseek-r1-distill-qwen-14b-qnn-npu
deepseek-r1-7b NPU chat-completion 3.71 GB MIT deepseek-r1-distill-qwen-7b-qnn-npu
phi-3-mini-128k CPU chat-completion 2.54 GB MIT Phi-3-mini-128k-instruct-generic-cpu
phi-3-mini-4k CPU chat-completion 2.53 GB MIT Phi-3-mini-4k-instruct-generic-cpu
mistral-7b-v0.2 CPU chat-completion 4.07 GB apache-2.0 mistralai-Mistral-7B-Instruct-v0-2-generic-cpu
phi-4-mini-reasoning NPU chat-completion 2.78 GB MIT Phi-4-mini-reasoning-qnn-npu
CPU chat-completion 4.52 GB MIT Phi-4-mini-reasoning-generic-cpu
qwen2.5-0.5b CPU chat-completion 0.80 GB apache-2.0 qwen2.5-0.5b-instruct-generic-cpu
qwen2.5-1.5b CPU chat-completion 1.78 GB apache-2.0 qwen2.5-1.5b-instruct-generic-cpu
qwen2.5-coder-0.5b CPU chat-completion 0.80 GB apache-2.0 qwen2.5-coder-0.5b-instruct-generic-cpu
qwen2.5-coder-7b CPU chat-completion 6.16 GB apache-2.0 qwen2.5-coder-7b-instruct-generic-cpu
qwen2.5-coder-1.5b CPU chat-completion 1.78 GB apache-2.0 qwen2.5-coder-1.5b-instruct-generic-cpu
qwen2.5-14b CPU chat-completion 11.06 GB apache-2.0 qwen2.5-14b-instruct-generic-cpu
qwen2.5-7b CPU chat-completion 6.16 GB apache-2.0 qwen2.5-7b-instruct-generic-cpu
qwen2.5-coder-14b CPU chat-completion 11.06 GB apache-2.0 qwen2.5-coder-14b-instruct-generic-cpu

[natke]

Support for AMD NPUs is coming in the next release, along with broader support for Qualcomm NPUs

[natke]

AMD NPU support is available since 0.7.117

[sunilkumar-rakuten]

@natke I used olive to convert custom model to be used with npu, I was able to use this with foundry v6 but with upgrade to v8 (to be able to use amd strix halo npu), I am getting below error. I have no issues in running the models provided with npu support on foundry local catalogue, is there any more documentation on library versions compatibility besides: https://learn.microsoft.com/en-us/azure/ai-foundry/foundry-local/how-to/how-to-compile-hugging-face-models?view=foundry-classic
#65

olive auto-opt `
    --model_name_or_path local_model `
    --output_path models/7B-ONNX-INT4-Olive-NPU-FoundryLocal `
    --device npu `
    --provider VitisAIExecutionProvider `
    --use_ort_genai `
    --precision int4 `
    --log_level 3 `
    --use_model_builder

Libraries

olive-ai @ git+https://github.com/microsoft/Olive.git@8b44cf49e03b8c4bf5d9f31ec3aa6a5f22d7677d
onnx==1.18.0
onnx-ir==0.1.12
onnx-tool==0.9.0
onnxoptimizer @ file:///C:/Program%20Files/RyzenAI/1.6.1/onnxoptimizer-0.3.19-cp312-cp312-win_amd64.whl#sha256=5ae6edeea455859216802a939fe7b4477122014fc42dc51f738614d1298546eb
onnxruntime-genai-directml-ryzenai @ file:///C:/Program%20Files/RyzenAI/1.6.1/onnxruntime_genai_directml_ryzenai-0.9.2-cp312-cp312-win_amd64.whl#sha256=496f23d9b47d7deaf8fcb58cbd17fab9753227b91b855c43c4908d4e2e44784a
onnxruntime-vitisai @ file:///C:/Program%20Files/RyzenAI/1.6.1/onnxruntime_vitisai-1.23.2-cp312-cp312-win_amd64.whl#sha256=4b48265bd2754f05a9bcbdfc5c54ea185a832c73d3fe61df15fadc4d8023b6d8
onnxruntime_extensions==0.13.0
onnxscript==0.5.6
onnxsim @ file:///C:/Program%20Files/RyzenAI/1.6.1/onnxsim-0.4.36-cp312-cp312-win_amd64.whl#sha256=da202a54288196105d1371a5dc0a792f2e9d97065c7778508a4021229111cb37

Error

Interactive Chat. Enter /? or /help for help.
Press Ctrl+C to cancel generation. Type /exit to leave the chat.

Interactive mode, please enter your prompt
> Hi
🧠 Thinking   [11:36:14 ERR] Connection id "0HNH8LSA3UHVR", Request id "0HNH8LSA3UHVR:00000001": An unhandled exception was thrown by the application.
Exception: Error during chat
The response ended prematurely. (ResponseEnded)
[11:36:14 ERR] LogException
Microsoft.AI.Foundry.Local.Common.FLException: Error during chat
 ---> System.Net.Http.HttpIOException: The response ended prematurely. (ResponseEnded)
   at System.Net.Http.HttpConnection.<FillAsync>d__82.MoveNext() + 0x265
--- End of stack trace from previous location ---
   at System.Net.Http.HttpConnection.ChunkedEncodingReadStream.<ReadAsyncCore>d__6.MoveNext() + 0x25b
--- End of stack trace from previous location ---
   at System.ClientModel.Internal.ReadTimeoutStream.<ReadAsync>d__21.MoveNext() + 0x24d
--- End of stack trace from previous location ---
   at System.Net.ServerSentEvents.SseParser`1.<FillLineBufferAsync>d__35.MoveNext() + 0x1ba
--- End of stack trace from previous location ---
   at System.Net.ServerSentEvents.SseParser`1.<EnumerateAsync>d__20.MoveNext() + 0x241
--- End of stack trace from previous location ---
   at OpenAI.AsyncSseUpdateCollection`1.AsyncSseUpdateEnumerator`1.<System-Collections-Generic-IAsyncEnumerator<U>-MoveNextAsync>d__12.MoveNext() + 0x1de
--- End of stack trace from previous location ---
   at OpenAI.AsyncSseUpdateCollection`1.<GetValuesFromPageAsync>d__8.MoveNext() + 0x261
--- End of stack trace from previous location ---
   at OpenAI.AsyncSseUpdateCollection`1.<GetValuesFromPageAsync>d__8.MoveNext() + 0x3ba
--- End of stack trace from previous location ---
   at System.ClientModel.AsyncCollectionResult`1.<GetAsyncEnumerator>d__1.MoveNext() + 0x339
--- End of stack trace from previous location ---
   at System.ClientModel.AsyncCollectionResult`1.<GetAsyncEnumerator>d__1.MoveNext() + 0x69f
--- End of stack trace from previous location ---
   at System.ClientModel.AsyncCollectionResult`1.<GetAsyncEnumerator>d__1.MoveNext() + 0x7d4
--- End of stack trace from previous location ---
   at Microsoft.AI.Foundry.Local.Commands.ModelRunCommand.<InteractiveNewRoundAsync>d__7.MoveNext() + 0x2a0
--- End of stack trace from previous location ---
   at Microsoft.AI.Foundry.Local.Commands.ModelRunCommand.<InteractiveNewRoundAsync>d__7.MoveNext() + 0x618
   --- End of inner exception stack trace ---
   at Microsoft.AI.Foundry.Local.Commands.ModelRunCommand.HandleExceptionDuringChat(Exception) + 0xee
   at Microsoft.AI.Foundry.Local.Commands.ModelRunCommand.<InteractiveNewRoundAsync>d__7.MoveNext() + 0x804
--- End of stack trace from previous location ---
   at Microsoft.AI.Foundry.Local.Commands.ModelRunCommand.<RunInteractiveNewRoundAsync>d__3.MoveNext() + 0x5c
--- End of stack trace from previous location ---
   at Microsoft.AI.Foundry.Local.Commands.ModelRunCommand.<<Create>b__1_0>d.MoveNext() + 0x172e
--- End of stack trace from previous location ---
   at Microsoft.AI.Foundry.Local.Common.CommandActionFactory.<>c__DisplayClass0_0`1.<<Create>b__0>d.MoveNext() + 0x1e7
--- End of stack trace from previous location ---
   at System.CommandLine.NamingConventionBinder.CommandHandler.<GetExitCodeAsync>d__66.MoveNext() + 0x50
--- End of stack trace from previous location ---
   at System.CommandLine.NamingConventionBinder.ModelBindingCommandHandler.<InvokeAsync>d__11.MoveNext() + 0x61
--- End of stack trace from previous location ---
   at System.CommandLine.Invocation.InvocationPipeline.<InvokeAsync>d__0.MoveNext() + 0x1cd
--- End of stack trace from previous location ---
   at Microsoft.AI.Foundry.Local.Program.<Main>d__1.MoveNext() + 0x4e4
Please check the service status with 'foundry service status'.

[EmmaNingMS]

Hi @sunilkumar-rakuten, which model are you using here? For model optimization using Olive for AMD NPU, here is an optimization receipt by the AMD team: https://github.com/microsoft/olive-recipes/tree/main/Qwen-Qwen2.5-1.5B-Instruct/VitisAI.

[sunilkumar-rakuten]

@EmmaNingMS thanks for sharing the recipes, the issue still remains same. I am trying mistral architecture with VitisAI

[EmmaNingMS]

Here is the optimization receipt for Mistral: https://github.com/microsoft/olive-recipes/tree/main/mistralai-Mistral-7B-Instruct-v0.3/VitisAI.

[sunilkumar-rakuten]

Yup, I used mistral recipe for conversion. @EmmaNingMS

I think the issue is originating from Layer Norm support. I tried loading model with onnxruntime and I get this.

ONNX Runtime version: 1.23.2.dev20251030
2025-11-27 12:15:54.2495128 [I:onnxruntime:, inference_session.cc:606 onnxruntime::InferenceSession::TraceSessionOptions] Session Options {  execution_mode:0 execution_order:DEFAULT enable_profiling:0 optimized_model_filepath: enable_mem_pattern:1 enable_mem_reuse:1 enable_cpu_mem_arena:1 profile_file_prefix:onnxruntime_profile_ session_logid: session_log_severity_level:1 session_log_verbosity_level:0 max_num_graph_transformation_steps:10 graph_optimization_level:4 intra_op_param:OrtThreadPoolParams { thread_pool_size: 0 auto_set_affinity: 0 allow_spinning: 1 dynamic_block_base_: 0 stack_size: 0 affinity_str:  set_denormal_as_zero: 0 } inter_op_param:OrtThreadPoolParams { thread_pool_size: 0 auto_set_affinity: 0 allow_spinning: 1 dynamic_block_base_: 0 stack_size: 0 affinity_str:  set_denormal_as_zero: 0 } use_per_session_threads:1 thread_pool_allow_spinning:1 use_deterministic_compute:0 ep_selection_policy:0 config_options: {  } }       
2025-11-27 12:15:54.2656812 [I:onnxruntime:, inference_session.cc:414 onnxruntime::InferenceSession::ConstructorCommon::<lambda_cd5baf9f006fce338b072dac99354b33>::operator ()] Flush-to-zero and denormal-as-zero are off
2025-11-27 12:15:54.2697336 [I:onnxruntime:, inference_session.cc:422 onnxruntime::InferenceSession::ConstructorCommon] Creating and using per session threadpools since use_per_session_threads_ is true
2025-11-27 12:15:54.2730494 [I:onnxruntime:, inference_session.cc:446 onnxruntime::InferenceSession::ConstructorCommon] Dynamic block base set to 0
Failed to create session: [ONNXRuntimeError] : 1 : FAIL : Load model from C:\Users\r10\Desktop\models\vai\model.onnx failed:Fatal error: com.amd:AMDSimplifiedLayerNormalization(-1) is not a registered function/op

I am using RyzenAI-1.6.1

[natke]

@sunilkumar-rakuten I've created a new issue for this, as the preceding discussion is not applicable #344