Minimum Memory Requirement for Running LLMs on NPU

[kimminsu38oo]

#134
#114

they mention that when running an LLM model on the NPU, the inference fails on devices with less than 12 GB of memory

Is it still genuinely the case that devices with under 12 GB of memory cannot run these models on the NPU?

Or has there been any update/solution for running on lower-memory devices?

[chenghuaWang]

We are currently wroking on Qnn AOT Compilation to generate qnn executable file on X86 machine. The code is in backends/qnn/aot