[Bug]: mllm-qwen-npu failed to run on OnePlus Ace5 pro

[wutiaojian000]

Prerequisites

• I have searched the existing issues and confirmed this is not a duplicate.
• I am using the latest version of the MLLM framework.

Bug Description

The environment similar to #574, include
Ubuntu: 22.04.5
QNN SDK: 2.41.0.251128
Hexagon NPU Runtime: 6.4.0.1
And i also trying to run mllm-qwen-npu with model Qwen1.5-1.8B-Chat on my OnePlus Ace5 pro but failed.

Steps to Reproduce

1. I tried the step on [v2] Missing demo scripts for Android QNN backend execution (was available in v1) #560, nothing but the NPU I used is v79
2. And the result

D_LIBRARY_PATH=/data/local/tmp/build-android-arm64-v8a-qnn/bin/ ./mllm-qwen-npu                                                         <
[INFO] /home/zcm/mllm/examples/qwen_npu/main.cpp:33 Mixed inference mode: NPU prefill + CPU decode
[INFO] /home/zcm/mllm/examples/qwen_npu/main.cpp:58 CPU decode model loaded from: /data/local/tmp/zhanghao/models/qwen1.5-1.8b-chat-rot_q4_0.mllm
[INFO] /home/zcm/mllm/examples/qwen_npu/main.cpp:62 Loading QNN model for prefill...
[INFO] /home/zcm/mllm/mllm/backends/qnn/QNNUtils.cpp:23 QNN Backend Lib: libQnnHtp.so
[INFO] /home/zcm/mllm/mllm/backends/qnn/QNNBackend.cpp:306 Registered Op Package: libQnnLLaMAPackage_CPU.so and interface provider: LLaMAPackageInterfaceProvider
[INFO] /home/zcm/mllm/mllm/backends/qnn/QNNBackend.cpp:306 Registered Op Package: libQnnLLaMAPackage_HTP.so and interface provider: LLaMAPackageInterfaceProvider
[INFO] /home/zcm/mllm/mllm/backends/qnn/QNNBackend.cpp:47 QNN Backend Build Id: v2.41.0.251128145156_191518
[INFO] /home/zcm/mllm/mllm/backends/qnn/QNNBackend.cpp:49 QNN backend supports tensor sparsity
[INFO] /home/zcm/mllm/mllm/backends/qnn/QNNBackend.cpp:52 QNN backend supports dynamic dimensions
[INFO] /home/zcm/mllm/mllm/backends/base/PluginSystem.cpp:89 Register customized op: DequantizeAdd:4097 -> QNN
[INFO] /home/zcm/mllm/examples/qwen_npu/main.cpp:72 Created shared StaticCache with 24 layers
[INFO] /home/zcm/mllm/examples/qwen_npu/main.cpp:77 QNN prefill model loaded from: /data/local/tmp/zhanghao/models/qwen1.5-1.8b-chat-rot-qnn.mllm
[INFO] /home/zcm/mllm/examples/qwen_npu/main.cpp:87 Configured 24 QNN KVCache layers to use shared StaticCache
[INFO] /home/zcm/mllm/examples/qwen_npu/main.cpp:97 Configured 24 CPU KVCache layers to use shared StaticCache
[INFO] /home/zcm/mllm/examples/qwen_npu/main.cpp:110 Input tokens: 194 tokens
[INFO] /home/zcm/mllm/examples/qwen_npu/main.cpp:179 Starting QNN prefill...
enter npu_model->trace(past, {})
leave npu_model->trace(past, {})
[INFO] /home/zcm/mllm/examples/qwen_npu/main.cpp:203 ************************enter graphBuildPM.run()
linalg.CPU.RMSNormOp [name="model.layers.0.input_layernorm"](%1577:tensor<[1, 194, 2048], Float32, QNN>[is_graph_input:true]) -> (%1578:tensor<[1, 194, 2048], Float32, QNN>)[ERROR] /home/zcm/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:48 Failed to cast to QNNRMSNormOp
[ERROR] /home/zcm/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:164 Failed to add node for op type: RMSNorm in graph 'model.layers.0_1'
[ERROR] /home/zcm/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp
[ERROR] /home/zcm/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:164 Failed to add node for op type: Linear in graph 'model.layers.0_2'
linalg.CPU.RMSNormOp [name="model.layers.1.input_layernorm"](%1644:tensor<[1, 194, 2048], Float32, QNN>[is_graph_input:true, is_graph_output:true]) -> (%1645:tensor<[1, 194, 2048], Float32, QNN>)[ERROR] /home/zcm/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:48 Failed to cast to QNNRMSNormOp
[ERROR] /home/zcm/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:164 Failed to add node for op type: RMSNorm in graph 'model.layers.1_1'
[ERROR] /home/zcm/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp
[ERROR] /home/zcm/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:164 Failed to add node for op type: Linear in graph 'model.layers.1_2'
...
[INFO] /home/zcm/mllm/examples/qwen_npu/main.cpp:205 ************************out graphBuildPM.run()
[WARN] /home/zcm/mllm/mllm/backends/cpu/kernels/common/ggml/vec_dot_type.hpp:181 Unsupported DataType Int8
[WARN] /home/zcm/mllm/mllm/backends/cpu/kernels/common/ggml/vec_dot_type.hpp:181 Unsupported DataType Int8
Segmentation fault

I read the source code and found when the Module reg its Layer, it also reg the BaseOp of the Layer

template<typename T, typename... Args>
  auto reg(const std::string& name, Args&&... args) {
    // Register a module
    if constexpr (std::is_base_of_v<Module, T>) {
      auto ret = T((impl_->getAbsoluteName() == "" ? name
                                                   : impl_->getAbsoluteName() + (name == "" ? "" : ".") +  // avoid double dot
                                                         name),
                   std::forward<Args>(args)...);
      impl_->regChildNode(ret.impl());
      ret.impl()->setName(name);
      return ret;
    }

    // Register to thisThread table
    if constexpr (std::is_base_of_v<Layer, T>) {
      auto ret = T(std::forward<Args>(args)...);
      impl_->regChildNode(ret.impl());
      ret.impl()->setAbsoluteName((impl_->getAbsoluteName() == ""
                                       ? name
                                       : impl_->getAbsoluteName() + (name == "" ? "" : ".") +  // avoid double dot
                                             name));
      ret.impl()->setName(name);

      auto& ctx = Context::instance();
      // Create Op
      BaseOp::ptr_t _op = nullptr;
      
      _op = ctx.getBackend(ret.impl()->getDevice())->createOp(ret.opType(), ret.refOptions());
      _op->setName(ret.impl()->getAbsoluteName());

      // Register Op
      ret.impl()->setInstancedOp(_op);

      return ret;
    }
  }

but the default value of ret.impl()->getDevice() is always kCPU, which means the Op's backend is CPUBackend(as well as the ). I don't know it's right, maybe i ingore some details. I dump the subgraph, just like

graph.SubGraphOp @model.layers.0_1 <QNN> {
    (%1577:tensor<[1, 194, 2048], Float32, QNN>[is_graph_input:true]) -> (%1594:tensor<[1, 16, 194, 128], Float32, QNN>[is_graph_output:true], %1595:tensor<[1, 16, 194, 128], Float32, QNN>[is_graph_output:true], %1596:tensor<[1, 16, 194, 128], Float32, QNN>[is_graph_output:true]) {
        linalg.CPU.RMSNormOp [name="model.layers.0.input_layernorm"](%1577:tensor<[1, 194, 2048], Float32, QNN>[is_graph_input:true]) -> (%1578:tensor<[1, 194, 2048], Float32, QNN>)
        linalg.QNN.ViewOp [name="model.layers.0_1.View.0"](%1578:tensor<[1, 194, 2048], Float32, QNN>) -> (%1579:tensor<[1, 194, 1, 2048], Float32, QNN>)
        linalg.QNN.CastTypeOp [name="model.layers.0_1.CastType.0"](%1579:tensor<[1, 194, 1, 2048], Float32, QNN>) -> (%1580:tensor<[1, 194, 1, 2048], Int16, QNN>)
        linalg.CPU.LinearOp [name="model.layers.0.self_attn.q_proj"](%1580:tensor<[1, 194, 1, 2048], Int16, QNN>) -> (%1582:tensor<[1, 194, 1, 2048], Int16, QNN>)
        linalg.CPU.LinearOp [name="model.layers.0.self_attn.k_proj"](%1580:tensor<[1, 194, 1, 2048], Int16, QNN>) -> (%1583:tensor<[1, 194, 1, 2048], Int16, QNN>)
        linalg.CPU.LinearOp [name="model.layers.0.self_attn.v_proj"](%1580:tensor<[1, 194, 1, 2048], Int16, QNN>) -> (%1584:tensor<[1, 194, 1, 2048], Int16, QNN>)
        linalg.QNN.ViewOp [name="model.layers.0_1.View.1"](%1582:tensor<[1, 194, 1, 2048], Int16, QNN>) -> (%1585:tensor<[1, 194, 16, 128], Int16, QNN>)
        linalg.QNN.ViewOp [name="model.layers.0_1.View.2"](%1583:tensor<[1, 194, 1, 2048], Int16, QNN>) -> (%1587:tensor<[1, 194, 16, 128], Int16, QNN>)
        linalg.QNN.ViewOp [name="model.layers.0_1.View.3"](%1584:tensor<[1, 194, 1, 2048], Int16, QNN>) -> (%1589:tensor<[1, 194, 16, 128], Int16, QNN>)
        linalg.QNN.DequantizeAdd [name="model.layers.0.self_attn.q_proj.dequantize"](%1585:tensor<[1, 194, 16, 128], Int16, QNN>) -> (%1591:tensor<[1, 194, 16, 128], Float32, QNN>)
        linalg.QNN.DequantizeAdd [name="model.layers.0.self_attn.k_proj.dequantize"](%1587:tensor<[1, 194, 16, 128], Int16, QNN>) -> (%1592:tensor<[1, 194, 16, 128], Float32, QNN>)
        linalg.QNN.DequantizeAdd [name="model.layers.0.self_attn.v_proj.dequantize"](%1589:tensor<[1, 194, 16, 128], Int16, QNN>) -> (%1593:tensor<[1, 194, 16, 128], Float32, QNN>)
        linalg.QNN.TransposeOp [name="model.layers.0_1.Transpose.0"](%1591:tensor<[1, 194, 16, 128], Float32, QNN>) -> (%1594:tensor<[1, 16, 194, 128], Float32, QNN>[is_graph_output:true])
        linalg.QNN.TransposeOp [name="model.layers.0_1.Transpose.1"](%1592:tensor<[1, 194, 16, 128], Float32, QNN>) -> (%1595:tensor<[1, 16, 194, 128], Float32, QNN>[is_graph_output:true])
        linalg.QNN.TransposeOp [name="model.layers.0_1.Transpose.2"](%1593:tensor<[1, 194, 16, 128], Float32, QNN>) -> (%1596:tensor<[1, 16, 194, 128], Float32, QNN>[is_graph_output:true])
        cf.ReturnOp (%1594:tensor<[1, 16, 194, 128], Float32, QNN>[is_graph_output:true], %1595:tensor<[1, 16, 194, 128], Float32, QNN>[is_graph_output:true], %1596:tensor<[1, 16, 194, 128], Float32, QNN>[is_graph_output:true]) -> ()
    }
}

Is it an issue with with the above QNN/Hexagon versions? Or could there be other possible causes?
Thanks for your help.

Expected Behavior

work correctly

Operating System

Android

Device

OnePlus Ace5 pro

MLLM Framework Version

current version

Model Information

No response

Additional Context

No response

[coderabbitai]

📝 CodeRabbit Plan Mode

Generate an implementation plan and prompts that you can use with your favorite coding agent.

• Create Plan

Examples

• Example 1
• Example 2

---

🔗 Similar Issues

Possible Duplicates

• ISSUE-574

Related Issues

• “Failed to register op package” Error when running./demo_qwen_npu createContext() #274
• Is there anyone who successfully built apk? #276
• run_qwen_qnn.sh failed to run on the Xiaomi 14. #574
• [Bug]: OpenCL backend example demo #570
• 添加llama3 模型支持时遇到了reshape错误 #286

🔗 Related PRs

#481 - feat(cli): add mllm-llm-benchmark tool for performance testing [merged]
#485 - feat(qnn): Basic QNN Prefill on v2 [merged]
#489 - feat: add kai&qnn-vl&opencl [merged]
#510 - fix: QNN Execute Return Order - handle output reordering [merged]
#537 - feat: Implement Qwen NPU Decoding Support with Memory Management Fixes [merged]

👤 Suggested Assignees

• jialilve
• HirokenOvo
• x-0D
• singlecatlmx
• ramcn

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[wutiaojian000]

I try to print the layer and it's op in the QwenDecoder

QwenDecoder(const std::string& name, const QwenNPUConfig& cfg) : nn::Module(name) {
    self_attn_proj_ = reg<QwenAttentionProjNPU>("", cfg);
    self_attn_proj_.to(kQNN);

    for(auto i : self_attn_proj_.impl()->refChildNodes()){
      std::cout << "QwenAttentionProjNPU device child:" << i->getName() << " " << i->getDevice() << std::endl;
      if(auto layer_impl = dynamic_cast<nn::LayerImpl*>(i.get())) {
        std::cout << "QwenAttentionProjNPU device child:" << layer_impl->getInstancedOp()->getName() << " " << layer_impl->getInstancedOp()->getDevice() << std::endl << std::endl;
      }
      if(auto layer_impl = dynamic_cast<nn::ModuleImpl*>(i.get()))
      {
        // std::cout << "QwenAttentionProjNPU device child:" << layer_impl->getInstancedOp()->getName() << " " << layer_impl->getInstancedOp()->getDevice() << std::endl << std::endl;
      }
    }

    self_attn_matmul_ = reg<QwenAttentionMatmul>("self_attn", cfg);
    self_attn_out_mlp_ = reg<QwenOutProjAndMLP>("", cfg);
    self_attn_out_mlp_.to(kQNN);
  }

And the result is

QwenAttentionProjNPU device child:input_layernorm 4
QwenAttentionProjNPU device child:model.layers.23.input_layernorm 1

It seems there is a contradiction.

[singlecatlmx]

@wutiaojian000 Hello, I encountered the same issue while reproducing this experiment. Are you able to run it successfully now?

[KAIWEILIUCC]

@wutiaojian000 I encountered the similar issue.

QNN Version: 2.40.0.251030
Hexagon SDK version: 6.4.0.2
Device: OnePlus 13

[INFO] /factory/mllm/examples/qwen_npu/main.cpp:33 Mixed inference mode: NPU prefill + CPU decode [INFO] /factory/mllm/examples/qwen_npu/main.cpp:55 CPU decode model loaded from: ./qwen1.5-1.8b-chat-rot_q4_0.mllm [INFO] /factory/mllm/examples/qwen_npu/main.cpp:59 Loading QNN model for prefill... [INFO] /factory/mllm/mllm/backends/qnn/QNNUtils.cpp:23 QNN Backend Lib: libQnnHtp.so [INFO] /factory/mllm/mllm/backends/qnn/QNNBackend.cpp:306 Registered Op Package: libQnnLLaMAPackage_CPU.so and interface provider: LLaMAPackageInterfaceProvider [INFO] /factory/mllm/mllm/backends/qnn/QNNBackend.cpp:306 Registered Op Package: libQnnLLaMAPackage_HTP.so and interface provider: LLaMAPackageInterfaceProvider [INFO] /factory/mllm/mllm/backends/qnn/QNNBackend.cpp:47 QNN Backend Build Id: v2.40.0.251030114326_189385 [INFO] /factory/mllm/mllm/backends/qnn/QNNBackend.cpp:49 QNN backend supports tensor sparsity [INFO] /factory/mllm/mllm/backends/qnn/QNNBackend.cpp:52 QNN backend supports dynamic dimensions [INFO] /factory/mllm/mllm/backends/base/PluginSystem.cpp:89 Register customized op: DequantizeAdd:4097 -> QNN [INFO] /factory/mllm/examples/qwen_npu/main.cpp:66 Created shared StaticCache with 24 layers [INFO] /factory/mllm/examples/qwen_npu/main.cpp:71 QNN prefill model loaded from: ./qwen1.5-1.8b-chat-rot-qnn.mllm [INFO] /factory/mllm/examples/qwen_npu/main.cpp:81 Configured 24 QNN KVCache layers to use shared StaticCache [INFO] /factory/mllm/examples/qwen_npu/main.cpp:91 Configured 24 CPU KVCache layers to use shared StaticCache [INFO] /factory/mllm/examples/qwen_npu/main.cpp:103 Input tokens: 194 tokens [INFO] /factory/mllm/examples/qwen_npu/main.cpp:161 Starting QNN prefill... [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:34 Failed to cast to QNNRMSNormOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: RMSNorm in graph 'model.layers.0_1' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: Linear in graph 'model.layers.0_2' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:34 Failed to cast to QNNRMSNormOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: RMSNorm in graph 'model.layers.1_1' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: Linear in graph 'model.layers.1_2' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:34 Failed to cast to QNNRMSNormOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: RMSNorm in graph 'model.layers.2_1' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: Linear in graph 'model.layers.2_2' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:34 Failed to cast to QNNRMSNormOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: RMSNorm in graph 'model.layers.3_1' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: Linear in graph 'model.layers.3_2' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:34 Failed to cast to QNNRMSNormOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: RMSNorm in graph 'model.layers.4_1' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: Linear in graph 'model.layers.4_2' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:34 Failed to cast to QNNRMSNormOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: RMSNorm in graph 'model.layers.5_1' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: Linear in graph 'model.layers.5_2' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:34 Failed to cast to QNNRMSNormOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: RMSNorm in graph 'model.layers.6_1' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: Linear in graph 'model.layers.6_2' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:34 Failed to cast to QNNRMSNormOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: RMSNorm in graph 'model.layers.7_1' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: Linear in graph 'model.layers.7_2' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:34 Failed to cast to QNNRMSNormOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: RMSNorm in graph 'model.layers.8_1' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: Linear in graph 'model.layers.8_2' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:34 Failed to cast to QNNRMSNormOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: RMSNorm in graph 'model.layers.9_1' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: Linear in graph 'model.layers.9_2' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:34 Failed to cast to QNNRMSNormOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: RMSNorm in graph 'model.layers.10_1' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: Linear in graph 'model.layers.10_2' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:34 Failed to cast to QNNRMSNormOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: RMSNorm in graph 'model.layers.11_1' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: Linear in graph 'model.layers.11_2' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:34 Failed to cast to QNNRMSNormOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: RMSNorm in graph 'model.layers.12_1' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: Linear in graph 'model.layers.12_2' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:34 Failed to cast to QNNRMSNormOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: RMSNorm in graph 'model.layers.13_1' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: Linear in graph 'model.layers.13_2' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:34 Failed to cast to QNNRMSNormOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: RMSNorm in graph 'model.layers.14_1' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: Linear in graph 'model.layers.14_2' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:34 Failed to cast to QNNRMSNormOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: RMSNorm in graph 'model.layers.15_1' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: Linear in graph 'model.layers.15_2' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:34 Failed to cast to QNNRMSNormOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: RMSNorm in graph 'model.layers.16_1' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: Linear in graph 'model.layers.16_2' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:34 Failed to cast to QNNRMSNormOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: RMSNorm in graph 'model.layers.17_1' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: Linear in graph 'model.layers.17_2' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:34 Failed to cast to QNNRMSNormOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: RMSNorm in graph 'model.layers.18_1' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: Linear in graph 'model.layers.18_2' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:34 Failed to cast to QNNRMSNormOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: RMSNorm in graph 'model.layers.19_1' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: Linear in graph 'model.layers.19_2' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:34 Failed to cast to QNNRMSNormOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: RMSNorm in graph 'model.layers.20_1' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: Linear in graph 'model.layers.20_2' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:34 Failed to cast to QNNRMSNormOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: RMSNorm in graph 'model.layers.21_1' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: Linear in graph 'model.layers.21_2' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:34 Failed to cast to QNNRMSNormOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: RMSNorm in graph 'model.layers.22_1' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: Linear in graph 'model.layers.22_2' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNRMSNormOp.cpp:34 Failed to cast to QNNRMSNormOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: RMSNorm in graph 'model.layers.23_1' [ERROR] /factory/mllm/mllm/backends/qnn/op/QNNLinearOp.cpp:135 Failed to cast to QNNLinearOp [ERROR] /factory/mllm/mllm/backends/qnn/passes/QNNGraphBuildPass.cpp:160 Failed to add node for op type: Linear in graph 'model.layers.23_2' [WARN] /factory/mllm/mllm/backends/cpu/kernels/common/ggml/vec_dot_type.hpp:181 Unsupported DataType Int8 [WARN] /factory/mllm/mllm/backends/cpu/kernels/common/ggml/vec_dot_type.hpp:181 Unsupported DataType Int8 Segmentation fault