Skip to content

[BUG] nnfusion fail: Unknown op count: 2 #533

@juinshell

Description

@juinshell

🐛 Bug
Hello, I'm trying to use nnfusion to gen cuda code for transformer-based model provided by hugging face. But after I get onnx format model by optimum-cli, I cannot deliver this onnx model to nnfusion correctly. I wonder whether I get the model right or nnfusion cannot smoothly transform these kind of transformer-based model?

To Reproduce
Steps to reproduce the behavior:

  1. use optimum-cli to convert the hugging face checkpoint to onnx format model:
optimum-cli export onnx --model google/vit-base-patch16-224-in21k vit-base-patch16-224-in21k_onnx_32/ --batch_size 32

(--bath_size arg seems not change the onnx output when I visualize both by netron)
2. use nnfusion to convert (ERROR)

nnfusion ./vit-base-patch16-224-in21k_onnx_32/model.onnx -f onnx -p "batch_size:32;num_channels:3;height:224;width:224"

and the error output meg:

...
  name: "main_graph"
  input {
    name: "pixel_values"
    type {
      tensor_type {
        elem_type: 1
        shape {
          dim {
            dim_param: "batch_size"
          }
          dim {
            dim_param: "num_channels"
          }
          dim {
            dim_param: "height"
          }
          dim {
            dim_param: "width"
          }
        }
      }
    }
  }
  output {
    name: "last_hidden_state"
    type {
      tensor_type {
        elem_type: 1
        shape {
          dim {
            dim_param: "batch_size"
          }
          dim {
            dim_param: "Addlast_hidden_state_dim_1"
          }
          dim {
            dim_param: "Addlast_hidden_state_dim_2"
          }
        }
      }
    }
  }
}
opset_import {
  version: 14
}

[INFO] 2024-08-10T12:33:27z thirdparty/ngraph/src/nnfusion/frontend/onnx_import/util/graph_convert.cpp 309        Converting Onnx Graph
[ERROR] 2024-08-10T12:33:28z thirdparty/ngraph/src/nnfusion/frontend/onnx_import/util/graph_convert.cpp 421       Unknown op type: Where
[ERROR] 2024-08-10T12:33:28z thirdparty/ngraph/src/nnfusion/frontend/onnx_import/util/graph_convert.cpp 421       Unknown op type: ConstantOfShape
[ERROR] 2024-08-10T12:33:28z src/nnfusion/util/errors.hpp 169   Failure at /root/nnfusion/thirdparty/ngraph/src/nnfusion/frontend/onnx_import/util/graph_convert.cpp:423:
Unknown op count: 2
terminate called after throwing an instance of 'nnfusion::errors::CheckError'
  what():  Failure at /root/nnfusion/thirdparty/ngraph/src/nnfusion/frontend/onnx_import/util/graph_convert.cpp:423:
Unknown op count: 2
Aborted (core dumped)

Expected behavior
Successfully generate CUDA code and run well.

Additional context
I use the docker image from your repo, it goes well when I just use the model nnfusion provided in example.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions