[Bug] ModuleNotFoundError & Crash when running model builder on v0.11.4 (macOS & Windows)

[jefferyby]

Description
I am trying to convert the HuggingFaceTB/SmolVLM-256M-Instruct model using the onnxruntime-genai model builder.
I have encountered a persistent ModuleNotFoundError: No module named 'onnx_ir' error on both macOS (M2 Pro) and Windows 11. On macOS, after attempting a workaround by setting PYTHONPATH, the script then crashes with a libc++abi: terminating... error.
It seems that the installed pip package (v0.11.4) has issues with path resolution for its binary extensions and may have underlying stability problems on macOS with certain model architectures.
System Information
Hardware 1: MacBook Pro M2 (Apple Silicon)
Hardware 2: Windows 11 PC (x64)
Python Version: 3.11 (Verified on both venv and Miniconda)
Package Version: onnxruntime-genai==0.11.4 (installed via pip)
Target Model: HuggingFaceTB/SmolVLM-256M-Instruct
Steps to Reproduce
Create a clean Python 3.11 environment.
Install the package: pip install onnxruntime-genai huggingface-hub
Run the model builder command:
code
Bash
python -m onnxruntime_genai.models.builder
-m ./SmolVLM-256M-Instruct
-o ./output_int4
-p int4
-e cpu
--trust_remote_code
Error Logs
The primary error is a ModuleNotFoundError:
code
Text
Traceback (most recent call last):
...
File ".../site-packages/onnxruntime_genai/models/builder.py", line 15, in
import onnx_ir as ir
ModuleNotFoundError: No module named 'onnx_ir'
On macOS, after setting PYTHONPATH to resolve the import, a C++ crash occurs:
code
Text
libc++abi: terminating due to uncaught exception of type std::runtime_error
Context
Our technical goal is to deploy Vision-Language Models on mobile platforms (iOS/Android) for efficient, on-device inference. The model builder is a critical part of our workflow for generating the required assets (.onnx and genai_config.json). This blocker currently prevents us from proceeding with our mobile deployment pipeline.
Any guidance or a fix would be greatly appreciated.

[kunal-vaishnavi]

The SmolVLM-256M-Instruct is not currently supported in the model builder. It is a vision-language model and prior versions of SmolLM were language models.

Nevertheless, an error should print stating this. I was unable to reproduce your issue on Windows but I was able to produce the expected error.

PS C:\Users\username> python -m venv venv
PS C:\Users\username> .\venv\Scripts\Activate.ps1  
(venv) PS C:\Users\username> pip install onnxruntime-genai
(venv) PS C:\Users\username> pip install onnx-ir
(venv) PS C:\Users\username> pip install torch transformers
(venv) PS C:\Users\username> python -m onnxruntime_genai.models.builder -m HuggingFaceTB/SmolVLM-256M-Instruct -o ./output_int4 -p int4 -e cpu --extra_options trust_remote_code=1                      
Valid precision + execution provider combinations are: FP32 CPU, FP32 CUDA, FP16 CUDA, FP16 DML, BF16 CUDA, FP16 TRT-RTX, INT4 CPU, INT4 CUDA, INT4 DML, INT4 WebGPU
Extra options: {'trust_remote_code': '1'}
config.json: 7.35kB [00:00, 3.64MB/s]
C:\Users\username\venv\Lib\site-packages\huggingface_hub\file_download.py:143: UserWarning: `huggingface_hub` cache-system uses symlinks by default to efficiently store duplicated files but your machine does not support them in C:\Users\username\cache_dir\models--HuggingFaceTB--SmolVLM-256M-Instruct. Caching files will still work but in a degraded version that might require more space on your disk. This warning can be disabled by setting the `HF_HUB_DISABLE_SYMLINKS_WARNING` environment variable. For more details, see https://huggingface.co/docs/huggingface_hub/how-to-cache#limitations.
To support symlinks on Windows, you either need to activate Developer Mode or to run Python as an administrator. In order to activate developer mode, see this article: https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development
  warnings.warn(message)
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\username\venv\Lib\site-packages\onnxruntime_genai\models\builder.py", line 479, in <module>
    create_model(
  File "C:\Users\username\venv\Lib\site-packages\torch\utils\_contextlib.py", line 120, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\username\venv\Lib\site-packages\onnxruntime_genai\models\builder.py", line 321, in create_model
    raise NotImplementedError(f"The {hf_name} model is not currently supported.")
NotImplementedError: The HuggingFaceTB/SmolVLM-256M-Instruct model is not currently supported.

Your PYTHONPATH change should not be needed. That may be causing the subsequent issues you observed.