Request for MLX Compatibility & Conversion Guide

[yourappleintelligence]

I’m interested in running the NAMO-R1 model on Apple Silicon using MLX (Machine Learning for Apple Devices). Since MLX is optimized for Apple hardware and supports transformer-based models, I’d like to convert NAMO-R1 into an MLX-compatible format. However, I need clarification on a few things:

Questions & Challenges:

1. Model Format & Conversion:

• The current model is in .safetensors, but MLX requires NumPy (.npz).
• Could you provide guidance on extracting and converting weights into a format MLX can use?

2. Understanding namo.ve (Vision Encoder?)

• I noticed namo.llm (the language model) and namo.ve (possibly a vision encoder?).
• Could you clarify if NAMO-R1 is a multimodal model? If so, are there specific dependencies required for image inputs?

3. Defining Model Architecture in MLX

• MLX requires an explicit model definition (nn.Module in mlx.nn).
• Do you have details on the exact architecture (hidden sizes, number of layers, activation functions) to help reproduce it in MLX?

Request:
Would it be possible to provide a model architecture description, weight extraction guide, or any hints for making NAMO-R1 run on MLX? I’d love to contribute a conversion script once I get a better understanding!

[lucasjinreal]

Hi, thanks so much for your interest! Our Namo-R1 model currently only supports inference on PyTorch, which can be directly inferred using the namo package.

However, we are doing exactly the same thing as you request these days. For details, we are paving two paths for deploying Namo models. First, we are developing Crane in pure Rust: https://github.com/lucasjinreal/Crane, which supports fast Namo inference on both CPU and GPU (mainly CPU). And second, we might use MLX. However, if Crane is fully supported, MLX might not be really needed.

The deployment is still in progress, and Crane is not only for inference on the Namo series but also supports a wide range of models.

When Namo-R1 v2 is training done, we will push Crane further to support the v2 model (which uses Siglip2 as the vision encoder).

However, in terms of MLX conversion, am not sure if namo package inference can be directly referenced or not. V1 model uses aimv2-large as image encoder with a dynamic input resolution feature.

[yourappleintelligence]

Appreciate it. I also explored Crane, and you’ve done an amazing job there. The reason I’m looking to utilize Namo with MLX is that it would allow me to run it on Apple's platform apps while leveraging Apple’s hardware, potentially boosting performance significantly. Given that Namo is already a strong performer, this could be quite impressive. I did attempt the conversion on my own, but due to Namo’s custom architecture not being directly supported by MLX, it requires additional effort and a deeper understanding of Namo. That’s why I’m reaching out!

[lucasjinreal]

We will release a detailed model arch description in next few days!