Compatibility with Qwen models

[AIpplier]

Hi,

I just wanted to confirm whether AQLM’s quantization methods can be applied to Qwen models, since they are part of the LLaMA-family architecture. Are there any known limitations or additional steps required for using AQLM with Qwen checkpoints?

Thanks!

[Godofnothing]

Hi, @AIpplier.

Llama and Qwen are very similar in terms of architecture, and it is likely that the code script working for Llama would work for Qwen without significant changes.

The potential cause of issues could be changes of Llama/Qwen implementations in more recent transformers versions. Specfically, RoPE is subject to major changes.