Is two-bit quantization only achievable under specific conditions?

[maoshanwen]

I would like to ask how the various hyperparameters of the command relate to the compression effect of the model (for example, compressing it to two bits)

[github-actions]

This issue is stale because it has been open for 30 days with no activity.

[Vahe1994]

Hello!
Sorry for the late reply.

You can refer to Section H in the paper's appendix for the formula to calculate the bitwidth in the AQLM:
https://arxiv.org/pdf/2401.06118

For more details on the parameters, please see:

• How to quantize a model with AQLM (README)
• main.py#L556 – relevant code section