he calculation of the compression ratio

[maoshanwen]

Hi, I would like to ask about the calculation of the compression ratio. Through experimentation, we found that the compression ratio for each layer is as follows;

however, according to your parameter settings, it seems the result is not a 2-bit compression.

python main.py /home/wangsf/maosw/used/model/Llama-3.1-8B wikitext2 --nsamples=2048 --num_codebooks=1 --nbits_per_codebook=15 --in_group_size=8 --relative_mse_tolerance=0.01 --finetune_lr=1e-4 --finetune_adam_beta1=0.90 --finetune_adam_beta2=0.95 --finetune_keep_best --finetune_batch_size=32 --local_batch_size=4 --offload_activations

[github-actions]

This issue is stale because it has been open for 30 days with no activity.

[Vahe1994]

Hello!
Sorry for the late answer. Please see formula (10) in appendix H https://arxiv.org/pdf/2401.06118 .
What are you compressing exactly? Why is your index nm/g r * 32 ? What is r and why are you storing index in 32 bits?

With best