How to fix "get_loading_attributes" bug? #4
Description
(semikong-env) XXXXXX@ytai:~/projects/llm_projects/semikong$ python training.py --bf16
Your GPU supports bfloat16, you can accelerate training with the argument --bf16
Traceback (most recent call last):
File "/home/zhaofei/projects/llm_projects/semikong/training.py", line 149, in
MODEL, TOKENIZER, PEFT_CONFIG = load_model(model_name)
^^^^^^^^^^^^^^^^^^^^^^
File "/home/zhaofei/projects/llm_projects/semikong/training.py", line 110, in load_model
model = AutoModelForCausalLM.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/zhaofei/.conda/envs/semikong-env/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 564, in from_pretrained
return model_class.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/zhaofei/.conda/envs/semikong-env/lib/python3.11/site-packages/transformers/modeling_utils.py", line 3344, in from_pretrained
config.quantization_config = AutoHfQuantizer.merge_quantization_configs(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/zhaofei/.conda/envs/semikong-env/lib/python3.11/site-packages/transformers/quantizers/auto.py", line 168, in merge_quantization_configs
loading_attr_dict = quantization_config_from_args.get_loading_attributes()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'BitsAndBytesConfig' object has no attribute 'get_loading_attributes'