Problem with lmms-eval: Code modification instructions don't match my conda env file #47
Description
Thank you for your great work. I'm trying to integrate FastV with lmms-eval, but I've encountered a problem with the code modifications mentioned in the README. The instructions say to modify three instances of cos, sin = self.rotary_emb(value_states, seq_len=kv_seq_len) in the conda environment's transformers/models/llama/modeling_llama.py file.
However, I've discovered that my conda environment's version of this file doesn't contain these exact lines. I can only find
cos, sin = position_embeddings
query_states, key_states = apply_rotary_pos_emb(query_states, key_states, cos, sin)
in /.conda/envs/fastv/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py.
Could you share your solution of this problem? Thanks in advance!