The type of past_seen_tokens is int,but it neet cache

[a1ei]

The code in llama2_split line 69, the origin code is
causal_mask = self._update_causal_mask(attention_mask, inputs_embeds, cache_position, past_seen_tokens)
there are two questions first is there neet a parameter output_attentions,what's more is the type of past_seen_tokens is int but it should be cache.
这里是作者漏掉了吗

[StupidTrees]

This should be due to the version difference of the transformers library.

[a1ei]

OK，but I try to use your version of transformers which is 4.40.0 and that cause "ImportError: cannot import name 'EncoderDecoderCache' from 'transformers' " so I update transformers to 4.48.0 but the decoder of it neet input （position_embedding） but yours is all position_ids .
Can you tell me your version of transformers, thank you

[StupidTrees]

try transformers==4.44.0

[a1ei]

hello When I try dlgattack(sma)，after calculating the grad （code is grad = torch.autograd.grad(loss, inter, create_graph=True)） It 'shows that RuntimeError: derivative for aten::_scaled_dot_product_efficient_attention_backward is not implemented which means it does not support differential operations.Is your pytorch version is 2.3.0?Thrank you for your reply

[a1ei]

I use llamma2 first and I add a line of code "#-----------------修改的地方create position embeddings position_embeddings = self.rotary_emb(hidden_states, position_ids) #--------------------------------------------------------- " in llamma2_split.py(forward function).But I also try other model like t5 or chatglm I think the reason why it occured is beacuse the version of transformers，try different version of it.For example ,you can see the code of llamma in official transformers and the code should be the same of model_split.py and change it will be ok. lei ***@***.***  

…

------------------ 原始邮件 ------------------ 发件人: "StupidTrees/SplitLLM" ***@***.***>; 发送时间: 2025年4月4日(星期五) 下午4:49 ***@***.***>; ***@***.******@***.***>; 主题: Re: [StupidTrees/SplitLLM] The type of past_seen_tokens is int,but it neet cache (Issue #3) OK，but I try to use your version of transformers which is 4.40.0 and that cause "ImportError: cannot import name 'EncoderDecoderCache' from 'transformers' " so I update transformers to 4.48.0 but the decoder of it neet input （position_embedding） but yours is all position_ids . Can you tell me your version of transformers, thank you Hello, when I was running this project, I also encountered a situation where position_embedding was None during the wandb stage. Could you please tell me how you solved this problem? Thank you for your reply. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***> lbc1355 left a comment (StupidTrees/SplitLLM#3) OK，but I try to use your version of transformers which is 4.40.0 and that cause "ImportError: cannot import name 'EncoderDecoderCache' from 'transformers' " so I update transformers to 4.48.0 but the decoder of it neet input （position_embedding） but yours is all position_ids . Can you tell me your version of transformers, thank you Hello, when I was running this project, I also encountered a situation where position_embedding was None during the wandb stage. Could you please tell me how you solved this problem? Thank you for your reply. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>