some question about the white box attack

[yuese1234]

Sorry to bother you again.
When I run your white-box attack code for LLaVA, the gradient is None. The specific problem is:
adv_noise.data = (adv_noise.data - alpha * adv_noise.grad.detach().sign()).clamp(0, 1)
AttributeError: 'NoneType' object has no attribute 'detach'. Can you give me some help?

[pyogher]

Hi @yuese1234, could you please provide your versions of torch and transformers? Alternatively, you can use transformers 4.36.2 and torch 2.1.0.

[yuese1234]

Hi @yuese1234, could you please provide your versions of torch and transformers? Alternatively, you can use transformers 4.36.2 and torch 2.1.0.

I use the transformers 4.37.2 and torch 2.1.2。I found that in the VIT of CLIP, the gradient disappeared, but I rewrote a piece of code and loaded the VIT separately, and the gradient existed.

[pyogher]

Hi @yuese1234, could you please provide your versions of torch and transformers? Alternatively, you can use transformers 4.36.2 and torch 2.1.0.

I use the transformers 4.37.2 and torch 2.1.2。I found that in the VIT of CLIP, the gradient disappeared, but I rewrote a piece of code and loaded the VIT separately, and the gradient existed.

Ok, we will update the version requirements in the README later. Please let us know if you have any other questions.

[yuese1234]

Hi @yuese1234, could you please provide your versions of torch and transformers? Alternatively, you can use transformers 4.36.2 and torch 2.1.0.

I use the transformers 4.37.2 and torch 2.1.2。I found that in the VIT of CLIP, the gradient disappeared, but I rewrote a piece of code and loaded the VIT separately, and the gradient existed.

Ok, we will update the version requirements in the README later. Please let us know if you have any other questions.

I just try transformers 4.36.2 and torch 2.1.0. the gradient is still None. As the follows:
images.requires_grad=True
image_features = self.encode_images(images)
print(f'image_features:{image_features.requires_grad}') the result is False . so crazy！

[TopEric]

I met the same problem, have you solved it?