How to use the code for ADI?

[theAverageArchit]

I am having trouble figuring out how to use the code for performing ADI. What are the settings that we need to set for performing ADI?

[tallesbrito]

Hi @theAverageArchit ,

I found that in the ImageNet implementation in this repository, you can set --adi_scale to non-zero (e.g. --adi_scale=0.2). This option activates a pre-trained Resnet18 as a student and activates the ADI loss computation. In this case, the ResNet18 weights are fixed (there is no backpropagated gradient to the ResNet18).

However, it seems to me that this is different from what is described in the paper: in the paper, the student model is initialized from scratch and is modified by gradients.

Can anyone explain this?

After all, is there a full implementation of ADI in this repository? Wouldn't I need to use knowledge distillation from teacher to student in order to enable ADI, as described in the paper?

[unistdJRZ]

Hi @theAverageArchit ,

I found that in the ImageNet implementation in this repository, you can set --adi_scale to non-zero (e.g. --adi_scale=0.2). This option activates a pre-trained Resnet18 as a student and activates the ADI loss computation. In this case, the ResNet18 weights are fixed (there is no backpropagated gradient to the ResNet18).

However, it seems to me that this is different from what is described in the paper: in the paper, the student model is initialized from scratch and is modified by gradients.

Can anyone explain this?

After all, is there a full implementation of ADI in this repository? Wouldn't I need to use knowledge distillation from teacher to student in order to enable ADI, as described in the paper?

I'm fully support what u saying, there is no graident update for student model, which means no-way to progressively generate new samples, which different from paper.