Was Whitening implemented incorrectly?

[jsrdcht]

whitening operation, which is implemented by a non-parametric layer normalization operator without scaling and bias

You mentioned that the whitening operation is non-parametric. But it seems you implemented it by norm operation from the original paper which is not non-parametric.

if self.feat_after_norm:
      if 'CLIP' in self.pred_feat:
          x_tgt = self.feature_model.visual.ln_post(x_tgt)

[yaoyuan10475]

I found that the author's teacher model output has gone through a norm(self.feature_model.norm(x_tgt)), and then there is self.ln_tgt(x_tgt). The output equivalent to the teacher model has gone through Layer Norm twice. I don't quite understand this.