DxD instead of NxN in the computation of the covariance matrix of Z?

[aarontyliu]

Hi, I came across the implementation of the covariance matrix of Z while trying to use VICReg in my project. I feel like I am missing something here, but I wonder if the implementation should be x @ x.T instead of x.T @ x?

Thanks for any information you can provide.

[steinate]

Same quetions. Did you figure out? I also think the former implementation is right.

[hfangcat]

The implementation should be x.T @ x, if x's dimension is (B, D), here B: batch_size, D: dimension, because they want the covariance matrix to be a function of the dimension