-
Notifications
You must be signed in to change notification settings - Fork 403
Open
Description
What an excellent work! I have learned so much from this paper.
But I have a question about normalization constant Z_v1 and Z_v2 in the ContrastMemory.
The Z_v1 and Z_v2 is initialized to -1,and Z_v1/Z_v2 is calculated based on out_v1/out_v2.
As shown in the following code, Z_v1/Z_v2 is updated only if Z_v1<0 and Z_v2<0,so Z_v1/Z_v2 will only be updated in the first batch.
if Z_v1 < 0:
self.params[2] = out_v1.mean() * outputSize
Z_v1 = self.params[2].clone().detach().item()
print("normalization constant Z_v1 is set to {:.1f}".format(Z_v1))
if Z_v2 < 0:
self.params[3] = out_v2.mean() * outputSize
Z_v2 = self.params[3].clone().detach().item()
print("normalization constant Z_v2 is set to {:.1f}".format(Z_v2))
But I think Z_v1 and Z_v2 should be updated in every batch.
What is your opinion on this,what are the benefits of this design?
Metadata
Metadata
Assignees
Labels
No labels