This repository was archived by the owner on Oct 31, 2023. It is now read-only.
This repository was archived by the owner on Oct 31, 2023. It is now read-only.
About the projection and prediction head dimension configs #39
Description
Hi,
Thank you for the amazing works and I am inspired a lot from it!
One thing I want to ask is how you designed the structure of your projection and predications heads, i.e., how to decide the number of layers and hidden and output dimensions? P.S. I notice that you have already mentioned the bottleneck structure of prediction MLP is helpful.
Thank you for the helps and time!