Adding dinov3 collection of vit models

[coder0143]

Resolves #98

Reference

• Huggingface implementation: https://github.com/huggingface/transformers/blob/main/src/transformers/models/dinov3_vit/modular_dinov3_vit.py

Checklist

• I have read the Contribution Guidelines and used pre-commit hooks to format this commit.
• I have added all the necessary unit tests for my change. (run_model.py for model usage, test_outputs.py and/or model_validation_colab.ipynb for quality).
• (If using an LLM) I have carefully reviewed and removed all superfluous comments or unneeded, commented-out code. Only necessary and functional code remains.
• I have signed the Contributor License Agreement (CLA).

[coder0143]

Can you please review my PR @chapman20j

[chapman20j]

Hi @coder0143. Thanks for the nice PR! I left a few comments. Having explicit configs here can help make it more clear what hyperparameters are used in constructing the model and could simplify some parts of the code. Also, including more testing ensures model correctness. Looking forward to the final version!

[coder0143]

Thankyou so much for reviewing and replying @chapman20j , I have made the following changes:

• Written custom configs for loading the specific models (modeling.py)
• Removed transformers dependency from modeling and params
• Removed cosine similarity and added more tests to the test_outputs file
• Updated the colab notebook

[jenriver]

Hi, could you ensure that the tests above are passing? i.e.

1. Please ensure you have run pre-commit run --all-files as in contribution guidelines.
2. The CI is currently failing with a GatedRepoError because the dinov3 checkpoint is restricted. Could you update the test to use randomly initialized weights instead? (Note: Please ensure the JAX and PyTorch models are initialized with the same random weights so the parity assertions still pass. Creating a random PyTorch model and converting it to JAX within setUp usually works best.)

[coder0143]

Thankyou for reviewing @jenriver , I have made the following changes:

• Checked and updated files based on ruff formatting.
• Removed run_model.py file, instead colab notebook can be used.
• Updated test_outputs.py file to use a randomly initialized model and run tests based on that, using the vit_b16 model
• the pre-commit command runs fine

[jenriver]

Hi, we're still seeing pre-commit failures as above -- could you ensure you have run pre-commit hooks?

i.e.
pre-commit run --all-files as in contribution guidelines.

[jenriver]

Could you not use local directory paths?

i.e. Something like

self.save_dir = snapshot_download(...)

[coder0143]

I have made the necessary changes and everything will pass now, thankyou for reviewing and guiding my pr @chapman20j and @jenriver, btw I have sent connect request on LinkedIn!

[jenriver]

This is quite a high tolerance. If RoPE casting and LayerNorms are correctly aligned, we should be seeing a value much tighter than this.

[coder0143]

Yeah, actually things are working just fine, I just updated the atol values and tested it many times, for first layer, setting atol to 2e-3 is ok but in a very worse case, we get atol as 0.0024 max, I have also updated and tested for other output functions. Actually pytorch casts to bfloat16 for RoPE calculation and then casts it back to float32, I have also done the same in jax, but mostly this is where error is introduced, other than some other fp operations