Question about the Kinetix evaluation

[tannq-jo]

Hi, thank you very much for your great work. It has truly opened my mind, and I really appreciate it!

By the way, I wanted to ask whether you have any plans to release the evaluation code for Kinetics using VLASH for the MLP-Mixer. It would be extremely helpful for my work.

Thank you again.

[Sakits]

Hi @tannq-jo ,

Thank you so much for your interest in our work! We’re really glad to hear that the work has been helpful.

We are actively working on a separate branch dedicated to simulation benchmarks, including Kinetix, where the async inference latency and accuracy results reported in the paper can be fully reproduced. We will release this simulation code in an upcoming update. Please stay tuned! :)

Thank you!

[tannq-jo]

Hi @Sakits,

Thanks so much for your reply! In the meantime, I’ll keep working on reproducing your VLASH results for the Kinetix evaluation. I just had one quick question, how did you implement “Efficient Fine-tuning with Shared Observation” in the Kinetix setting, since the MLP-Mixer doesn’t have any attention layer?

Thank you!

[Sakits]

Hi @tannq-jo ,

Thanks for the question!

We did not use "Efficient Fine-tuning with Shared Observation" when training the model for the Kinetix evaluation. This technique is purely an efficiency optimization and does not affect the training objective or final performance, so it was omitted in the Kinetix experiments.

Please feel free to reach out if anything is unclear! :)

Best,
Jiaming

[Sakits]

Hi @tannq-jo ,

We just released the Kinetix benchmark code and checkpoints at: https://github.com/mit-han-lab/vlash/tree/sim/kinetix/benchmarks.

While it’s not yet seamlessly integrated into the VLASH codebase, it can be used to reproduce the results reported in the paper. Please feel free to check it out, and please let us know if you run into any issues! :)