Megatron support #1
Description
🚀 The feature, motivation and pitch
The current Olmo is trained with fsdp but we want to reproduce the results with Megatron.
Alternatives
No response
Additional context
No response
{{ message }}
The current Olmo is trained with fsdp but we want to reproduce the results with Megatron.
No response
No response