does it support megatron+deepspeed? #9
Description
Can I combine megatron with deepspeed(like zero1 or zero2)?
BTW, can I use this repo to pretrain 65B Llama?
{{ message }}
Can I combine megatron with deepspeed(like zero1 or zero2)?
BTW, can I use this repo to pretrain 65B Llama?