adapt fastmoe into deepspeed moe#1

Open

KimmiShi wants to merge 6 commits intomasterfrom

Commits on Jun 26, 2023

Commits on Jun 28, 2023

update fmoe expert def
KimmiShi
committed

Commits on Jun 29, 2023

add kwargs for gate
KimmiShi
committed

Commits on Jul 4, 2023

update fmoe args
KimmiShi
committed

Commits on Jul 6, 2023

try to use fstermoe,failed
KimmiShi
committed