-
Notifications
You must be signed in to change notification settings - Fork 7
Open
Description
While the idea of this library is to write your own training script, suited to your needs, it would still be good to have example scripts for single-GPU and multi-GPU training. Currently, we have one script per experiment, but this causes a lot of code duplication. The same is true for eval scripts.
Ideally, we should have one eval script, one SFT script and 2 training scripts (one for single-GPU training and one for FSDP2).
One idea to make this work is to build a registry that maps strings to classes, such that general factory methods can be used with a training / eval config file that specifies the details.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels