Skip to content

Implement "sanity checks" training runs #20

@AmitMY

Description

@AmitMY

We should implement a few training configuration files to train small models to do:

  1. String repetition (--dataset_text_template "<text> {text} <again> {text}")
  2. String repetition + (--bytes_encoder_model_name_or_path None)
  3. Word deconstructions ("Strawberry S1 T1 R3 A1 W1 B1 E1 Y1") - lots of words, maybe only english

and evaluate actually in generation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions