Search results

Rouge-L score of pretraining with pt_16h is less than 40

Status: Open.

#47 In PiotrNawrot/nanoT5;

· andrewivan123 opened on May 26, 2025

Documentation

Status: Open.

#46 In PiotrNawrot/nanoT5;

· dmitryshribak opened on Jan 26, 2025

How to change training objective from next token prediction to Masked Language Modeling?

Status: Open.

#45 In PiotrNawrot/nanoT5;

· HaninZeyad opened on Sep 12, 2024

The weird curve

Status: Open.

#42 In PiotrNawrot/nanoT5;

· nguyenvannghiem0312 opened on Aug 18, 2024

How to replace the tokenizer with another one?

Status: Open.

#41 In PiotrNawrot/nanoT5;

· hifarer opened on Aug 9, 2024

nanoT5 initializes lm_head weights with 768x too much variance, probably

Status: Open.

#25 In PiotrNawrot/nanoT5;

· Birch-san opened on Dec 26, 2023