From cfe4a3c702ca222a78bbe7d0e4892ddf9d4e02d2 Mon Sep 17 00:00:00 2001 From: koshieguchi Date: Mon, 5 May 2025 16:25:28 +0900 Subject: [PATCH] docs: fix minor typo in README --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 4c3e022c8..7b8d48aba 100644 --- a/README.md +++ b/README.md @@ -95,7 +95,7 @@ python scripts/train.py configs/tiny/OLMo-20M.yaml --save_overwrite ### Stage 1 -Stage 1 is the biggest stage, where we train on 4T or 5T tokens on largely web-based data. +Stage 1 is the biggest stage, where we train on 4T or 5T tokens on largely web-based data. | | OLMo2 1B | OLMo2 7B | OLMo2 13B | |-----------------|-----------------|-------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------| @@ -111,7 +111,7 @@ For the 1B model, we have trained three times with different data order on 50B h | | Checkpoint | Training config | WandB | |------------------------|-------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------|-------------| -| random seed 42049 | [stage2-ingredient1-step23852-tokens50B](https://huggingface.co/allenai/OLMo-2-0425-1B/tree/stage2-ingredient1-step23852-tokens51B) | [OLMo2-1B-stage2-seed42069.yaml](configs/official-0425/OLMo2-1B-stage2-seed42069.yaml) | [wandb.ai/OLMo2-1B](https://api.wandb.ai/links/ai2-llm/izdtrtu0) | +| random seed 42069 | [stage2-ingredient1-step23852-tokens50B](https://huggingface.co/allenai/OLMo-2-0425-1B/tree/stage2-ingredient1-step23852-tokens51B) | [OLMo2-1B-stage2-seed42069.yaml](configs/official-0425/OLMo2-1B-stage2-seed42069.yaml) | [wandb.ai/OLMo2-1B](https://api.wandb.ai/links/ai2-llm/izdtrtu0) | | random seed 666 | [stage2-ingredient2-step23852-tokens50B](https://huggingface.co/allenai/OLMo-2-0425-1B/tree/stage2-ingredient2-step23852-tokens51B) | [OLMo2-1B-stage2-seed666.yaml](configs/official-0425/OLMo2-1B-stage2-seed666.yaml) | [wandb.ai/OLMo2-1B](https://api.wandb.ai/links/ai2-llm/izdtrtu0) | | random seed 42 | [stage2-ingredient3-step23852-tokens50B](https://huggingface.co/allenai/OLMo-2-0425-1B/tree/stage2-ingredient3-step23852-tokens51B) | [OLMo2-1B-stage2-seed42.yaml](configs/official-0425/OLMo2-1B-stage2-seed42.yaml) | [wandb.ai/OLMo2-1B](https://api.wandb.ai/links/ai2-llm/izdtrtu0) | | **final souped model** | [main](https://huggingface.co/allenai/OLMo-2-0425-1B/tree/main) | no config, we just averaged the weights in Python | @@ -235,12 +235,12 @@ where `body.json` is of the form: ```bibtex @misc{olmo20242olmo2furious, - title={2 OLMo 2 Furious}, + title={2 OLMo 2 Furious}, author={Team OLMo and Pete Walsh and Luca Soldaini and Dirk Groeneveld and Kyle Lo and Shane Arora and Akshita Bhagia and Yuling Gu and Shengyi Huang and Matt Jordan and Nathan Lambert and Dustin Schwenk and Oyvind Tafjord and Taira Anderson and David Atkinson and Faeze Brahman and Christopher Clark and Pradeep Dasigi and Nouha Dziri and Michal Guerquin and Hamish Ivison and Pang Wei Koh and Jiacheng Liu and Saumya Malik and William Merrill and Lester James V. Miranda and Jacob Morrison and Tyler Murray and Crystal Nam and Valentina Pyatkin and Aman Rangapur and Michael Schmitz and Sam Skjonsberg and David Wadden and Christopher Wilhelm and Michael Wilson and Luke Zettlemoyer and Ali Farhadi and Noah A. Smith and Hannaneh Hajishirzi}, year={2024}, eprint={2501.00656}, archivePrefix={arXiv}, primaryClass={cs.CL}, - url={https://arxiv.org/abs/2501.00656}, + url={https://arxiv.org/abs/2501.00656}, } ```