-
Notifications
You must be signed in to change notification settings - Fork 298
Open
Description
Hi, excellent work! I’m trying to reproduce the general-knowledge results and I’m seeing discrepancies vs Table 2 / Figure 4 in the paper. I’d appreciate clarification on the exact reproduction settings.
Paper (Continued Pretraining, n=200) reports:
Train on Passage + GPT‑4.1 Synthetic: 39.4
SEAL: 43.8
But the repo outputs (e.g. general-knowledge/results/cpt/200) show:
gpt4_1_200.json adapter_accuracy = 0.594
iter2_200.json adapter_accuracy = 0.582
Metadata
Metadata
Assignees
Labels
No labels