Skip to content

Results don’t match paper #13

@owoakuma

Description

@owoakuma

Hi, excellent work! I’m trying to reproduce the general-knowledge results and I’m seeing discrepancies vs Table 2 / Figure 4 in the paper. I’d appreciate clarification on the exact reproduction settings.
Paper (Continued Pretraining, n=200) reports:
Train on Passage + GPT‑4.1 Synthetic: 39.4
SEAL: 43.8
But the repo outputs (e.g. general-knowledge/results/cpt/200) show:
gpt4_1_200.json adapter_accuracy = 0.594
iter2_200.json adapter_accuracy = 0.582

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions