Skip to content

docs: add experiment strategy tips to program.md#187

Open
tykoo-chen wants to merge 1 commit intokarpathy:masterfrom
tykoo-chen:docs/experiment-strategy
Open

docs: add experiment strategy tips to program.md#187
tykoo-chen wants to merge 1 commit intokarpathy:masterfrom
tykoo-chen:docs/experiment-strategy

Conversation

@tykoo-chen
Copy link

Problem

AI agents running autoresearch can get stuck or waste experiments on low-impact changes. The current program.md tells agents what to do but not how to do it effectively.

Solution

Add an "Experiment Strategy Tips" section with guidance on:

  • High-impact changes to prioritize (LR, model size, batch size)
  • Common pitfalls to avoid (changing too many things, abandoning ideas too early)
  • When stuck strategies (revert, combine ideas, remove complexity)
  • Signs of a good experiment (clear hypothesis, minimal change, interpretable)

This should help agents be more efficient researchers.

Guidance for AI agents on effective experimentation:
- High-impact changes to try first
- Common pitfalls to avoid
- What to do when stuck
- Signs of a good experiment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant