Skip to content

docs(ai): add acceptance test guide and skill#19887

Merged
ajsutton merged 2 commits intodevelopfrom
aj/docs/acceptance-tests
Apr 2, 2026
Merged

docs(ai): add acceptance test guide and skill#19887
ajsutton merged 2 commits intodevelopfrom
aj/docs/acceptance-tests

Conversation

@ajsutton
Copy link
Copy Markdown
Contributor

@ajsutton ajsutton commented Apr 2, 2026

Summary

  • Add docs/ai/acceptance-tests.md — full guide for building deps and running acceptance tests locally
  • Add .claude/skills/run-acceptance-tests/SKILL.md — actionable skill for AI agents
  • Refactor op-acceptance-tests/justfile:
    • Extract build-deps target from acceptance-test
    • Add just test *args target for running individual tests with automatic dep building
    • Remove git submodule update from local builds (can revert in-progress submodule work)
    • Always rebuild cannon prestates instead of conditional skip
  • Reference new doc from AGENTS.md and docs/ai/dev-workflow.md
  • Recommend RUST_JIT_BUILD=1 for local runs so op-reth and other Rust binaries build on demand

Test plan

  • Ran RUST_JIT_BUILD=1 cd op-acceptance-tests && just test -run TestUnsafeGapFillAfterSafeReorg ./op-acceptance-tests/tests/sync/elsync/reorg/ — passed in ~79s
  • Verify just acceptance-test still works for full suite
  • Verify just acceptance-test base works for gated subset

🤖 Generated with Claude Code

Add docs/ai/acceptance-tests.md covering how to build dependencies and
run acceptance tests locally. Add a run-acceptance-tests skill for AI
agents. Refactor op-acceptance-tests/justfile to extract build-deps
target and add a lightweight `just test` target for running individual
tests with automatic dependency building. Remove git submodule update
from local builds to avoid reverting in-progress submodule work.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@ajsutton ajsutton requested a review from a team as a code owner April 2, 2026 01:35
@sebastianst
Copy link
Copy Markdown
Member

I ran the new just target locally to confirm it work, with mise active: RUST_JIT_BUILD=1 just test -run TestUnsafeGapFillAfterSafeReorg ./op-acceptance-tests/tests/sync/elsync/reorg/

Copy link
Copy Markdown
Contributor

@wwared wwared left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be good to ensure some just target or workflow still runs git submodule update --init --recursive --single-branch -j 8 because not all PRs change things in the submodules, and remembering to keep them updated can lead to weird test errors similarly to outdated contract or cannon artifacts. It might be enough to add a new just target update-deps (or similar) that does this and mentioning it in the ai doc/skill

Slim down the acceptance test skill to a thin wrapper that @-includes the
doc. Add `mise exec --` prefix to all just commands since AI agent shells
don't have mise activated. Document mise setup in dev-workflow.md.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@ajsutton
Copy link
Copy Markdown
Contributor Author

ajsutton commented Apr 2, 2026

We don't generally have just targets to do git updates and I think updating submodules belongs with that - it's just done as part of your git workflow. If the submodules aren't up to date, it shows up as the git checkout not being clean so AI should sort that out itself and humans usually can as well.

@ajsutton ajsutton enabled auto-merge April 2, 2026 21:35
@ajsutton ajsutton added this pull request to the merge queue Apr 2, 2026
Merged via the queue into develop with commit 0053287 Apr 2, 2026
108 checks passed
@ajsutton ajsutton deleted the aj/docs/acceptance-tests branch April 2, 2026 22:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants