Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .claude/skills/run-acceptance-tests/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
name: run-acceptance-tests
description: Build dependencies and run acceptance tests locally in the Optimism monorepo. Handles contracts, cannon prestates, Rust binaries, and optional kona prestate.
---

# Run Acceptance Tests

## When to Use

- Before merging changes that affect devnet behavior
- When CI acceptance tests fail and you need to reproduce locally
- When asked to verify a change works end-to-end

## Guide

@docs/ai/acceptance-tests.md
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ More detailed guidance for AI agents can be found in:
- [docs/ai/contract-dev.md](docs/ai/contract-dev.md) - Smart contract development
- [docs/ai/go-dev.md](docs/ai/go-dev.md) - Go service development
- [docs/ai/rust-dev.md](docs/ai/rust-dev.md) - Rust development (kona, op-reth, alloy crates)
- [docs/ai/acceptance-tests.md](docs/ai/acceptance-tests.md) - Building and running acceptance tests locally

## External References

Expand Down
99 changes: 99 additions & 0 deletions docs/ai/acceptance-tests.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# Acceptance Tests

Guidance for AI agents building and running acceptance tests in the Optimism monorepo. See [dev-workflow.md](dev-workflow.md) for tool versions and general workflow.

## What Are Acceptance Tests?

Acceptance tests live in `op-acceptance-tests/tests/` and run full in-process devnet scenarios. They exercise the entire stack — contracts, Go services, and Rust binaries — in a single `go test` process. This means all dependencies must be built locally before running them.

## Running Tests

All `just` targets below automatically build dependencies (contracts, cannon prestates, Rust binaries) before running tests. The builds are incremental — re-running is fast when nothing changed.

**Prerequisites:** mise tools must be installed (see [dev-workflow.md](dev-workflow.md#setup)), and a working C toolchain (`clang` or `gcc`) must be available for Rust builds.

Always set `RUST_JIT_BUILD=1` when running locally. This lets the test framework automatically build any Rust binaries it needs (e.g. op-reth) using cargo's rebuild detection, so you don't have to pre-build them separately.

Run from `op-acceptance-tests/`:

### Specific Tests or Packages (recommended)

```bash
# Run a single test
RUST_JIT_BUILD=1 cd op-acceptance-tests && mise exec -- just test -run TestMyTest ./op-acceptance-tests/tests/base/

# Run a package
RUST_JIT_BUILD=1 cd op-acceptance-tests && mise exec -- just test ./op-acceptance-tests/tests/base/...
```

The `just test` target builds deps, then runs `go test -count=1 -timeout 30m` with your arguments.

### All Tests

```bash
RUST_JIT_BUILD=1 cd op-acceptance-tests && mise exec -- just acceptance-test
```

Runs all test packages with gotestsum, structured logging, and auto-tuned parallelism.

### Gated Subsets

Gate files in `op-acceptance-tests/gates/` list package subsets:

```bash
RUST_JIT_BUILD=1 cd op-acceptance-tests && mise exec -- just acceptance-test base
```

This runs only packages listed in `gates/base.txt`.

### Kona Reproducible Prestate

Some tests (e.g. superfaultproofs, interop fault proofs) require a reproducible kona prestate. This is **not** handled by `build-deps` or `RUST_JIT_BUILD`:

```bash
mise exec -- just reproducible-prestate-kona
```

**Requires Docker.** If Docker is not available in your environment, ask the user to run this command for you.

## What `build-deps` Does

The `just build-deps` target (called automatically by `just test` and `just acceptance-test`) runs these steps when not in CI:

1. **mise**`mise install` (ensures gotestsum, forge, etc. are available)
2. **Contracts**`cd packages/contracts-bedrock && just install && just build-no-tests`
3. **Cannon prestates**`just cannon-prestates` (builds cannon, op-program, and prestate artifacts)
4. **Rust binaries**`just build-rust-release` (kona-node, op-rbuilder, rollup-boost)

You can also run `just build-deps` directly to pre-build without running tests.

## Tuning Parallelism (`acceptance-test` only)

When using `just acceptance-test`, the runner auto-detects CPU count and sets:
- `ACCEPTANCE_TEST_JOBS` — number of packages to test in parallel (default: CPU count)
- `ACCEPTANCE_TEST_PARALLEL``go test -parallel` value per package (default: CPU count / 2)
- `ACCEPTANCE_TEST_TIMEOUT` — per-package timeout (default: 2h)

Override with environment variables:

```bash
ACCEPTANCE_TEST_PARALLEL=2 ACCEPTANCE_TEST_TIMEOUT=1h cd op-acceptance-tests && mise exec -- just acceptance-test
```

## Log Output (`acceptance-test` only)

When using `just acceptance-test`, logs are written to `op-acceptance-tests/logs/testrun-<timestamp>/`:
- `all.log` — full test output
- `raw_go_events.log` — JSON test events
- `flaky-tests.txt` — tests marked with `MarkFlaky()`

Results XML goes to `op-acceptance-tests/results/results.xml`.

When using `just test`, output goes to stdout only.

## Common Issues

- **Missing prestates** — Run `cd op-acceptance-tests && mise exec -- just build-deps` or `mise exec -- just cannon-prestates` from the repo root.
- **Stale contracts** — Rebuild with `cd packages/contracts-bedrock && mise exec -- just build-no-tests`.
- **Missing Rust binaries** — Run `mise exec -- just build-rust-release` from the repo root.
- **gotestsum not found** — Run `mise install` to install all pinned tools.
10 changes: 9 additions & 1 deletion docs/ai/dev-workflow.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,21 @@
# Development Workflow

Common workflow guidance for AI agents working in the Optimism monorepo. Language-specific details are in [go-dev.md](go-dev.md) and [rust-dev.md](rust-dev.md).
Common workflow guidance for AI agents working in the Optimism monorepo. Language-specific details are in [go-dev.md](go-dev.md) and [rust-dev.md](rust-dev.md). For end-to-end testing, see [acceptance-tests.md](acceptance-tests.md).

## Tool Versions

All tool versions are pinned in `mise.toml` at the repo root. Always access tools through mise — never install or invoke system-global versions directly. Check `mise.toml` for current pinned versions when you need to know what's available.

If mise reports the repo isn't trusted, ask the user to run `mise trust` — never trust it automatically.

### Setup

Run `mise install` to install all pinned tools (just, gotestsum, forge, etc.). AI agent shells typically do not have mise activated, so prefix commands with `mise exec --` to ensure tools are on `PATH`:

```bash
mise exec -- just <target>
```

## Build System

The repo uses [Just](https://github.com/casey/just) as its build system. Shared justfile infrastructure lives in `justfiles/`. Each component has its own justfile — run `just --list` in any directory to see available targets.
Expand Down
62 changes: 33 additions & 29 deletions op-acceptance-tests/justfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,40 @@ REPO_ROOT := `realpath ..` # path to the root of the optimism monorepo
default:
@just acceptance-test

# Build all dependencies for local acceptance test runs. Skipped in CI where jobs handle this.
build-deps:
#!/usr/bin/env bash
set -euo pipefail
if [[ -n "${CIRCLECI:-}" ]]; then exit 0; fi

echo "Building contracts (local build)..."
cd {{REPO_ROOT}}
echo " - Installing mise..."
mise install
cd packages/contracts-bedrock
echo " - Installing contracts..."
just install
echo " - Forge build..."
just build-no-tests
cd {{REPO_ROOT}}

echo "Building cannon dependencies..."
cd {{REPO_ROOT}}
just cannon-prestates

echo "Building Rust binaries (kona-node, op-rbuilder, rollup-boost)..."
cd {{REPO_ROOT}}
just build-rust-release

# Run specific acceptance tests. Builds dependencies first, then passes all arguments to go test.
# Examples:
# just test ./op-acceptance-tests/tests/base/...
# just test -run TestMyTest ./op-acceptance-tests/tests/base/
test *args: build-deps
cd {{REPO_ROOT}} && go test -count=1 -timeout 30m {{args}}

# Run acceptance tests, optionally filtered by a gate (reads packages from gates/<gate>.txt)
acceptance-test gate="":
acceptance-test gate="": build-deps
#!/usr/bin/env bash
set -euo pipefail
GATE="{{gate}}"
Expand All @@ -15,34 +47,6 @@ acceptance-test gate="":
echo -e "DEVNET: in-memory, MODE: all tests\n"
fi

# Build dependencies for in-process mode if not in CI.
# In CI jobs already take care of this, so we skip it.
if [[ -z "${CIRCLECI:-}" ]]; then
echo "Building contracts (local build)..."
cd {{REPO_ROOT}}
echo " - Updating submodules..."
git submodule update --init --recursive --single-branch -j 8
echo " - Installing mise..."
mise install
cd packages/contracts-bedrock
echo " - Installing contracts..."
just install
echo " - Forge build..."
just build-no-tests
cd {{REPO_ROOT}}

echo "Checking cannon dependencies..."
if [ ! -e {{REPO_ROOT}}/cannon/bin/cannon ] || [ ! -e {{REPO_ROOT}}/op-program/bin/prestate-mt64.bin.gz ]; then
echo "Building cannon dependencies..."
cd {{REPO_ROOT}}
make cannon-prestates
fi

echo "Building Rust binaries (kona-node, op-rbuilder, rollup-boost)..."
cd {{REPO_ROOT}}
just build-rust-release
fi

LOG_LEVEL="$(echo "${LOG_LEVEL:-info}" | grep -E '^(debug|info|warn|error)$' || echo 'info')"
echo "LOG_LEVEL: $LOG_LEVEL"

Expand Down
Loading