-
Notifications
You must be signed in to change notification settings - Fork 3.9k
feat(nuts): add update-nuts and verify-nuts commands #19463
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
maurelian
wants to merge
31
commits into
develop
Choose a base branch
from
feat/nut-update-verify
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
31 commits
Select commit
Hold shift + click to select a range
4bee640
feat(nuts): add update-nuts command
maurelian 81801b5
fix(nuts): remove bundle generation from update-nuts
maurelian 7e8b858
docs(nuts): add README documenting NUT bundle workflow
maurelian c56b564
Simplify README
maurelian ef2621f
Apply suggestions from code review
maurelian 4bf25a6
refactor(nuts): extract shared ForkLockEntry into op-core/nuts package
maurelian ca33700
fix(nuts): restrict bundle file permissions to 0600
maurelian ad0a8d7
feat(nuts): add fork locking mechanism
maurelian c9b6483
refactor(nuts): rename scripts for consistency
maurelian 4dfa9db
fix(nuts): regenerate karst bundle and populate commit field
maurelian b75a8ad
revert(nuts): keep check-nut-locks name to minimize diff
maurelian 5908e9f
docs(nuts): remove contracts-bedrock concerns from README
maurelian 63acba7
refactor(nuts): remove locking mechanism
maurelian 9a2f3bc
fix(nuts): update lock file comments and add provenance CI
maurelian 6a92a23
fix(ci): only verify provenance for changed forks
maurelian 542ebff
docs(ci): add comment explaining provenance verify logic
maurelian 84910a0
fix(nuts): move reviewer comment to bottom of fork_lock.toml
maurelian 5c9c41f
fix: move part of comment to top
maurelian 0c6b670
fix(nuts): verify lock file commits exist in git history
maurelian ace326a
broken demo: show that a manually edited bundle breaks check-nut-locks
maurelian bc967a9
Revert "broken demo: show that a manually edited bundle breaks check-…
maurelian debc9ef
broken demo: show that a manually edited lock commit breaks check-nut…
maurelian b7b76f5
Revert "broken demo: show that a manually edited lock commit breaks c…
maurelian 85745bc
docs(op-core/nuts): clarify that just nut-provenance-verify does not …
maurelian d173d60
fix(nuts): record merge-base commit to survive squash-merge
maurelian faccb27
docs(nuts): rearrange lock file comments, document merge-base workflow
maurelian a498641
refactor(ci): extract provenance verify logic to bash script
maurelian 6afd773
test(nuts): add tests for check-nut-locks validation
maurelian b7b7c7c
test(nuts): add tests for nut-provenance-verify
maurelian 15c22b2
docs(ctb): warn against generate-nut-bundle rename
maurelian 0da7f52
fix(nuts): fix goimports formatting in provenance test
maurelian File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| # NUT Bundles | ||
|
|
||
| Network Upgrade Transaction (NUT) bundles define the L2 deposit transactions that activate a hardfork. Each bundle is a JSON file containing ordered transactions (implementation deployments, proxy upgrades, etc.) that the rollup node embeds and executes at the fork activation block. | ||
|
|
||
| ## Files | ||
|
|
||
| | File | Purpose | | ||
| |------|---------| | ||
| | `fork_lock.toml` | Lock file mapping fork names to bundle paths, sha256 hashes, and source commits | | ||
| | `op-node/rollup/derive/<fork>_nut_bundle.json` | Embedded bundle consumed by op-node at fork activation | | ||
|
|
||
| ## Workflow | ||
|
|
||
| ### Generating a bundle | ||
|
|
||
| ```bash | ||
| cd packages/contracts-bedrock | ||
| just generate-nut-bundle | ||
| ``` | ||
|
|
||
| ### Snapshotting a bundle for a fork | ||
|
|
||
| ```bash | ||
| just nut-snapshot-for <fork> | ||
| ``` | ||
|
|
||
| This copies `current-upgrade-bundle.json` to `op-node/rollup/derive/<fork>_nut_bundle.json` and updates `fork_lock.toml` with the sha256 hash and the merge-base commit with `origin/develop`. | ||
|
|
||
| **Important:** The recorded commit is the merge-base with develop, not HEAD. This ensures the commit survives squash-merge. Contract changes must be merged to develop in a separate PR *before* snapshotting the bundle. | ||
|
|
||
|
|
||
| ### Verifying a bundle | ||
|
|
||
| ```bash | ||
| just nut-provenance-verify <fork> | ||
| ``` | ||
|
|
||
| Checks that: | ||
| 1. The bundle file exists and its sha256 matches the lock | ||
| 2. Creates a temporary worktree at the recorded commit, regenerates the bundle, and compares byte-for-byte | ||
|
|
||
| Requires `forge` for the provenance check (step 2). | ||
|
|
||
| ### CI checks | ||
|
|
||
| - **`check-nut-locks`** — Verifies all bundle hashes match their lock entries, all entries have a commit, and every `*_nut_bundle.json` file has a corresponding lock entry. Runs in CI on every PR. | ||
|
|
||
| ## fork_lock.toml schema | ||
|
|
||
| ```toml | ||
| [<fork-name>] | ||
| bundle = "op-node/rollup/derive/<fork>_nut_bundle.json" # repo-relative path | ||
| hash = "sha256:<hex>" # sha256 of bundle contents | ||
| commit = "<full-sha>" # commit that produced the bundle | ||
| ``` | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,6 +1,10 @@ | ||
| # NUT Bundle Fork Lock | ||
| # To update a locked bundle, update both the bundle file and this hash in the same PR. | ||
| # To update a fork's bundle, run: just nut-snapshot-for <fork> | ||
| # Contract changes must be merged to develop before snapshotting. | ||
|
|
||
| [karst] | ||
| bundle = "op-node/rollup/derive/karst_nut_bundle.json" | ||
| hash = "sha256:b9c610d09ca05ab24ef84ea38e4f563d71401f592f9eff13fa97dac879bee600" | ||
| bundle = "op-node/rollup/derive/karst_nut_bundle.json" | ||
| hash = "sha256:6145f900384f0aa2cdc63f2267a747b8c958bf1c09bacce8c29037c3eaa75d44" | ||
| commit = "cba7aba0c98aae22720b21c3a023990a486cb6e0" | ||
|
|
||
| # REVIEWER NOTE: Changes to this file affect which NUT bundles are embedded | ||
| # into op-node for hardfork activations. Review carefully. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| package nuts | ||
|
|
||
| import ( | ||
| "fmt" | ||
| "os" | ||
| "path/filepath" | ||
|
|
||
| "github.com/BurntSushi/toml" | ||
|
|
||
| opservice "github.com/ethereum-optimism/optimism/op-service" | ||
| ) | ||
|
|
||
| // ForkLockEntry represents a single fork's entry in fork_lock.toml. | ||
| type ForkLockEntry struct { | ||
| Bundle string `toml:"bundle"` | ||
| Hash string `toml:"hash"` | ||
| Commit string `toml:"commit"` | ||
| } | ||
|
|
||
| // ForkLock is the full contents of fork_lock.toml, keyed by fork name. | ||
| type ForkLock map[string]ForkLockEntry | ||
|
|
||
| // LockFilePath returns the absolute path to fork_lock.toml relative to the given directory. | ||
| func LockFilePath(dir string) (string, error) { | ||
| root, err := opservice.FindMonorepoRoot(dir) | ||
| if err != nil { | ||
| return "", fmt.Errorf("finding monorepo root: %w", err) | ||
| } | ||
| return filepath.Join(root, "op-core", "nuts", "fork_lock.toml"), nil | ||
| } | ||
|
|
||
| // ReadLockFile reads and parses fork_lock.toml from the monorepo root. | ||
| func ReadLockFile(dir string) (ForkLock, string, error) { | ||
| lockPath, err := LockFilePath(dir) | ||
| if err != nil { | ||
| return nil, "", err | ||
| } | ||
| var locks ForkLock | ||
| if _, err := toml.DecodeFile(lockPath, &locks); err != nil { | ||
| return nil, "", fmt.Errorf("reading fork lock file: %w", err) | ||
| } | ||
| return locks, lockPath, nil | ||
| } | ||
|
|
||
| // WriteLockFile writes fork_lock.toml back to disk with a header comment. | ||
| func WriteLockFile(lockPath string, locks ForkLock) error { | ||
| f, err := os.Create(lockPath) | ||
| if err != nil { | ||
| return fmt.Errorf("opening fork lock file for writing: %w", err) | ||
| } | ||
| defer f.Close() | ||
|
|
||
| _, err = fmt.Fprint(f, `# To update a fork's bundle, run: just nut-snapshot-for <fork> | ||
| # Contract changes must be merged to develop before snapshotting. | ||
|
|
||
| `) | ||
| if err != nil { | ||
| return err | ||
| } | ||
|
|
||
| enc := toml.NewEncoder(f) | ||
| if err := enc.Encode(locks); err != nil { | ||
| return fmt.Errorf("writing fork lock file: %w", err) | ||
| } | ||
|
|
||
| _, err = fmt.Fprint(f, ` | ||
| # REVIEWER NOTE: Changes to this file affect which NUT bundles are embedded | ||
| # into op-node for hardfork activations. Review carefully. | ||
| `) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| return nil | ||
| } |
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,60 @@ | ||
| package main | ||
|
|
||
| import ( | ||
| "crypto/sha256" | ||
| "encoding/hex" | ||
| "testing" | ||
|
|
||
| "github.com/ethereum-optimism/optimism/op-core/nuts" | ||
| "github.com/stretchr/testify/require" | ||
| ) | ||
|
|
||
| func hashOf(data []byte) string { | ||
| h := sha256.Sum256(data) | ||
| return "sha256:" + hex.EncodeToString(h[:]) | ||
| } | ||
|
|
||
| func TestValidateEntry_MatchingHash(t *testing.T) { | ||
| content := []byte(`{"transactions":[]}`) | ||
| entry := nuts.ForkLockEntry{ | ||
| Bundle: "op-node/rollup/derive/test_nut_bundle.json", | ||
| Hash: hashOf(content), | ||
| Commit: "abc123", | ||
| } | ||
| err := validateEntry("test", entry, content) | ||
| require.NoError(t, err) | ||
| } | ||
|
|
||
| func TestValidateEntry_HashMismatch(t *testing.T) { | ||
| content := []byte(`{"transactions":[]}`) | ||
| entry := nuts.ForkLockEntry{ | ||
| Bundle: "op-node/rollup/derive/test_nut_bundle.json", | ||
| Hash: "sha256:0000000000000000000000000000000000000000000000000000000000000000", | ||
| Commit: "abc123", | ||
| } | ||
| err := validateEntry("test", entry, content) | ||
| require.ErrorContains(t, err, "bundle hash mismatch") | ||
| } | ||
|
|
||
| func TestValidateEntry_EmptyCommit(t *testing.T) { | ||
| content := []byte(`{"transactions":[]}`) | ||
| entry := nuts.ForkLockEntry{ | ||
| Bundle: "op-node/rollup/derive/test_nut_bundle.json", | ||
| Hash: hashOf(content), | ||
| Commit: "", | ||
| } | ||
| err := validateEntry("test", entry, content) | ||
| require.ErrorContains(t, err, "no commit recorded") | ||
| } | ||
|
|
||
| func TestValidateEntry_ModifiedBundle(t *testing.T) { | ||
| original := []byte(`{"transactions":[{"intent":"deploy"}]}`) | ||
| modified := []byte(`{"transactions":[{"intent":"modified"}]}`) | ||
| entry := nuts.ForkLockEntry{ | ||
| Bundle: "op-node/rollup/derive/test_nut_bundle.json", | ||
| Hash: hashOf(original), | ||
| Commit: "abc123", | ||
| } | ||
| err := validateEntry("test", entry, modified) | ||
| require.ErrorContains(t, err, "bundle hash mismatch") | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| #!/usr/bin/env bash | ||
| set -euo pipefail | ||
|
|
||
| # Verifies provenance for all forks whose bundle hash changed vs develop. | ||
| # For each fork whose hash changed, checks out the recorded commit, | ||
| # regenerates the bundle, and verifies it matches byte-for-byte. | ||
| # Unchanged forks are skipped to avoid expensive forge rebuilds. | ||
|
|
||
| git show origin/develop:op-core/nuts/fork_lock.toml > /tmp/base_lock.toml 2>/dev/null || true | ||
| for fork in $(yq -p toml -o json op-core/nuts/fork_lock.toml | jq -r 'keys[]'); do | ||
| base_hash=$(yq -p toml ".${fork}.hash" /tmp/base_lock.toml 2>/dev/null || echo "") | ||
| curr_hash=$(yq -p toml ".${fork}.hash" op-core/nuts/fork_lock.toml) | ||
| if [ "$base_hash" != "$curr_hash" ]; then | ||
| echo "Verifying $fork (hash changed)..." | ||
| go run ./ops/scripts/nut-provenance-verify "$fork" | ||
| else | ||
| echo "Skipping $fork (unchanged)" | ||
| fi | ||
| done |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.