chore: migrate to protocol `v0.14.0-beta.1` by mmagician · Pull Request #1784 · 0xMiden/node

mmagician · 2026-03-11T19:44:50Z

Summary

Update all miden-protocol dependencies from 0.14.0-alpha.1 to 0.14.0-beta.1
Update miden-crypto from 0.19.7 to 0.22
Remove miden-air dependency
Bump node workspace version to 0.14.0-beta.1

Key API migrations

Serialization traits moved to utils::serde submodule
Felt::as_int() -> Felt::as_canonical_u64()
falcon512_rpo -> falcon512_poseidon2
ProvenTransactionBuilder removed, use ProvenTransaction::new() + TxAccountUpdate (refactor: remove ProvenTransactionBuilder in favor of ProvenTransaction::new protocol#2567)
OutputNote variants: Full -> Public, Header -> Private
Asset is now key-value (two words instead of one)
MmrProof fields are now methods
account_id_to_smt_key -> AccountIdKey::from()
SmtStorage trait methods now take &mut self
AccountStateForest rewritten to replace removed LargeSmtForest

Test plan

cargo check --all-targets passes
make lint passes (clippy, formatting, machete)
CI passes

🤖 Generated with Claude Code

Update all miden-protocol dependencies from 0.14.0-alpha.1 to 0.14.0-beta.1 (crates.io). Update miden-crypto from 0.19.7 to 0.22. Bump node workspace version to 0.14.0-beta.1. Key migration changes: - Serialization traits moved to utils::serde submodule - Felt::as_int() -> Felt::as_canonical_u64() - falcon512_rpo -> falcon512_poseidon2 - ProvenTransactionBuilder removed, use ProvenTransaction::new() - OutputNote variants: Full -> Public, Header -> Private - Asset is now key-value (two words instead of one) - MmrProof fields are now methods - account_id_to_smt_key -> AccountIdKey::from() - miden-air removed (replaced by miden-core 0.21) - SmtStorage trait methods now take &mut self - Various other API changes in miden-crypto 0.22

crates/large-smt-backend-rocksdb/src/lib.rs

mmagician · 2026-03-12T10:29:25Z

(force pushed coz I had unsigned commits)

igamigo · 2026-03-11T20:04:47Z

crates/ntx-builder/src/actor/execute.rs

    #[error("the ntx task panicked")]
+    #[expect(dead_code)]
    Panic(#[source] JoinError),


Probably can be removed

crates/validator/src/signers/kms.rs

- Use LargeSmt::load() instead of LargeSmt::new() in load_smt to support loading from non-empty RocksDB storage on restart (new() now errors with StorageNotEmpty in miden-crypto 0.22) - Fix prune logic to remove all versions before cutoff while keeping at least one version per lineage to preserve current state - Only pop roots from SmtForest that are no longer referenced by any lineage (prevents removing shared roots) - Fix test helpers tree_id_for_root/tree_id_for_vault_root to use get_root_at_block instead of latest_root - Update all prune test assertions for the new version-tracking model (our AccountStateForest uses explicit version entries rather than LargeSmtForest's deduplicated tree count)

Replace ~130 lines of hand-rolled DER parsing with proper library calls: - Use spki::SubjectPublicKeyInfoRef::from_der() for SPKI public key parsing - Use k256::ecdsa::Signature::from_der() for DER signature decoding This restores the simplicity of the original code before the migration, where PublicKey::from_der() and Signature::from_der() were available directly on the miden-crypto types.

Resolve conflicts in ntx-builder query tests by keeping test_utils imports and fixing TransactionId::new() signature for the beta API.

crates/block-producer/src/test_utils/note.rs

crates/large-smt-backend-rocksdb/src/rocksdb.rs

mmagician · 2026-03-12T13:56:15Z

crates/large-smt-backend-rocksdb/src/rocksdb.rs

            batch.put_cf(metadata_cf, ENTRY_COUNT_KEY, new_entry_count.to_be_bytes());
        }

-        batch.put_cf(metadata_cf, ROOT_KEY, new_root.to_bytes());


why was this removed?

crates/rpc/src/tests.rs

crates/store/src/db/tests.rs

mmagician · 2026-03-12T14:59:12Z

crates/block-producer/src/test_utils/proven_tx.rs

            self.initial_account_commitment,
            self.final_account_commitment,
            Word::empty(),
+            AccountUpdateDetails::Private,


I was wondering why this is set to Private.
ProvenTransaction::new used to default to Private if no account_update was specified, so this preserves the behavior

Remove stale comment about missing re-exports (fixed in miden-crypto 0.22.5) and import from miden_protocol::crypto::merkle::smt to match the convention used on next.

Public and private accounts were created with overlapping seed index ranges, which in the v0.14 beta can produce accounts with duplicate ID prefixes. Use separate non-overlapping index offsets for each storage mode within a block.

The previous approach used low-entropy seeds (index padded with zeros) which could produce AccountId prefix collisions during the hash grinding process. Use a seeded PRNG to generate full 32-byte random seeds per account while maintaining deterministic behavior.

Resolve conflict in state/loader.rs imports: keep RocksDbOptions from next's rocksdb config PR and AccountIdKey from our beta migration.

crates/remote-prover-client/src/remote_prover/batch_prover.rs

crates/remote-prover-client/src/remote_prover/tx_prover.rs

crates/remote-prover-client/src/remote_prover/block_prover.rs

crates/rpc/src/server/api.rs

mmagician · 2026-03-12T16:16:48Z

crates/rpc/src/server/api.rs

+            let rebuilt = PublicOutputNote::new(public_note.as_note().clone())
+                .expect("rebuilding an already-valid public output note should not fail");
+            OutputNote::Public(rebuilt)


this works but we might want to propagate minify_script to PublicOutputNote to avoid the reconstruction.

minify_script() is called in PublicOutputNote::new() - so, by construction all PublicOutputNotes have minified scripts. But maybe I didn't understand the comment?

Not quite: I was thinking of mutating PublicOutputNote, something like:

fn strip_output_note_decorators<'a>( notes: impl Iterator<Item = &'a OutputNote> + 'a, ) -> impl Iterator<Item = OutputNote> + 'a { notes.map(|note| match note { OutputNote::Public(public_note) => { let minified = public_note.clone().minify_script(); // <------ would need this OutputNote::Public(public_note) }, OutputNote::Private(header) => OutputNote::Private(header.clone()), }) }

so rather than fully reconstructing PublicOutputNote::new(), we would avoid some of the other checks that new constructor performs. These checks are not expensive at all, but I think it would also be cleaner to "only do the work necessary" to strip the decorators, using the constructor to do it feels somewhat hacky.

mmagician · 2026-03-12T16:45:23Z

crates/store/src/account_state_forest/mod.rs

+    /// `SmtForest` for efficient account storage reconstruction.
    /// Populated during block import with storage and vault SMTs.
-    forest: LargeSmtForest<ForestInMemoryBackend>,
+    forest: SmtForest,


hmm is this change correct? What's the reason this was changed to not use LargeSmtForest anymore?

Seems like the ForestInMemoryBackend is not available in in miden-crypto 0.22 and it decided to replace it with SmtForest plus some extra functionality. @drahnr (or maybe @bobbinth) should ForestInMemoryBackend be ported to 0.22 or what is the expectation here?

I'd prefer to not undo the migration to LargeSmtForest in next, it simplified a lot of code.

- Remove dead `Panic(JoinError)` variant from `NtxError` - Remove unused `get_root`/`set_root` and `ROOT_KEY` from `RocksDbStorage` - Replace verbose `std::iter::empty` with `Vec::new()` in rpc tests - Extract inline fee construction to `test_fee()` variable in store tests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

mmagician · 2026-03-12T17:11:16Z

crates/store/src/db/tests.rs

+    // cutoff = 53 - 50 = 3: each of the 3 lineages has one version < 3 that gets removed
+    // (account1 v=1, account2 v=2, account3 v=2), since each also has a newer version.


also I don't fully understand the lineage changes cc @drahnr

sergerad · 2026-03-12T18:39:52Z

crates/validator/src/signers/kms.rs

+            .subject_public_key
+            .as_bytes()
+            .ok_or_else(|| DeserializationError::InvalidValue("Invalid SPKI BIT STRING".into()))?;
+        let pub_key = PublicKey::read_from_bytes(sec1_bytes)?;


We don't want these diffs. As in we want to keep:

let pub_key = PublicKey::from_der(spki_der)?;

I think the parsing was introduced during the migration because miden-crypto 0.22 does not contain the PublicKey::from_der() and Signature::from_der() methods. Should this also be ported to 0.22.X?

If from_der() is not in v0.22, @sergerad - could you make a quick PR to cherry-pick relevant commits into v0.22.x branch in miden-crypto and I'll make another patch release.

Just in case, there's also this that might need porting. You probably saw already but mentioning it in the unlikely case you didn't, so that we don't waste a patch release.

crates/validator/src/signers/kms.rs

SantiagoPittella · 2026-03-13T16:49:22Z

I created #1793 using this PR as base. It is a really simple one. If you prefer I can push the commit here, or feel free to just copy the code.

mmagician · 2026-03-13T17:02:33Z

go right ahead @SantiagoPittella!

This reverts commit 99c7134.

igamigo · 2026-03-13T19:22:59Z

Pushed a commit to point to 0xMiden/protocol#2598 which fixes the stress test (and the overall functionality of the account tree). Also removed the crypto patch in favor of updating the package version to the recently-released 0.22.6

mmagician added the no changelog This PR does not require an entry in the `CHANGELOG.md` file label Mar 11, 2026

mmagician force-pushed the mmagician-claude/update-base-to-beta branch from b785bde to 922cd86 Compare March 11, 2026 19:46

mmagician changed the title ~~chore: migrate to protocol v0.14.0-beta.1~~ chore: migrate to protocol v0.14.0-beta.1 Mar 11, 2026

igamigo reviewed Mar 11, 2026

View reviewed changes

crates/large-smt-backend-rocksdb/src/lib.rs Outdated Show resolved Hide resolved

SantiagoPittella mentioned this pull request Mar 11, 2026

chore: use next from protocol #1785

Closed

igamigo and others added 4 commits March 11, 2026 17:12

chore: block_on

bc85d59

chore: patch crypto until a 0.22.x release is done

765169b

chore: update to crypto v0.22.5

44d5214

chore: fmt

5320e57

mmagician force-pushed the mmagician-claude/update-base-to-beta branch from bb9974c to 5320e57 Compare March 12, 2026 10:29

Merge branch 'next' into mmagician-claude/update-base-to-beta

4d53991

mmagician force-pushed the mmagician-claude/update-base-to-beta branch from 4c74ff6 to 4d53991 Compare March 12, 2026 10:32

SantiagoPittella mentioned this pull request Mar 12, 2026

Replace middle-step serialization of InputNoteCommitment with direct constructor #1783

Open

igamigo reviewed Mar 12, 2026

View reviewed changes

claude added 3 commits March 12, 2026 13:39

Merge branch 'next' into mmagician-claude/update-base-to-beta

a5f88db

Resolve conflicts in ntx-builder query tests by keeping test_utils imports and fixing TransactionId::new() signature for the beta API.

mmagician commented Mar 12, 2026

View reviewed changes

claude added 4 commits March 12, 2026 15:05

chore: import smt types from miden-protocol instead of miden-crypto

5767361

Remove stale comment about missing re-exports (fixed in miden-crypto 0.22.5) and import from miden_protocol::crypto::merkle::smt to match the convention used on next.

Merge branch 'next' into mmagician-claude/update-base-to-beta

71c29d4

Resolve conflict in state/loader.rs imports: keep RocksDbOptions from next's rocksdb config PR and AccountIdKey from our beta migration.

mmagician commented Mar 12, 2026

View reviewed changes

claude and others added 2 commits March 12, 2026 16:55

chore: cleanup imports

0490027

mmagician commented Mar 12, 2026

View reviewed changes

sergerad reviewed Mar 12, 2026

View reviewed changes

crates/validator/src/signers/kms.rs Outdated Show resolved Hide resolved

test: single source of randomness

99c7134

mmagician mentioned this pull request Mar 13, 2026

chore: prepare patch release v0.22.6 0xMiden/crypto#897

Merged

mmagician added 2 commits March 13, 2026 12:25

temp: use mmagician-0.22.6 branch

9d5ff52

chore: revert kms changes

f9d4412

fix: remove middle step serialization for InputNoteCommitment

c5c87b5

SantiagoPittella mentioned this pull request Mar 13, 2026

fix: remove middle step serialization for InputNoteCommitment #1793

Closed

igamigo added 3 commits March 13, 2026 15:20

chore: cargo lock

98497ba

Revert "test: single source of randomness"

8079f10

This reverts commit 99c7134.

chore: remove crypto patch

b6b3af0

		// cutoff = 53 - 50 = 3: each of the 3 lineages has one version < 3 that gets removed
		// (account1 v=1, account2 v=2, account3 v=2), since each also has a newer version.

Conversation

mmagician commented Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Key API migrations

Test plan

Uh oh!

Uh oh!

mmagician commented Mar 12, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mmagician Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

SantiagoPittella commented Mar 13, 2026

Uh oh!

mmagician commented Mar 13, 2026

Uh oh!

igamigo commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

mmagician commented Mar 11, 2026 •

edited

Loading

mmagician Mar 12, 2026 •

edited

Loading

igamigo commented Mar 13, 2026 •

edited

Loading