Skip to content

Feature/monero xmr support#235

Open
conorpp wants to merge 41 commits intomainfrom
feature/monero-xmr-support
Open

Feature/monero xmr support#235
conorpp wants to merge 41 commits intomainfrom
feature/monero-xmr-support

Conversation

@conorpp
Copy link
Copy Markdown
Contributor

@conorpp conorpp commented Apr 2, 2026

No description provided.

When two transactions both have ShouldCreateDurableNonce=true for the same
nonce account, they conflict because only one CreateAccountWithSeed can
succeed for a given derived address. Without this check, both would attempt
to create the nonce account and one would revert.
- New DriverMonero with Ed25519 signing and Monero-specific key derivation
  (spend key -> view key -> address)
- Monero base58 encoding, address generation, and validation
- Subaddress generation for exchange use case (unique per-user deposit addresses)
- View key-based blockchain scanning for balance and tx-info
- RPC client for Monero daemon (get_transactions, get_block, get_fee_estimate)
- Output ownership detection using ECDH derivation + stealth address matching
- RingCT encrypted amount decryption
- Chain config in mainnet.yaml with 12 decimal places
Public Monero nodes reject bulk get_transactions requests in restricted
mode. Split into batches of 25 to stay within limits.
- Bulletproofs+ prover: Pedersen commitments, inner product argument,
  generator point derivation (H, Gi, Hi vectors)
- Full transaction builder: output stealth keys, amount encryption,
  pseudo-output commitment balancing, BP+ integration
- Tx module with CLSAG sighash interface: Sighashes() returns data
  for ring signing, SetSignatures() attaches CLSAG sigs
- Serialization in Monero's wire format (version 2, RCT type 6)
- CLSAG ring signatures to be implemented next (FROST MPC compatible)
- CLSAG ring signature implementation (single-party local signer)
- Decoy ring member selection from daemon RPC with gamma-like distribution
- Key image computation: I = x * H_p(P)
- Output scanning during FetchTransferInput to find spendable outputs
- Deterministic RNG for reproducible transaction construction
- Full transfer dry-run working end-to-end:
  scan outputs → build tx → BP+ proof → CLSAG sign → serialize
- Monero signer pass-through (CLSAG computed in builder, not standard Ed25519)
- Fetch 15 decoy outputs from daemon via get_outs for each input
- Build sorted rings with relative key offsets
- Fix fee estimation to use JSON-RPC endpoint
- Fetch global output indices for owned outputs
- Detailed rejection logging from send_raw_transaction
- Transfer builds and serializes with full 16-member rings
- Node rejects with invalid_input - CLSAG/key derivation needs
  alignment with Monero's exact cryptographic constants
…H generator

Ported Monero's C reference crypto-ops code as CGO for exact compatibility:
- ge_fromfe_frombytes_vartime (Elligator map, not trial decompression)
- sc_reduce32 (32-byte mod L, not 64-byte SetUniformBytes)
- generate_key_derivation (8 * secret * public with cofactor)
- generate_key_image (secret * hash_to_ec(public))
- H generator point from precomputed constant in crypto-ops-data.c

Unit tests with Monero test vectors (tests/crypto/tests.txt):
- hash_to_point: 5 vectors pass
- hash_to_ec: 5 vectors pass
- generate_key_derivation: 3 vectors pass
- generate_key_image: 3 vectors pass

Commitment mask derivation now matches on-chain values:
- mask = H_s("commitment_mask" || shared_scalar) per rctOps.cpp
- Verified: computed commitment == on-chain commitment for our deposit

Note: CGO_ENABLED=0 builds no longer supported for monero package.
…tion

- D_full (= z * Hp) used for ring R computation (matching both prove/verify)
- sig.D (= D/8) used for aggregation hash (matching both prove/verify)
- Increased scan depth to 1000 blocks for longer-running tests
- Still getting invalid_input - likely serialization format issue
…_prunable)

Major serialization refactor:
- Separate serializePrefix, serializeRctBase, serializeRctPrunable
- RCT base: type(1) || fee(varint) || ecdhInfo(8 bytes each) || outPk(32 each)
- BP+ prunable hash: raw key concatenation (A,A1,B,r1,s1,d1,L[],R[])
- CLSAG message = Keccak(prefix_hash || H(rct_base) || H(bp_prunable))
- Builder phases: build tx → compute message → sign CLSAGs
- CLSAG serialization: s[0..n-1](32 each) || c1(32) || D(32), NO size prefix
- Fix: pseudo-out commits to totalInput (not totalInput-fee)
- Balance equation: sum(pseudo_outs) = sum(out_commitments) + fee*H
- CLSAG sign + verify unit tests (ring size 4 and 16, both pass)
- CLSAGVerify function for local signature validation
Major fix: replace our Go BP+ implementation with Monero's exact C++
bulletproof_plus_PROVE via CGO linkage.

- Built Monero's ringct C++ code as static library (libbpplus.a)
- CGO wrapper: BPPlusProve, BPPlusVerify, ParseBPPlusProof
- Cache BP+ proof in TxInput for deterministic repeated Transfer() calls
- Transaction now accepted by Monero mainnet node (no more invalid_input!)
- Tx hash tracking issue: local hash may differ from network hash
- Tx hash = H(H(prefix) || H(rct_base) || H(rct_prunable))
  Verified against real Monero tx hash (exact match)
- CLSAG message computed from serialized blob bytes (not separate methods)
  Ensures message matches what verifier computes from the tx blob
- computeCLSAGMessageFromBlob parses blob to extract boundaries
…parsing

- CLSAG message computed from Tx.SerializePrefix/RctBase/BpPrunable
- Verified: Go and C++ produce identical CLSAG message from same blob
- Exported Serialize methods for builder access
- Higher fee (200x base) to ensure quick mining
- BP+ verification confirmed PASS by Monero C++ verifier
- Commitment balance confirmed CORRECT by Monero C++ code
Critical fixes:
- Output key derivation now uses cofactor (8*r*pubView via CGO)
  Previously used r*pubView (no cofactor), making outputs unscannable
- Amount encryption uses proper ECDH shared scalar per recipient
  Previously used wrong derivation, making amounts undecryptable
- Both fixes verified with roundtrip test: builder output == scanner output

Fixed view key:
- All crosschain Monero addresses share a single view key
- Derived from H("crosschain_monero_view_key")
- Enables single-key scanning across all user addresses
- Each user still gets unique address (different spend key)
- Testnet address prefix (0x35 = 53, addresses start with '9')
- Testnet chain config in testnet.yaml
- Testnet node: testnet.xmr-tw.org:28081
- Address builder detects testnet from chain_id config
- Validate accepts both mainnet and testnet prefixes
The builder now serializes the tx first, then parses the blob to compute
the CLSAG message - guaranteeing it matches what the verifier computes.

Previously the Tx.Serialize methods produced slightly different bytes
than the full Serialize() output, causing a CLSAG message mismatch.

Testnet transfer verified:
- TX accepted by testnet node (no invalid_input)
- Both send and change outputs scannable with fixed view key
- Balance correctly reflects received outputs
- Output masks derived from ECDH shared secret (genCommitmentMask)
  instead of random values - enables spending our own change outputs
- Filter spent outputs by computing key images and checking
  is_key_image_spent before selecting inputs
- Sort outputs largest-first for optimal input selection
- Testnet transfer to different wallet (AUX_1) confirmed on chain:
  tx e2ead28997ea7e5b84ca198a6021a995bc28e743a0fdacdb89cb1ae2d7e1198b
FetchTxInfo now decodes ALL outputs using the fixed view key without
requiring any private spend key. This matches the exchange use case:
see every deposit to any user address with a single view key.

Before: only showed outputs matching one wallet's spend key (0.0079 XMR)
After:  shows ALL outputs decryptable by the view key (0.002 + 0.0079 XMR)
For each output, reverse-derive the public spend key: pubSpend = P - s*G
Then reconstruct the full address from (prefix, pubSpend, pubView).
No private spend key needed - the fixed view key is sufficient.

tx-info now shows the exact recipient address for each output:
- Output 1: 0.002 XMR → AUX_1 address (9zcd2gzb...)
- Output 2: 0.0079 XMR → sender address (9uUpcHKY...) [change]
- NewTxInfo, NewMovement, AddSource, AddDestination, NewBalance
- Populates asset, asset_id, contract, address, address_id, chain
- state/final computed automatically by NewTxInfo from block height
Crypto primitives (from monero-project/tests/crypto/tests.txt):
- hash_to_ec: 3 vectors
- generate_key_derivation: 3 vectors
- generate_key_image: 3 vectors

Constants:
- H generator point matches Monero's crypto-ops-data.c
- Fixed view key is deterministic and independent of spend key

Transaction hashing:
- Three-hash structure verified against real mainnet tx
  (197d45b6a07c9ccafb7cf8e5f72c18edff1c294f1490b7dd34c8d3ff0e669814)

View key + output scanning:
- Commitment mask matches on-chain commitment for real deposit
- Output key derivation roundtrip: builder == scanner
- Amount encryption/decryption roundtrip
- Commitment mask consistency between builder and scanner

Proofs:
- BP+ prove + verify via Monero C++ library (1 and 2 outputs)
- BP+ proof field parsing (L/R round counts)
- CLSAG sign + verify (ring size 4 and 16)
New vectors from Monero test suite and confirmed testnet tx:
- derive_public_key: 5 vectors from tests/crypto/tests.txt
- derivation_to_scalar: 7 vectors generated from C reference
- BP+ commitment: known amount+mask -> expected Pedersen commitment
- Tx hash from blob: prefix/rct_base/prunable component hashes
  from confirmed testnet tx e2ead289...
- CLSAG message: three-hash computation verified by both Go and C++
- CLSAG signature: c1, D, s[0..15], ring keys, commitments, pseudoOut
  all from the confirmed testnet tx for verifier testing

Total: 19 tests covering all crypto primitives needed for pure Go rewrite.
TestCLSAGSignatureVector now verifies the actual CLSAG signature from
confirmed testnet tx e2ead289... using:
- 16 ring member public keys from the blockchain
- 16 ring member commitments from the blockchain
- 16 response scalars s[0..15] from the tx blob
- c1, D, key image from the tx blob
- pseudoOut from the tx blob
- CLSAG message computed from the three-hash structure

This is a concrete test vector for validating a pure Go CLSAG verifier.
Port Monero's hash-to-point using filippo.io/edwards25519/field:
- Field element constants (feMA, feMA2, feSqrtM1, feFFfb1-4)
  computed dynamically using field.SqrtRatio
- feDivPowM1 using field.Pow22523
- High bit handling: add 2^255 ≡ 19 (mod p) when SetBytes strips it
- All 5 hash_to_point + 5 hash_to_ec test vectors pass
- No CGO needed for this function
- ScReduce32PureGo: uses SetCanonicalBytes with SetUniformBytes fallback
- GenerateKeyDerivationPureGo: cofactor ECDH (8 * sec * pub) via 3 doublings
- GenerateKeyImagePureGo: sec * hash_to_ec(pub) using pure Go Elligator
- GetHPureGo: hardcoded H constant from Monero's crypto-ops-data.c
- All verified against CGO reference: sc_reduce32, key derivation (3 vectors),
  key image (3 vectors), H point, hash_to_ec
Initial port of BP+ prover from Monero C++. Includes:
- Generator tracking through inner product rounds
- Folding of Gprime/Hprime vectors
- A1/B final round computation
- Full proof serialization

Does not pass C++ verifier yet - needs debugging of:
- computeLR generator offset handling
- Fiat-Shamir transcript exact byte matching
- weighted_inner_product edge cases

All other crypto primitives (hash_to_point, sc_reduce32, key derivation,
CLSAG) have working pure Go implementations.
- HashToEC, HashToPoint, ScReduce32 now use pure Go implementations
- GenerateKeyDerivation uses pure Go cofactor ECDH
- ComputeKeyImage uses pure Go hash_to_ec
- H and Gi/Hi generators initialized from pure Go hash_to_point
- Fixed init ordering: lazy init for Elligator constants,
  const hex string for H point (avoids init() ordering issues)
- All 19 tests pass with pure Go crypto primitives
- Only remaining CGO: BP+ prove/verify (cref.BPPlusProve)
- Generators now use correct derivation: hash_to_p3(Keccak(H || "bulletproof_plus" || varint(i)))
  Previously used wrong base (missing H prefix)
- initial_transcript is hash_to_p3 of domain string (a point), not a scalar
- Transcript uses raw 32-byte keys, not scalars (avoids mod L reduction)
- V commitments match between Go and C++ (confirmed)
- initial_transcript matches between Go and C++ (confirmed)
- BP+ prover still needs inner product rounds debugging
Root cause: hash_to_p3 double-hashes (cn_fast_hash internally before Elligator)
- initial_transcript: need Keccak(Keccak("bulletproof_plus_transcript"))
- Gi/Hi generators: need Keccak(Keccak(H||prefix||varint(i)))
- Now matches C++ exactly: initial_transcript, V, y, z all verified

Pure Go BP+ implementation:
- Inner product argument with generator folding
- Weighted inner product, Hadamard fold
- Fiat-Shamir transcript with raw 32-byte keys
- Final round A1/B computation with folded generators
- All verified against C++ Monero bulletproof_plus_VERIFY

All 20 tests pass. Ready for Phase 5: remove CGO entirely.
- Replaced cref.BPPlusProve with BPPlusProvePureGo in builder
- Replaced cref.BPPlusFields with crypto.BPPlusFields
- Replaced cref.ParseBPPlusProof with crypto.ParseBPPlusProofGo
- Removed all cref imports from crypto, builder, tx packages
- Tests use pure Go BP+ (no C++ verifier)
- CGO_ENABLED=0 go build ./... passes
- CGO_ENABLED=0 go test ./chain/monero/crypto/ passes (19 tests)
- CGO_ENABLED=0 go install ./cmd/xc/ passes

The cref/ package still exists for reference but is no longer imported
by any production code.
TX cd5c146bf9a9c2b22fd38205e6ecc3b11c95b120bbca897435c09963a2955340
Block 2973822, built with CGO_ENABLED=0.

Fix: filter outputs with insufficient decoys (< 15) to ensure valid ring size.
Testnet has sparser outputs which caused some decoy fetches to return too few.
Outputs from old transfers (before commitment mask fix) have incorrect
masks and can't be spent. Now we compute the expected commitment and
compare against the on-chain value before including an output.

Second pure Go transfer confirmed: 6600e915... (block 2973841)
Builder changes:
- No loadKeys(), no signer.ReadPrivateKeyEnv() - zero private key access
- Uses args.GetPublicKey() for sender's public spend/view keys
- Uses pre-computed CommitmentMask from TxInput (set by client)
- Uses RngSeed from TxInput for deterministic randomness
- Builds unsigned tx, stores CLSAGContexts for signer

Two-phase signing via Sighashes/AdditionalSighashes:
1. Sighashes() → signer computes key images (32 bytes each)
2. SetSignatures() fills key images into tx inputs
3. AdditionalSighashes() → signer produces CLSAG ring signatures
   (using the correct CLSAG message computed AFTER key images are set)
4. SetSignatures() attaches CLSAG signatures

TxInput changes:
- Added TxPubKey, CommitmentMask fields (pre-computed by client)
- Added RngSeed for deterministic builder operations
- Removed ViewKeyHex (private key shouldn't be in TxInput)

Signer changes:
- Phase 1: derives one-time key, returns key image
- Phase 2: produces full CLSAG signature with deterministic RNG
The get_output_distribution endpoint returns millions of integers for
the entire chain, causing 30GB+ memory usage. Now use get_info for
total output count estimation and cap decoy indices at the real
output's global index.

Also added debug logging for decoy fetching phase.
When indexer_url is configured, the client uses monero-lws endpoints:
- /login: register address + view key (auto-tracks subaddresses)
- /get_unspent_outs: instant UTXO query (no block scanning!)
- /get_address_info: instant balance

Falls back to block scanning if LWS is unavailable.

Architecture:
- LWSClient in client/lws.go handles all LWS communication
- ConvertLWSOutputs maps LWS response to tx_input.Output format
- Fee estimates come from LWS (per_byte_fee, fee_mask)
- Decoys still fetched from daemon via get_outs
- One registration covers all subaddresses (via lookahead)

Config: set indexer_url in chain config to enable:
  XMR:
    url: "http://localhost:18081"        # monerod
    indexer_url: "http://localhost:8443"  # monero-lws
monero-lws returns per_byte_fee and fee_mask as JSON numbers,
not strings. Use uint64 for fee fields and json.Number for
amount/global_index which can be either format.

Tested against local monero-lws on mainnet:
- /login: registers address successfully
- /get_address_info: instant balance
- /get_unspent_outs: returns fee estimates + outputs
- Falls back to block scan when LWS has no outputs
- LWS get_unspent_outs needs mixin=15 (decoys), not 16 (ring size)
- Fix SetSignatures phase detection: use signingPhase counter
  instead of signature size heuristic
- Fix infinite loop: AdditionalSighashes now returns nil after phase 2
- Full LWS transfer flow works in 3 seconds (vs 5+ min scanning):
  LWS outputs → build → key image → CLSAG sign → submit
- Node rejected with double_spend (LWS unaware of prior spends),
  but the protocol and crypto are correct
- Filter outputs where LWS has a known key image (already spent)
- Submit transactions to LWS via /submit_raw_tx so it tracks our
  key images for future spent detection
- Confirmed testnet transfer via LWS: 5d225dae... (block 2976878)
  Full flow in ~3 seconds (LWS outputs → build → sign → submit)
- Remove duplicate CheckError from client/client.go (canonical is errors.go)
- Consolidate BP+ constants: remove bpMaxN/M/MN dupes, use maxN/M/MN
- Extract CommitmentMaskLabel constant to crypto package
- Consolidate txBatchSize to package-level const in client
- Remove local clsagInputContext struct, use tx.CLSAGInputContext directly
- Add cross-reference comments for intentional duplications
  (MoneroSighash in tx/ and crypto/, BuildRing in client/ and builder/)
- Add constants.go with documented default values
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants