Skip to content

CLAUDE.md

Latest commit

 

History

History
138 lines (111 loc) · 5.95 KB

CLAUDE.md

File metadata and controls

138 lines (111 loc) · 5.95 KB

Project Status

Recently Completed

BIP 157/158: Compact Block Filters ✅

Status: Fully implemented and tested (October 2025)

Implemented complete Golomb-Coded Sets (GCS) for client-side block filtering:

Implementation Details:

  • GCS Filter Construction: Extracts scripts from blocks per BIP 158 specification
    • Previous output scripts (scriptPubKeys being spent)
    • Current output scriptPubKeys (except OP_RETURN)
    • Automatic deduplication and lexicographic sorting
    • NO input outpoints (key bug fix - basic filters don't include these)
  • Golomb Encoding: Unary quotient + P-bit remainder (P=19, M=784931)
  • Hash-to-Range: Multiply-and-shift technique (fastReduction) instead of modulo
    • 64×64→128 bit multiplication returning upper 64 bits
    • Avoids expensive modulo while maintaining uniform distribution
    • This was the ROOT CAUSE of initial test failures
  • SipHash-2-4: Keyed hash function using first 16 bytes of block hash (little-endian)
  • BitStream: MSB-first bit-level I/O for compact encoding/decoding
  • Lenient Script Parsing: Handle malformed scripts by storing raw bytes
    • script.ReadScriptBytes() reads without parsing
    • TxOut.RawScriptBytes() returns raw bytes even if parse failed
  • Filter Serialization: VarInt N (item count) + Golomb-coded deltas

Key Bug Fixes:

  1. Wrong hash-to-range algorithm - Used modulo instead of multiply-and-shift
    • Researched btcd implementation to discover fastReduction technique
  2. Included outpoints incorrectly - Basic filters only include scripts, not outpoints
  3. Rejected malformed scripts - Some test vectors have intentionally unparseable scripts
  4. Script serialization corruption - Created RawBytes() to avoid varint prefix issues

Test Results: All 10 BIP 158 official test vectors passing

  • Genesis block
  • Standard blocks (2, 3)
  • Non-standard OP_RETURN outputs
  • Empty output scripts
  • Duplicate pushdata (malformed)
  • Unparseable coinbase (malformed)
  • Witness data transactions
  • Empty data blocks

P2P Message Types Implemented (6 new messages):

  • getcfilters, cfilter - Request/receive compact filters
  • getcfheaders, cfheaders - Request/receive filter headers
  • getcfcheckpt, cfcheckpt - Request/receive filter checkpoints

Files:

  • internal/network/gcs.go - GCS implementation with fastReduction
  • internal/network/gcs_test.go - BIP 158 test vectors
  • internal/block/block.go - ExtractBasicFilterItems() method
  • internal/script/script.go - ReadScriptBytes() and RawBytes() methods
  • internal/transactions/txinputs.go - Lenient ParseTxOut()
  • internal/encoding/bitstream.go - Bit-level I/O

Next Task: Create TestSPVFlow using BIP 158 filters instead of BIP 37 bloom filters

Bech32 Address Encoding (BIP 173) ✅

Status: Fully implemented and tested

Implemented complete bech32 encoding for native SegWit addresses:

  • P2WPKH addresses (bc1q... for mainnet, tb1q... for testnet)
  • P2WSH addresses (bc1q... with 32-byte witness programs)
  • Full BIP 173 compliance with official test vectors
  • Polymod checksum calculation
  • 8-bit to 5-bit data conversion
  • HRP (human-readable part) expansion

Key Bug Fix: Witness version is already 5-bit compatible (0-16), so it's added directly to the data array rather than being converted from 8-bit.

Files:

  • internal/address/bech32.go - Core bech32 encoding implementation
  • internal/address/address.go - Address type and generation functions
  • internal/address/address_test.go - BIP 173 test vectors

Address API Refactoring ✅

Status: Complete migration across codebase

Refactored address generation to use a unified internal/address package:

Old API (deprecated):

addr := script.Address(testnet bool) string  // Returns string, boolean flags
addr := publicKey.Address(compressed, testnet bool) string

New API (current):

addrObj, err := script.AddressV2(network address.Network) (*address.Address, error)
addrObj, err := address.FromPublicKey(pubkey []byte, addrType address.AddrType, network address.Network)
addrObj, err := address.FromHash160(hash160 []byte, addrType address.AddrType, network address.Network)
addrObj, err := address.FromWitnessProgram(version byte, program []byte, network address.Network)

Benefits:

  • Type-safe address.Network enum (MAINNET, TESTNET) instead of boolean flags
  • Supports all address types: P2PKH, P2SH, P2WPKH, P2WSH
  • Centralized address logic in internal/address package
  • Returns structured Address object with .String property
  • Proper error handling

Migration Complete:

  • ✅ All script.Address() calls replaced with script.AddressV2()
  • ✅ Code examples in README.md updated
  • ✅ No remaining deprecated API usage in codebase

TODO

Next Implementation: SPV with Compact Block Filters

Create TestSPVFlow using BIP 158 Filters

Replace the existing BIP 37 bloom filter-based SPV test with a BIP 158 compact block filter implementation.

Goal: Implement client-side block filtering using GCS filters instead of bloom filters

Why This Matters:

  • BIP 37 bloom filters have privacy issues (server knows what you're looking for)
  • BIP 158 filters are downloaded by the client, so server doesn't learn what you're searching for
  • Filters are deterministic and can be cached
  • Better privacy model for light clients

Tasks:

  1. Implement BIP 157 P2P protocol messages (already have message types defined)
  2. Create test that downloads compact filters from a BIP 157-enabled node
  3. Match filter contents against target addresses/scripts
  4. Request full blocks for matches
  5. Verify transactions found

Files to Modify:

  • internal/network/spv_test.go - Create new TestSPVFlowWithCompactFilters
  • May need to implement message serialization for the 6 BIP 157 message types

Reference: