Skip to content

Conversation

@Giulio2002
Copy link
Contributor

@Giulio2002 Giulio2002 commented Feb 1, 2026

Summary

Introduces a file-based diffset storage system (DiffsetDatabase) for storing per-block state change sets, with zero-allocation optimizations for high-performance serialization.

Key Changes

  • New DiffsetDatabase (db/diffsetdb/) - File-based storage for block diffsets

    • Stores each block's diffset as a separate file: {blockNumber}+{blockHash}.diffset
    • Uses a simple mutex for thread safety
    • Async writes with goroutine to avoid blocking callers
  • Zero-allocation optimizations:

    • Pre-allocated reusable buffers for serialization (serializeBuf), path building (pathBuf, filenameBuf)
    • SerializeDiffSet uses fixed-size arrays instead of maps for dictionary building (linear search beats map for small N)
    • Pre-computed hex lookup table for filename encoding
    • StateChangeSet.SerializeTo() method for buffer reuse
  • Serialization improvements (db/state/changeset/):

    • Empty diffset fast path (returns 5 bytes: dict len + diffSet len)
    • Pre-calculates total buffer size to avoid reallocations
    • Uses binary.BigEndian.AppendUint* for inline encoding

Benchmark Results

Operation Allocs Bytes/op
WriteDiffSet 10 ~1 KB
SerializeDiffSet (with buffer reuse) 0 0 B
ReadDiffSet 31 69 KB

@Giulio2002 Giulio2002 enabled auto-merge (squash) February 2, 2026 18:59
@Giulio2002 Giulio2002 changed the title [WIP] Performance: diffset store Performance: diffset store Feb 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants