Append-Only File (AOF) Style Logging for High-Performance Durable Writes

[sectasy0]

Feature Title

Append-Only File (AOF) Style Logging for High-Performance Durable Writes

Feature Description

Implement an Append-Only File (AOF) subsystem to provide optionally durable and replayable write operations. Instead of logging low-level binary mutations, Redis-style AOF logs text-based or encoded operation commands (e.g., SET, DEL) in a sequential log file.

This mechanism allows:

• Fast, append-only writes, bypassing immediate changes to disk-based data structures.
• Configurable durability guarantees, including buffered and batched flush modes.
• Crash recovery via replay of the append-only log on startup.
• Simplified durability model, where recovery is achieved by re-executing the command log.

The AOF system should support:

• Command-based entries (text or lightweight encoded).
• Configurable flush strategies:
  • always: flush on every operation.
  • everysec: flush once per second.
  • no: let the OS manage flushing.
• AOF file rotation or rewriting (compaction).
• Recovery via command replay at startup.
• Optional snapshotting (RDB-style) to complement AOF for faster restarts.

Motivation

Current writes are applied directly to memory or disk structures, risking data loss on crash. AOF introduces a high-performance durability layer that can be tuned for different use cases:

• Strict durability (fsync=always) for mission-critical writes.
• Buffered durability (everysec) as a balanced trade-off.
• No durability (fsync=no) for ephemeral or cache-like workloads.

This model matches Redis and is ideal for:

• High-throughput systems.
• Log-style data ingestion.
• Systems where full transaction durability is not always necessary.

Proposed Solution

• On write (SET, INCR, etc.), log the full command into an append-only log file (appendonly.aof).
• Depending on config, the file is flushed immediately, periodically, or left to the OS.
• A background applier periodically rewrites or compacts the AOF file to reduce size and remove obsolete operations.
• On startup, the system replays the AOF line-by-line to reconstruct the latest state.
• Optional snapshotting (RDB) can reduce startup time by checkpointing state alongside AOF.

API/config changes:

.aof = .{
  .flush_strategy = "everysec", // options: "always", "everysec", "no", default: "everysec"
  .path = "./data/appendonly.aof", // default: ./appendonly.aof
  .rewrite_percentage = 100, // 100%
  .rewrite_min_size = 64 // 64MB
},

• CLI flag: --replay-aof to manually trigger journal replay.
• Internal API:
  • AOF.append(command: str) - Appends to journal (logs the given command to the append-only file)
  • AOF.replay() - Replays the commands from the AOF to restore the database state on startup
  • AOF.rewrite() - Overwrites the journal file with commands representing the current actual database state (compaction)

Alternatives Considered

• Full binary WAL with fsync on commit: safer, but slower and more complex.
• Snapshot-only model (RDB): faster, but cannot recover recent writes.
• Journaling actual data structures: heavy and unnecessary for log-based writes.
• Storing commands as parsed arrays only: simplifies execution but requires serializing commands to string format on every AOF append, which may introduce overhead.
• Storing both the original command string and its parsed array representation: uses more memory but enables faster AOF appends by writing the pre-serialized string directly, avoiding repeated serialization while preserving efficient in-memory command execution.

AOF offers a middle ground, with strong durability for many use cases and minimal overhead.

Additional Context

• Based on Redis AOF implementation.
• Pairs well with snapshotting for fast restarts.
• Could be extended to support:
  • Batched command replay
  • Logical replication
  • AOF file checksum verification

References:

• Redis persistence documentation
• Redis AOF fsync modes