Skip to content

Transactions perfomance improvements#12

Open
rugwirobaker wants to merge 2 commits intomainfrom
perfomance
Open

Transactions perfomance improvements#12
rugwirobaker wants to merge 2 commits intomainfrom
perfomance

Conversation

@rugwirobaker
Copy link
Owner

Add a simple benchmark harness(to be improved)

  • Allows me to run simple Writes, Reads, and Transactions benchmarks.
  • Allows me to generate perf data and flamegraphs.
  • Add corresponding documentation(DRAFT)

WAL changes

Reducing lock contention on the hot path.

  • Remove cascading locks on WAL since it already had interior
    mutability.
  • Replace RwLock with ArcSwap on Active memtable. More at
    docs/performance-profiling-2025-10-08.md.
  • Since the header is read only we don't need to represent it in memory,
    therefore eliminating the need for the associated lock that was
    getting in the way of perfomance. See: docs/wal-header-design-flaw-2025-10-08.md

All in all we halved the number of futex calls per transaction

* Right now direct IO is a net negative until we can implement Async
  Direct io(ref: https://www.scylladb.com/2017/10/05/io-access-methods-scylla/).
* using fallocate(FALLOC_FL_KEEP_SIZE) on Linux.
* Simply WAL writes with buffered io(risk losing data on kernel
  crashes).
* Make the header read only therefore.

* Remove cascading locks on WAL since it already had interior
  mutability.
* Replace RwLock with ArcSwap on Active memtable. More at
  docs/performance-profiling-2025-10-08.md.
* Since the header is read only we don't need to represent it in memory,
  therefore eliminating the need for the associated lock that was
getting in the way of perfomance. See: docs/wal-header-design-flaw-2025-10-08.md

All in all we halved the number of futex calls per transaction
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant