Skip to content

Conversation

@francisdb
Copy link
Contributor

@francisdb francisdb commented Dec 16, 2025

Improves the writing speed by keeping track of free sectors.
Adds write benchmark cargo bench

Related to #42

Further work is still needed to improve big stream writing. I think mostly the chaining code.

Before:

     Running benches/benchmark.rs (target/release/deps/benchmark-227a51791cd8dd8e)
Gnuplot not found, using plotters backend
Benchmarking write many smaller streams: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.3s, or reduce sample count to 60.
write many smaller streams
                        time:   [74.316 ms 74.338 ms 74.365 ms]
                        change: [+0.0742% +0.6368% +1.0568%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe

write several medium streams
                        time:   [31.027 ms 31.121 ms 31.220 ms]
                        change: [−0.1073% +0.4565% +1.0339%] (p = 0.12 > 0.05)
                        No change in performance detected.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe

Benchmarking write large stream/size: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 21.8s.
write large stream/size time:   [2.2079 s 2.2408 s 2.2698 s]
                        thrpt:  [112.78 MiB/s 114.24 MiB/s 115.94 MiB/s]
                 change:
                        time:   [−2.1473% −0.2973% +1.3868%] (p = 0.75 > 0.05)
                        thrpt:  [−1.3678% +0.2982% +2.1944%]
                        No change in performance detected.

After:

     Running benches/benchmark.rs (target/release/deps/benchmark-227a51791cd8dd8e)
Gnuplot not found, using plotters backend
Benchmarking write many smaller streams: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.3s, or reduce sample count to 60.
write many smaller streams
                        time:   [72.948 ms 72.974 ms 73.008 ms]
                        change: [−1.8865% −1.8352% −1.7804%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe

write several medium streams
                        time:   [12.985 ms 13.070 ms 13.163 ms]
                        change: [−58.315% −58.003% −57.691%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  8 (8.00%) high mild
  1 (1.00%) high severe

Benchmarking write large stream/size: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 18.5s.
write large stream/size time:   [1.8420 s 1.8530 s 1.8629 s]
                        thrpt:  [137.42 MiB/s 138.15 MiB/s 138.98 MiB/s]
                 change:
                        time:   [−18.456% −17.307% −15.995%] (p = 0.00 < 0.05)
                        thrpt:  [+19.040% +20.930% +22.633%]
                        Performance has improved.

@francisdb francisdb force-pushed the feat/improve_writing_speed branch 2 times, most recently from 2980dcb to c70728e Compare December 16, 2025 13:28
@francisdb francisdb force-pushed the feat/improve_writing_speed branch from c70728e to 0967eed Compare December 18, 2025 15:29
@mdsteele mdsteele merged commit 6ac8c8a into mdsteele:master Dec 24, 2025
4 checks passed
@mdsteele
Copy link
Owner

Thanks!

@francisdb francisdb deleted the feat/improve_writing_speed branch December 24, 2025 13:29
@francisdb
Copy link
Contributor Author

I plan to work on a follow-up PR that improves big file writing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants