Add a new picking algorithm in fifo compaction#14326
Add a new picking algorithm in fifo compaction#14326xingbowang wants to merge 4 commits intofacebook:mainfrom
Conversation
xingbowang
force-pushed
the
2026_02_11_fifo_compact_policy
branch
2 times, most recently
from
e82b2a8 to
933c40f
Compare
| @@ -0,0 +1,672 @@ | |||
| # FIFO Compaction Strategy | |||
There was a problem hiding this comment.
I didn't find wiki files in our github repo. I guess they are maintained outside of the source code base. I think we should probably move them inside the code base, as we are using AI to generate these document more and more frequently. keeping them in sync within the code base would help keep it up to date more easily.
There was a problem hiding this comment.
Good idea, though the idea of a wiki is easy editing by a broad set of people. If it's revision controlled and gated by maintainers, then it's just "documentation." We have gotten good outside contributions to the wiki, but I could see a model using revision control working better nowadays as most of the high-level user documentation updates could be automated.
| const std::vector<FileMetaData*>& last_level_files = | ||
| vstorage->LevelFiles(last_level); | ||
|
|
||
| if (last_level == 0 && |
There was a problem hiding this comment.
relocated to PickIntraL0Compaction function.
xingbowang
force-pushed
the
2026_02_11_fifo_compact_policy
branch
from
933c40f to
fee5d71
Compare
|
@xingbowang has imported this pull request. If you are a Meta employee, you can view this in D93257941. |
pdillinger
left a comment
pdillinger
left a comment
There was a problem hiding this comment.
I'm not a compaction expert but overall LGTM, thanks!
| // - PickTTLCompaction: estimate remaining data after dropping expired SSTs | ||
| // - PickSizeCompaction: estimate data freed per dropped SST file | ||
| // - PickRatioBasedIntraL0Compaction: compute SST ratio for target sizing | ||
| uint64_t EstimateTotalDataForSST(uint64_t sst_bytes, uint64_t total_sst, |
There was a problem hiding this comment.
Generally looks good, but I find this function confusing, like what total_sst means. See https://www.internalfb.com/metamate/message/f68259e1-5513-4943-ae1f-75649182c25e
xingbowang
force-pushed
the
2026_02_11_fifo_compact_policy
branch
from
64c2626 to
a7cf1bf
Compare
|
@xingbowang has imported this pull request. If you are a Meta employee, you can view this in D93257941. |
|
@xingbowang merged this pull request in b040ab8. |
3 participants
Summary:
Add a new kv ratio based compaction picking algorithm in fifo compaction
Test Plan:
Unit test
Reviewers:
Subscribers:
Tasks:
Tags: