chore: update data_substrate — fix BackFill dirty_data_key_count_ underflow#458
chore: update data_substrate — fix BackFill dirty_data_key_count_ underflow#458githubzilla wants to merge 1 commit intoeloqdata:mainfrom
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
WalkthroughThis PR updates the Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~1 minute Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
📝 Coding Plan
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…erflow Update data_substrate submodule to include the fix for an intermittent assertion failure in AdjustDataKeyStats where dirty_data_key_count_ underflows during cluster scale-out. See eloqdata/tx_service#465 for details.
ec5f92a to
da92602
Compare
Summary
Update
data_substratesubmodule to include the fix for an intermittent assertion failure during cluster scale-out.Problem
test_add_node_with_double_writeintermittently crashes with:dirty_data_key_count_is being decremented below zero — more entries are "un-dirtied" than were ever counted as dirty.Root Cause
BackFill()inTemplateCcMapcalled operations in the wrong order:SetCkptTs(commit_ts)— sets the flush bit (marks entry as persistent)OnFlushed()— accounting sees the entry as persistent, no dirty-count incrementSetCommitTsPayloadStatus(commit_ts, status)— overwrites the field entirely, clearing the flush bitThe entry ends up dirty with no matching dirty-count increment. When checkpoint later flushes and decrements the counter, it underflows.
Fix (in data_substrate)
template_cc_map.h—BackFill(): Reorder soSetCommitTsPayloadStatus()runs beforeSetCkptTs(), preserving the flush bit. AddOnCommittedUpdate()to reconcile stats.template_cc_map.h—RemoteReadOutsideCc(): AddOnCommittedUpdate()for consistency.cc_req_misc.cpp—UpdateCceCkptTsCc::Execute(): Relax 4 assertions that checked!IsPersistent()—BackFillcan now legitimately mark entries persistent before the checkpoint callback fires.Testing
Ran
test_add_node_with_double_write6 consecutive times after the fix — all passed with zero assertion failures:Summary by CodeRabbit