Skip to content

Conversation

Copy link

Copilot AI commented Nov 17, 2025

When transaction queue or memory limits are exceeded, CDeltaWriter::addToQueue was performing inline synchronous saves with complex lock orchestration. This refactoring delegates all saves to the async writer thread for consistency.

Changes

Added belowLimitsSem semaphore

  • Signals when queue drops below limits after async writer completes save
  • Enables waiting threads to resume when capacity is available

Modified addToQueue method

  • Replaced inline save(pending) with requestAsyncWrite() + semaphore wait
  • Removed blockedSaveCrit and inline lock management from limit-triggered path
  • Added 10s timeout loop with warning for long waits

Updated threadmain method

  • Signals belowLimitsSem after save when queue is below both transactionMaxMem and transactionQueueLimit

Updated stop() method

  • Signals belowLimitsSem to prevent deadlock on shutdown

Before/After

// Before: inline synchronous save
if (needsSyncSave) {
    CHECKEDCRITICALBLOCK(blockedSaveCrit, fakeCritTimeout);
    CriticalBlock b(pendingCrit);
    if (items && save(pending)) {
        pendingSz = 0;
        // ...
    }
}

// After: trigger async and wait
if (needsSyncSave) {
    {
        CriticalBlock b(pendingCrit);
        if (!writeRequested)
            requestAsyncWrite();
    }
    while (!belowLimitsSem.wait(10000)) {
        if (aborted) return;
        WARNLOG("Waiting on CDeltaWriter to clear queue below limits");
    }
}

All delta writes now flow through the async writer thread, eliminating dual save paths and simplifying lock ordering requirements.

Original prompt

Look at dali/base/dasds.cpp, and the CDeltaWriter implementation, and in particular the addToQueue handling when needsSyncSave is triggered.
Instead of handling the save inline in this function, it should be refactored so it triggers a save (as it would if less than thresholds) and then wait on a Semaphore until writing thread (CDeltaWriter::threadmain) clears down below the limits and signals it.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Instead of performing inline synchronous saves when needsSyncSave is triggered,
the code now:
- Triggers an async write via requestAsyncWrite()
- Waits on belowLimitsSem semaphore until writer thread clears queue below limits
- Writer thread signals belowLimitsSem after save when queue is below thresholds
- Added belowLimitsSem.signal() in stop() to prevent deadlock on shutdown

Co-authored-by: jakesmith <902700+jakesmith@users.noreply.github.com>
Copilot AI changed the title [WIP] Refactor addToQueue handling for sync save triggering Refactor CDeltaWriter::addToQueue to use async writer thread for limit-triggered saves Nov 17, 2025
Copilot AI requested a review from jakesmith November 17, 2025 19:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants