Skip to content

Fix desync between clean and evict#55

Merged
johnramsden merged 2 commits intomainfrom
invalidate-eviction
Jan 2, 2026
Merged

Fix desync between clean and evict#55
johnramsden merged 2 commits intomainfrom
invalidate-eviction

Conversation

@johnramsden
Copy link
Copy Markdown
Owner

There's a problem right now for chunk eviction. In get_evict_targets we remove from the lru and modify the priority queue accordingly.

We then take a subset of what has been modified and clean it after retrieving the relevant zones in get_clean_targets.

The problem is the priority queue can now be potentially adjusted by reads that occur, meaning the priorities can be in an inconsistent state since we do not evict them immediately.

Need to track if chunk is valid, maintain a array of list of chunks representing all zones.

Chunks should be initialized to invalid initially.

When a first write occurs it should be set to valid.

When it's removed from the lru it should be set to invalid and we update the priority queue accordingly.

If we ever do a read, but it is to an invalid chunk, that means we need to update the priority queue.

This also means we do not need to remove from the lru in get_clean_targets, saving us a lot of time. We simply remove the retain, and then in get_evict_targets, when we are removing from the lru we check if it is an invalid entry and if so we remove it and skip it. So basically our lru may end up holding invalid items but that's fine.

There's a problem right now for chunk eviction. In get_evict_targets we remove from the lru and modify the priority queue accordingly.

We then take a subset of what has been modified and clean it after retrieving the relevant zones in get_clean_targets.

The problem is the priority queue can now be potentially adjusted by reads that occur, meaning the priorities can be in an inconsistent state since we do not evict them immediately.

Need to track if chunk is valid, maintain a array of list of chunks representing all zones.

Chunks should be initialized to invalid initially.

When a first write occurs it should be set to valid.

When it's removed from the lru it should be set to invalid and we update the priority queue accordingly.

If we ever do a read, but it is to an invalid chunk, that means we need to update the priority queue.

This also means we do not need to remove from the lru in get_clean_targets, saving us a lot of time. We simply remove the retain, and then in get_evict_targets, when we are removing from the lru we check if it is an invalid entry and if so we remove it and skip it. So basically our lru may end up holding invalid items but that's fine.
@johnramsden johnramsden requested a review from 2a46m4 January 2, 2026 02:40
@johnramsden johnramsden self-assigned this Jan 2, 2026
Copy link
Copy Markdown
Collaborator

@2a46m4 2a46m4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, just minor question

@johnramsden johnramsden merged commit a0ab3c9 into main Jan 2, 2026
2 checks passed
@johnramsden
Copy link
Copy Markdown
Owner Author

Tested, working

@johnramsden johnramsden deleted the invalidate-eviction branch January 2, 2026 21:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants