PSMDB-1997: Add deferred encryption key cleanup to avoid race with checkpoint#1786
Open
PSMDB-1997: Add deferred encryption key cleanup to avoid race with checkpoint#1786
Conversation
There was a problem hiding this comment.
Pull request overview
Adds infrastructure to defer deletion of per-database encryption keys (Percona KeyDB) and periodically clean up orphaned keys after verifying they are no longer referenced by any WiredTiger ident, to avoid races with checkpoint/drop-pending cleanup.
Changes:
- Added parsing helper to extract
encryption.keyidfrom WiredTiger metadata configs, with unit tests. - Extended the KeyDB/KVEngine API and WiredTigerKVEngine implementation to enumerate keys, delete keys, and scan WT metadata for keyIds currently in use.
- Added a timestamp-monitor-driven background cleanup in
StorageEngineImpl, gated by new server parameters.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| src/mongo/db/storage/wiredtiger/wiredtiger_util.h | Declares helper to extract encryption keyId from WT config strings. |
| src/mongo/db/storage/wiredtiger/wiredtiger_util.cpp | Implements getEncryptionKeyId() via WiredTigerConfigParser. |
| src/mongo/db/storage/wiredtiger/wiredtiger_util_test.cpp | Adds coverage for keyId extraction across config variants. |
| src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.h | Exposes new key-management methods on the engine. |
| src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp | Implements key enumeration/deletion and “keys in use” scan. |
| src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine_encryption_key_test.cpp | Adds tests for key enumeration/deletion + server parameter toggling. |
| src/mongo/db/storage/wiredtiger/wiredtiger_global_options.idl | Introduces encryptionKeyCleanupDeferred + cleanup interval parameters. |
| src/mongo/db/storage/wiredtiger/encryption_keydb.h | Declares KeyDB enumeration API. |
| src/mongo/db/storage/wiredtiger/encryption_keydb.cpp | Implements getAllKeyIds() cursor scan; guards null encryptor pointers. |
| src/mongo/db/storage/storage_engine_impl.h | Adds cleanup listener + last-run tracking. |
| src/mongo/db/storage/storage_engine_impl.cpp | Implements periodic orphaned key cleanup with catalog + WT metadata checks. |
| src/mongo/db/storage/keydb_api.h | Extends KeyDB API surface for deferred cleanup support. |
| src/mongo/db/shard_role/shard_catalog/collection_catalog_helper.cpp | Removes immediate key deletion from dropDatabase path; documents deferred cleanup. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine_encryption_key_test.cpp
Outdated
Show resolved
Hide resolved
b2b3f91 to
ef48a8e
Compare
…eckpoint This commit implements deferred encryption key cleanup to prevent race conditions between encryption key deletion and checkpoint cleanup during dropDatabase operations.
Add unit tests for the new encryption key management functionality:
Add two test files and corresponding test suites for verifying deferred encryption key cleanup functionality: - deferred_key_cleanup_drop_database.js: Tests basic database create/drop operations with deferred cleanup enabled - deferred_key_cleanup_stress.js: Stress test with 16 parallel threads performing 200 iterations each of database create/drop Test suites run with both AES256-GCM and AES256-CBC cipher modes. The stress test uses db.getSiblingDB() instead of conn.getDB() to ensure the database connection works correctly when the worker function is serialized and executed in a parallel shell context.
Fix Python version to 3.13 to match pyproject.toml requirements.
fea52cd to
765829e
Compare
Author
|
The |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.