PSMDB-1997: Add deferred encryption key cleanup to avoid race with checkpoint

[plebioda]

No description provided.

[Copilot]

Pull request overview

Adds infrastructure to defer deletion of per-database encryption keys (Percona KeyDB) and periodically clean up orphaned keys after verifying they are no longer referenced by any WiredTiger ident, to avoid races with checkpoint/drop-pending cleanup.

Changes:

• Added parsing helper to extract encryption.keyid from WiredTiger metadata configs, with unit tests.
• Extended the KeyDB/KVEngine API and WiredTigerKVEngine implementation to enumerate keys, delete keys, and scan WT metadata for keyIds currently in use.
• Added a timestamp-monitor-driven background cleanup in StorageEngineImpl, gated by new server parameters.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
src/mongo/db/storage/wiredtiger/wiredtiger_util.h	Declares helper to extract encryption keyId from WT config strings.
src/mongo/db/storage/wiredtiger/wiredtiger_util.cpp	Implements getEncryptionKeyId() via WiredTigerConfigParser.
src/mongo/db/storage/wiredtiger/wiredtiger_util_test.cpp	Adds coverage for keyId extraction across config variants.
src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.h	Exposes new key-management methods on the engine.
src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp	Implements key enumeration/deletion and “keys in use” scan.
src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine_encryption_key_test.cpp	Adds tests for key enumeration/deletion + server parameter toggling.
src/mongo/db/storage/wiredtiger/wiredtiger_global_options.idl	Introduces encryptionKeyCleanupDeferred + cleanup interval parameters.
src/mongo/db/storage/wiredtiger/encryption_keydb.h	Declares KeyDB enumeration API.
src/mongo/db/storage/wiredtiger/encryption_keydb.cpp	Implements getAllKeyIds() cursor scan; guards null encryptor pointers.
src/mongo/db/storage/storage_engine_impl.h	Adds cleanup listener + last-run tracking.
src/mongo/db/storage/storage_engine_impl.cpp	Implements periodic orphaned key cleanup with catalog + WT metadata checks.
src/mongo/db/storage/keydb_api.h	Extends KeyDB API surface for deferred cleanup support.
src/mongo/db/shard_role/shard_catalog/collection_catalog_helper.cpp	Removes immediate key deletion from dropDatabase path; documents deferred cleanup.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[plebioda]

The clang-tidy check fails due to bazel running out of memory:

FATAL: bazel ran out of memory and crashed. Printing stack trace:
java.lang.OutOfMemoryError: Java heap space
	at java.base/jdk.internal.misc.Unsafe.allocateUninitializedArray(Unknown Source)
	at java.base/java.lang.StringConcatHelper.newArray(Unknown Source)
	at java.base/java.lang.StringConcatHelper.simpleConcat(Unknown Source)
	at java.base/java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(DirectMethodHandle$Holder)
	at java.base/java.lang.invoke.LambdaForm$MH/0x000000080008c400.invoke(LambdaForm$MH)
	at java.base/java.lang.invoke.Invokers$Holder.linkToTargetMethod(Invokers$Holder)
	at com.google.devtools.build.lib.buildeventstream.transports.JsonFormatFileTransport.serializeEvent(JsonFormatFileTransport.java:55)
	at com.google.devtools.build.lib.buildeventstream.transports.FileTransport$$Lambda/0x00000008004b4dc0.apply(Unknown Source)
	at com.google.devtools.build.lib.buildeventstream.transports.FileTransport$SequentialWriter.run(FileTransport.java:140)
	at java.base/java.lang.Thread.runWith(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)

[igorsol]

• Why do we need encryptionKeyCleanupDeferred server parameter? And why default is false? Why not enable cleanup unconditionally?
• LOGV2: 29060-29069, 29071-74 ids are occupied by other code. Please use unoccupied range
• wiredtiger_kv_engine.cpp: remove formatting changes in code unrelated to this PR

[igorsol]

Redundant check: EncryptionKeyDB::isSpecialKeyId already excludes special key ids

[igorsol]

count() is O(n) operation. In case of thousands of idents this check might get pretty expensive.
Consider sorting both keyIds and keyIdsInUse vectors and implementing a single-pass algorithm traversing both vectors.

[igorsol]

Code review

Found 1 issue:

1. Encryption keys for dropped databases are never cleaned up under the default configuration. The PR removes the immediate keydbDropDatabase() call from dropCollectionsWithPrefix() without providing a fallback: the replacement background cleanup in _cleanupOrphanedEncryptionKeys() only runs when encryptionKeyCleanupDeferred=true, but this parameter defaults to false. As a result, all deployments that do not explicitly opt into the new parameter will silently accumulate encryption keys indefinitely after every dropDatabase.

percona-server-mongodb/src/mongo/db/shard_role/shard_catalog/collection_catalog_helper.cpp

Lines 386 to 398 in d9e50d4

	// NOTE: Encryption key cleanup is NOT performed here.
	// The key cannot be safely deleted at this point because drop-pending idents
	// (encrypted with this key) may still exist in storage and will be accessed
	// during checkpoint cleanup. Deleting the key here would cause WT_NOTFOUND errors.
	// Instead, orphaned encryption keys are cleaned up by a background process
	// (_cleanupOrphanedEncryptionKeys) that runs periodically and verifies:
	// 1. The database no longer exists in the catalog
	// 2. No storage idents (including drop-pending ones) use the key
	// This deferred cleanup is enabled via the encryptionKeyCleanupDeferred server parameter.
	return dropCollections(opCtx, toDrop, collectionNamePrefix);
	}

percona-server-mongodb/src/mongo/db/storage/storage_engine_impl.cpp

Lines 1042 to 1047 in d9e50d4

	void StorageEngineImpl::_cleanupOrphanedEncryptionKeys(OperationContext* opCtx) {
	// Check if deferred encryption key cleanup is enabled
	if (!_engine->isEncryptionKeyCleanupDeferred()) {
	return;
	}

percona-server-mongodb/src/mongo/db/storage/wiredtiger/wiredtiger_global_options.idl

Lines 302 to 313 in d9e50d4

	encryptionKeyCleanupDeferred:
	description: >-
	Encryption keys are never deleted on dropDatabase. When this parameter
	is enabled, orphaned keys are cleaned up asynchronously by a background
	process that verifies the associated database no longer exists. This
	helps avoid race conditions between key deletion and checkpoint cleanup.
	The cleanup frequency is controlled by encryptionKeyCleanupIntervalSeconds.
	set_at: [startup, runtime]
	cpp_vartype: "AtomicWord<bool>"
	cpp_varname: gEncryptionKeyCleanupDeferred
	default: false
	redact: false

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}