Wallet fixes for Knots 29.3#249
Open
luke-jr wants to merge 58 commits intobitcoinknots:29.x-knotsfrom
Open
Conversation
Github-Pull: bitcoin#31423 Rebased-From: d04f6a9
Github-Pull: bitcoin#31423 Rebased-From: 1de423e
Github-Pull: bitcoin#34215 Rebased-From: f78f6f1
Track what RestoreWallet creates so only those files and directories are removed during a failure and nothing else. Preexisting paths must be left untouched. Note: Using fs::remove_all() instead of fs::remove() in RestoreWallet does not cause any problems currently, but the change is necessary for the next commit which extends RestoreWallet to work with existing directories, which may contain files that must not be deleted. Github-Pull: bitcoin#34156 Rebased-From: 4ed0693
When migrating any legacy unnamed wallet, a failed migration would cause the cleanup logic to remove its parent directory. Since this type of legacy wallet lives directly in the main '/wallets/' folder, this resulted in unintentionally erasing all wallets, including the backup file. To be fully safe, we will no longer call `fs::remove_all`. Instead, we only erase the individual db files we have created, leaving everything else intact. The created wallets parent directories are erased only if they are empty. As part of this last change, `RestoreWallet` was modified to allow an existing directory as the destination, since we no longer remove the original wallet directory (we only remove the files we created inside it). This also fixes the restore of top-level default wallets during failures, which were failing due to the directory existence check that always returns true for the /wallets/ directory. This bug started after: bitcoin@f6ee59b Previously, the `fs::copy_file` call was failing for top-level wallets, which prevented the `fs::remove_all` call from being reached. Github-Pull: bitcoin#34156 Rebased-From: f4c7e28
Verifies that a failed migration of the unnamed (default) wallet does not erase the main /wallets/ directory, and also that the backup file exists. Github-Pull: bitcoin#34156 Rebased-From: 36093bd
…rune failure The first test verifies that restoring into an existing empty directory or a directory with no .dat db files succeeds, while restoring into a dir with a .dat file fails. The second test covers restoring into the default unnamed wallet (wallet.dat), which also implicitly exercises the recovery path used after a failed migration. The third test covers failure during restore on a prune node. When the wallet last sync was beyond the pruning height. Github-Pull: bitcoin#34156 Rebased-From: f011e0f
Right now, after migration the last message users see is "migration completed", but the migration isn't actually finished yet. We still need to load the new wallets to ensure consistency, and if that fails, the migration will be rolled back. This can be confusing for users. This change logs the post-migration loading step and if a wallet fails to load and the migration will be rolled back. Github-Pull: bitcoin#34156 Rebased-From: d70b159
Because the default wallet has no name, the watch-only and solvables wallets created during migration end up having no name either. This fixes it by applying the same prefix name we use for the backup file for an unnamed default wallet. Before: watch-only wallet named "_watchonly" After: watch-only wallet named "default_wallet_watchonly" Github-Pull: bitcoin#34156 Rebased-From: 82caa81
…eight Github-Pull: bitcoin#34156 Rebased-From: b7c34d0
Refactor a common way to perform the failed migration test that exists for default wallets, and add relative-path wallets and absolute-path wallets. Github-Pull: 34226 Rebased-From: eeaf28d
This check ensures that when migrating a legacy wallet with a direct filename, the backup file is named as expected. Co-authored-by: Ava Chow <github@achow101.com> Github-Pull: bitcoin#32273 Rebased-From: e22c359
Co-authored-by: Russell Yanofsky <russ@yanofsky.org> Github-Pull: bitcoin#32273 Rebased-From: 70f1c99
Github-Pull: bitcoin#32273 Rebased-From: 63c6d36
Github-Pull: bitcoin#32273 Rebased-From: 41faef5
Github-Pull: bitcoin#32273 Rebased-From: f0bb3d5
Github-Pull: bitcoin#32273 Rebased-From: 76fe0e5
…allet If any other files exist in the directory, we cannot assume the sharable files are exclusively for this wallet. But if they are, this also cleans up other log.* files
…ils, log the correct wallet name in error message
…eason Since we no longer delete the wallet directory, there's no need to vacate it The moving only served to risk errors by crossing filesystem boundaries (which fs::rename can't handle)
…ating from non-directory While 30.x+ keep backup files in walletdir, 29.x places them in the migrated wallet directory
…et, but not with a shared db environment
Github-Pull: bitcoin#32984 Rebased-From: 060695c
Github-Pull: bitcoin#34226 Rebased-From: eeaf28d
…node_fails Github-Pull: bitcoin#34156 Rebased-From: b7c34d0
Github-Pull: bitcoin#34176 Rebased-From: 504e214
Previously, wallets in non-writable directories were loaded, leading to crashes on any subsequent write. Github-Pull: bitcoin#34176 Rebased-From: 2ef6add
Wallet creation/opening errors turn into RPC_MISC_ERROR without this
Github-Pull: bitcoin#31423 Rebased-From: e86d71b
…legacy wallet Currently, the migration process creates a brand-new descriptor wallet with no connection to the user's legacy wallet when the legacy wallet lacks key material and contains only watch-only scripts. This behavior is not aligned with user expectations. If the legacy wallet contains only watch-only scripts, the migration process should only generate a watch-only wallet instead. Github-Pull: bitcoin#31423 Rebased-From: b789907
After a wallet is migrated and we are trying to load it, if it could not be loaded, don't try to set the wallet name. Github-Pull: bitcoin#32984 Rebased-From: 8a4cfdd
- Comment updates - Use fs::remove when deleting empty migration target - Adapt tests to only having watchonly migration output where applicable - break out of wallet reload loop when any fail Originally-From: f4c7e28 and d70b159 and 82caa81 (bitcoin#34156)
…point and lsn_reset
…atabase" subdirectory Otherwise, there's a race condition if another program/instance opens a wallet before we delete it
…tdown It's hypothetically possible, albeit difficult to arrange, that other non-loaded wallets are still relying on it
…vironment::Close() This reverts commit 8a06c8cf55f67934a4d86732217e38e42ac5198e.
… BerkeleyEnvironment::Flush
luke-jr
force-pushed
the
knots29.3_walletfixes
branch
from
b8561d2 to
cf43a63
Compare
kwsantiago
reviewed
Jan 26, 2026
| std::vector<fs::path> Files() override | ||
| { | ||
| std::vector<fs::path> files; | ||
| files.emplace_back(m_dir_path / fs::PathFromString(m_file_path)); |
There was a problem hiding this comment.
m_file_path is already the full absolute path (used directly for sqlite3_open_v2). The / operator with an absolute RHS ignores the LHS. Consider using fs::PathFromString(m_file_path) directly.
kwsantiago
reviewed
Jan 26, 2026
| }; | ||
| try { | ||
| if (build_files_list(files, env, m_filename)) return files; | ||
| } catch (...) { |
There was a problem hiding this comment.
Catching all exceptions silently could mask filesystem errors and lead to incomplete cleanup. Consider logging when falling back to the smaller file list.
kwsantiago
reviewed
Jan 26, 2026
| // In case we're migrating from file to directory, move the backup into it | ||
| this_wallet_dir = fs::absolute(fs::PathFromString(local_wallet->GetDatabase().Filename())).parent_path(); | ||
| backup_path = this_wallet_dir / backup_filename; | ||
| fs::rename(res.backup_path, backup_path); |
There was a problem hiding this comment.
If rename fails (e.g., cross-device move), this throws after migration is complete, leaving backup in unexpected location. Consider copy+remove fallback or try-catch.
Collaborator
Author
There was a problem hiding this comment.
Since the directory was just created, it should never be cross-device?
kwsantiago
reviewed
Jan 26, 2026
|
|
||
| // Second, delete the created wallet directories and nothing else. They must be empty at this point. | ||
| for (const fs::path& dir : wallet_empty_dirs_to_remove) { | ||
| Assume(fs::is_empty(dir)); |
There was a problem hiding this comment.
Assume() may not halt in release builds. If directory is not empty, fs::remove() will silently fail, leaving orphaned directories.
kwsantiago
reviewed
Jan 26, 2026
| // Clean up the parent directory if we created it during restoration. | ||
| // As we have created it, it must be empty after deleting the wallet file. | ||
| if (created_parent_dir) { | ||
| Assume(fs::is_empty(wallet_path)); |
There was a problem hiding this comment.
Same Assume() pattern - may not halt in release builds if directory is not empty.
Collaborator
Author
|
Merged in v29.3.knots20260210 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a cherry-picking of multiple fix branches (including Core 29.3 backports) for review purposes