Open
Conversation
Add reorg detection when RPC reports a tip at or below already-processed heights: BlockIndexer.onBlockChange now detects height regressions, calls onHeightRegressionDetected which reverts the chain and restarts tasks. ReorgWatchdog gains an additional canonical-hash check and restores the blockchain on mismatch. ChainSynchronisation can fetch header-only blocks in resync mode to avoid downloading full tx data. PoC improves blockProcessedLock handling to avoid lock jams and logs failures; BlockWitnessManager adds an onBlockWitness path and deduplicates witness handling. Misc: reduce WITNESS max instances to 2, adjust VMStorage method signature formatting, and minor comment/whitespace cleanups. Add extensive tests covering reorg edge cases, watchdog behavior, fetcher/watchBlockChanges, and witness thread behavior.
BlobMaster41
added
bug
Contributor
There was a problem hiding this comment.
Pull request overview
This PR hardens reorg handling across the indexer by detecting height regressions / same-height competing blocks, restoring canonical state when mismatches are found, and reducing unnecessary data fetching during resyncs. It also adjusts witness-thread behavior and expands the test suite around these edge cases.
Changes:
- Add height-regression detection in
BlockIndexer.onBlockChangeand same-height canonical hash verification inReorgWatchdog.verifyChainReorgForBlock. - Add a resync “header-only” fetch path in
ChainSynchronisationto avoid downloading full transaction data. - Improve PoC witness dispatch locking/error handling, adjust witness-thread scaling, and add/extend reorg + witness tests.
Reviewed changes
Copilot reviewed 26 out of 26 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/witness/WitnessThread.test.ts | Test refactors/formatting; import ordering and mocking adjustments. |
| tests/reorg/watchdog/sameHeightReorgEdgeCases.test.ts | New watchdog edge-case coverage for same-height hash comparisons and restore behavior. |
| tests/reorg/watchdog/sameHeightReorg.test.ts | New “same-height reorg” tests validating canonical-hash checks. |
| tests/reorg/watchdog/reorgDetection.test.ts | Updates existing watchdog tests to use onBlockChange + Reflect helpers for private methods. |
| tests/reorg/sync/queryBlockResync.test.ts | New tests describing header-only resync “contract” behavior. |
| tests/reorg/purge/mempoolPreservation.test.ts | Minor comment tweak. |
| tests/reorg/fetcher/watchBlockChangesReorg.test.ts | New tests for RPC block-change notifications and reorg-adjacent behaviors. |
| tests/reorg/edge-cases/integration.test.ts | Minor comment tweaks in integration edge cases. |
| tests/reorg/blockindexer/startupPurge.test.ts | Comment tweaks around startup purge behavior. |
| tests/reorg/blockindexer/onBlockChangeReorg.test.ts | New tests describing prior BlockIndexer reorg-detection gaps. |
| tests/reorg/blockindexer/onBlockChangeEdgeCases.test.ts | New tests covering regression detection guard/boundary cases. |
| src/src/vm/storage/databases/VMMongoStorage.ts | Minor comment change in purge path. |
| src/src/vm/storage/VMStorage.ts | Signature formatting change for revertDataUntilBlock. |
| src/src/services/ServicesConfigurations.ts | Reduce WITNESS thread max instances from 4 → 2. |
| src/src/poc/witness/WitnessThreadManager.ts | Clarify thread-linking comment. |
| src/src/poc/witness/WitnessThread.ts | Minor comment tweak. |
| src/src/poc/witness/WitnessSerializer.ts | Import/function signature formatting changes. |
| src/src/poc/networking/p2p/BlockWitnessManager.ts | Reorders/deduplicates onBlockWitness placement. |
| src/src/poc/mempool/manager/Mempool.ts | Minor comment tweak. |
| src/src/poc/PoC.ts | Prevent blockProcessedLock from jamming on rejection; log dispatch failures. |
| src/src/blockchain-indexer/sync/classes/ChainSynchronisation.ts | Add header-only queryBlockHeaderOnly for resync mode. |
| src/src/blockchain-indexer/processor/reorg/ReorgWatchdog.ts | Add same-height canonical hash mismatch detection + restore. |
| src/src/blockchain-indexer/processor/BlockIndexer.ts | Add height regression detection + revert/restart path. |
| src/src/api/Server.ts | Add CORS maxAge for preflight caching. |
| src/src/Core.ts | Import formatting cleanup. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Add a comprehensive BlockIndexer height-regression test covering the full revert flow (tests/reorg/blockindexer/heightRegression.test.ts). Remove legacy/overlapping tests that focused on vulnerable behaviors (tests/reorg/blockindexer/onBlockChangeReorg.test.ts and tests/reorg/sync/queryBlockResync.test.ts). Update RPCBlockFetcher tests to emphasize hash-based change detection and error recovery, and add a rapid same-height hash-flip scenario. Clean up ReorgWatchdog same-height test by removing large-gap performance cases and minor formatting changes. Overall this reorganizes and refocuses reorg-related tests toward explicit revert behavior and hash-change detection.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR strengthens deep reorg handling across the indexer pipeline by detecting same-height/height-regression reorgs earlier, restoring canonical chain state when hashes diverge, and expanding test coverage around these edge cases.
Changes:
- Add height-regression detection in
BlockIndexer.onBlockChange()and canonical-hash verification inReorgWatchdog.verifyChainReorgForBlock(). - Add resync “header-only” fetch path in
ChainSynchronisationplus PoC/witness-thread robustness tweaks (lock handling, witness handling, thread instance limits). - Add/extend Vitest suites covering watchdog/indexer/fetcher/witness reorg edge cases and behavior.
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/witness/WitnessThread.test.ts | Adjusts witness thread tests/mocking layout and formatting. |
| tests/reorg/watchdog/sameHeightReorgEdgeCases.test.ts | New edge-case suite for same-height hash comparison and restore behavior. |
| tests/reorg/watchdog/sameHeightReorg.test.ts | New “same-height reorg” regression tests. |
| tests/reorg/watchdog/reorgDetection.test.ts | Refactors tests to use onBlockChange and reflective helpers for private methods. |
| tests/reorg/purge/mempoolPreservation.test.ts | Minor comment update. |
| tests/reorg/fetcher/watchBlockChangesReorg.test.ts | New tests asserting hash-based change detection and polling recovery. |
| tests/reorg/edge-cases/integration.test.ts | Minor comment updates around batching expectations. |
| tests/reorg/blockindexer/startupPurge.test.ts | Minor comment updates for startup purge behavior. |
| tests/reorg/blockindexer/onBlockChangeEdgeCases.test.ts | New tests for onBlockChange regression detection guards and boundaries. |
| tests/reorg/blockindexer/heightRegression.test.ts | New tests for full revert flow on detected height regression. |
| src/src/vm/storage/databases/VMMongoStorage.ts | Comment tweak around target epoch purging. |
| src/src/vm/storage/VMStorage.ts | Signature formatting for revertDataUntilBlock. |
| src/src/services/ServicesConfigurations.ts | Reduces WITNESS thread max instances from 4 → 2. |
| src/src/poc/witness/WitnessThreadManager.ts | Documents RPC link being established from the RPC manager side. |
| src/src/poc/witness/WitnessThread.ts | Minor comment tweak around buffering/flush behavior. |
| src/src/poc/witness/WitnessSerializer.ts | Import formatting + sync response reconstruction signature formatting. |
| src/src/poc/networking/p2p/BlockWitnessManager.ts | Moves/keeps onBlockWitness path and witness queue handling. |
| src/src/poc/mempool/manager/Mempool.ts | Comment tweak for sequential broadcast path. |
| src/src/poc/PoC.ts | Prevents blockProcessedLock from jamming on failures; logs broadcast/dispatch failures. |
| src/src/blockchain-indexer/sync/classes/ChainSynchronisation.ts | Adds header-only fetch in resync mode (queryBlockHeaderOnly). |
| src/src/blockchain-indexer/processor/reorg/ReorgWatchdog.ts | Adds same-height canonical block-hash mismatch detection and restore call. |
| src/src/blockchain-indexer/processor/BlockIndexer.ts | Adds height regression detection & onHeightRegressionDetected() revert/restart path. |
| src/src/api/Server.ts | Adds CORS maxAge. |
| src/src/Core.ts | Import formatting cleanup. |
| package.json | Bumps @btc-vision/transaction version. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+345
to
+351
| // In resync mode, only download block headers, no transaction data needed. | ||
| // Transactions are preserved from the original sync; only headers/witnesses | ||
| // are re-generated. | ||
| if (Config.DEV.RESYNC_BLOCK_HEIGHTS) { | ||
| return this.queryBlockHeaderOnly(blockNumber); | ||
| } | ||
|
|
Comment on lines
+6
to
+7
| // First, let's get the class itself: | ||
| import { WitnessThread } from '../../src/src/poc/witness/WitnessThread.js'; |
| } from '../networking/protobuf/packets/blockchain/common/BlockHeaderWitness.js'; | ||
| import { ISyncBlockHeaderResponse } from '../networking/protobuf/packets/blockchain/responses/SyncBlockHeadersResponse.js'; | ||
| import { | ||
| ISyncBlockHeaderResponse |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add reorg detection when RPC reports a tip at or below already-processed heights: BlockIndexer.onBlockChange now detects height regressions, calls onHeightRegressionDetected which reverts the chain and restarts tasks. ReorgWatchdog gains an additional canonical-hash check and restores the blockchain on mismatch. ChainSynchronisation can fetch header-only blocks in resync mode to avoid downloading full tx data. PoC improves blockProcessedLock handling to avoid lock jams and logs failures; BlockWitnessManager adds an onBlockWitness path and deduplicates witness handling. Misc: reduce WITNESS max instances to 2, adjust VMStorage method signature formatting, and minor comment/whitespace cleanups. Add extensive tests covering reorg edge cases, watchdog behavior, fetcher/watchBlockChanges, and witness thread behavior.
Type of Change
Checklist
Build & Tests
npm installcompletes without errorsnpm run buildcompletes without errorsnpm testpasses all testsCode Quality
Documentation
Security
OP_NET Node Specific
Testing
Consensus Impact
Related Issues
By submitting this PR, I confirm that my contribution is made under the terms of the project's license.