test: reorganize a real chain end to end, and fix what that found - #578
Conversation
|
Warning Review limit reached
Next review available in: 41 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (21)
📝 WalkthroughWalkthroughThe PR adds synchronized header-tip adoption, asynchronous chain reorganization, active-chain header persistence, transactional header replacement, active-chain storage checks, corrected P2PKH unlocking scripts, and end-to-end reorganization tests. ChangesReorganization synchronization
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant Orchestrator
participant ExecuteReorg
participant BlockChain
participant HeaderOrganizer
Orchestrator->>ExecuteReorg: request reorganization
ExecuteReorg->>BlockChain: pause writers and wait for barrier
ExecuteReorg->>BlockChain: switch to branch head
BlockChain-->>ExecuteReorg: return switch result
ExecuteReorg->>HeaderOrganizer: adopt new tip
ExecuteReorg->>BlockChain: replace active headers
ExecuteReorg-->>Orchestrator: return reorg outcome
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (2)
src/node/test/reorg_cycle.cpp (1)
129-133: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAssert that the tasks finished before the timeout expires.
run_forreturns when the work completes or when 120 seconds elapse. The test does not distinguish these cases here. If the tasks stall, the failure surfaces later as a wrong height and hides the real cause. Add an explicit progress check afterrun_for.♻️ Proposed refactor
ctx.run_for(std::chrono::seconds(120)); + + // Fail at the point of the stall, not at a later height assertion. + auto const built = chain.get_utxo_built_height(); + REQUIRE(built); + REQUIRE(*built >= end_height);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/node/test/reorg_cycle.cpp` around lines 129 - 133, Update the test flow around ctx.run_for in reorg_cycle so it explicitly verifies that all tasks completed before the timeout, using the progress/completion state exposed by the test context. Fail immediately when the 120-second deadline expires, then retain the output draining loop for completed tasks.src/blockchain/test/regtest_miner.hpp (1)
146-153: 🩺 Stability & Availability | 🔵 Trivial | 💤 Low valueBound the nonce loop and hash the header only.
The loop has no exit other than a valid proof of work.
noncewraps atUINT32_MAX, so a failure to find a solution becomes a silent infinite loop instead of a test failure. The loop also copies the full transaction list on each attempt, which is unnecessary because only the nonce changes. Mutate the header of a single block and add a bound.♻️ Proposed refactor
- for (uint32_t nonce = 0; ; ++nonce) { + for (uint64_t nonce = 0; nonce <= std::numeric_limits<uint32_t>::max(); ++nonce) { domain::chain::block attempt( - domain::chain::header{0x20000000, prev, merkle, timestamp, regtest_bits, nonce}, + domain::chain::header{0x20000000, prev, merkle, timestamp, regtest_bits, uint32_t(nonce)}, domain::chain::transaction::list(candidate.transactions())); if (attempt.header().is_valid_proof_of_work(attempt.hash(), /*retarget*/ false)) { return attempt; } } + throw std::runtime_error("regtest_miner: no nonce satisfied the regtest target");🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/blockchain/test/regtest_miner.hpp` around lines 146 - 153, Update the mining loop around the regtest block construction to reuse one block/header and mutate only its nonce, avoiding reconstruction and transaction-list copies on each attempt. Bound nonce attempts so exhaustion returns a test failure instead of wrapping indefinitely, while validating proof of work against the updated header hash only.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/blockchain/test/regtest_miner.hpp`:
- Around line 111-119: Validate the result of create_endorsement before
dereferencing it in the signed-input construction. If the expected is
unsuccessful, report or propagate its error using the test’s existing
failure-handling convention; only pass the endorsement value to
to_pay_public_key_hash_pattern_unlocking after confirming has_value().
In `@src/node/src/sync/orchestrator.cpp`:
- Around line 888-890: After execute_reorg establishes a fork by making a side
branch active, subsequent request construction paths incorrectly convert
headers_synced_to directly to header indices without accounting for the new
active chain. Replace the direct index conversions of headers_synced_to with
active_at() lookups at each get_hash() call site (around lines 952, 982, 1005,
1070, 1124, and 1206) to resolve the correct entry indices within the now-active
branch, and handle the case where active_at() returns null_hash.
In `@src/node/src/sync/reorg.cpp`:
- Around line 29-49: Use an RAII guard immediately after
chain.request_reorg_pause(true) in the reorganization coroutine to ensure
request_reorg_pause(false) runs during normal completion, abort returns, and
exception unwinding. Remove the manual release calls and preserve the existing
abort logging and outcome handling.
In `@src/node/test/reorg_cycle.cpp`:
- Line 302: Update the test assertion around index.find(a101.hash()) to first
require that the lookup result is not header_index::null_index, then pass the
validated index to has_block_data. Keep the existing block-data assertion after
the explicit lookup check.
- Around line 224-225: Update the assertions around chain.get_last_heights() and
chain.get_utxo_built_height() to check each optional has a value before
dereferencing it, matching the safe assertion pattern used near lines 195-202.
Apply the same guarded form to the unchecked assertions near lines 283 and
290-291, preserving the expected height checks.
---
Nitpick comments:
In `@src/blockchain/test/regtest_miner.hpp`:
- Around line 146-153: Update the mining loop around the regtest block
construction to reuse one block/header and mutate only its nonce, avoiding
reconstruction and transaction-list copies on each attempt. Bound nonce attempts
so exhaustion returns a test failure instead of wrapping indefinitely, while
validating proof of work against the updated header hash only.
In `@src/node/test/reorg_cycle.cpp`:
- Around line 129-133: Update the test flow around ctx.run_for in reorg_cycle so
it explicitly verifies that all tasks completed before the timeout, using the
progress/completion state exposed by the test context. Fail immediately when the
120-second deadline expires, then retain the output draining loop for completed
tasks.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: ffdadbea-7a82-4ca2-a987-cd971ab140cb
⛔ Files ignored due to path filters (1)
src/database/include/kth/database/databases/header_database.ippis excluded by!**/*.ippand included bysrc/database/**
📒 Files selected for processing (10)
src/blockchain/include/kth/blockchain/pools/header_organizer.hppsrc/blockchain/src/pools/header_organizer.cppsrc/blockchain/test/regtest_miner.hppsrc/domain/src/chain/script.cppsrc/node/CMakeLists.txtsrc/node/include/kth/node/sync/reorg.hppsrc/node/src/sync/block_tasks.cppsrc/node/src/sync/orchestrator.cppsrc/node/src/sync/reorg.cppsrc/node/test/reorg_cycle.cpp
00c1865 to
dcf32ad
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
src/node/include/kth/node/sync/reorg.hpp (1)
37-68: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winMove the
execute_reorgdocumentation next to its declaration.The block at Lines 37-51 documents
execute_reorg, but thepersist_active_headersblock (Lines 52-62) and its declaration (Lines 63-68) come between them. A reader attaches the first block topersist_active_headers.Move Lines 37-51 to directly above Line 70.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/node/include/kth/node/sync/reorg.hpp` around lines 37 - 68, Move the documentation block describing the reorganization sequence, abort behavior, and chain switching directly above the execute_reorg declaration. Keep the persist_active_headers documentation immediately above persist_active_headers, so each comment is adjacent to its corresponding declaration.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/blockchain/include/kth/blockchain/interface/block_chain.hpp`:
- Around line 151-154: Guard the truncate_headers_from declaration in
block_chain.hpp and its definition in block_chain.cpp with the matching `#if`
!defined(KTH_DB_READONLY) guard, alongside the database truncate_headers_from
availability. Update both affected sites: the header declaration and the
block_chain.cpp implementation; no other behavior needs to change.
In `@src/node/src/sync/reorg.cpp`:
- Around line 113-132: Guard the header truncation path in the reorg flow before
converting new_tip_height to size_t: reject non-positive values and do not call
chain.truncate_headers_from when active_tip_height() is -1 or lower. Preserve
the existing truncation and error logging for positive tip heights, ensuring the
requested truncation height remains at least 1.
---
Nitpick comments:
In `@src/node/include/kth/node/sync/reorg.hpp`:
- Around line 37-68: Move the documentation block describing the reorganization
sequence, abort behavior, and chain switching directly above the execute_reorg
declaration. Keep the persist_active_headers documentation immediately above
persist_active_headers, so each comment is adjacent to its corresponding
declaration.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: c0d41180-8373-494d-90ba-dcda77d4f367
⛔ Files ignored due to path filters (1)
src/database/include/kth/database/databases/header_database.ippis excluded by!**/*.ippand included bysrc/database/**
📒 Files selected for processing (17)
src/blockchain/include/kth/blockchain/interface/block_chain.hppsrc/blockchain/include/kth/blockchain/pools/header_organizer.hppsrc/blockchain/src/interface/block_chain.cppsrc/blockchain/src/pools/header_organizer.cppsrc/blockchain/test/header_organizer.cppsrc/blockchain/test/regtest_miner.hppsrc/database/include/kth/database/data_base.hppsrc/database/include/kth/database/databases/internal_database.hppsrc/database/src/data_base.cppsrc/domain/src/chain/script.cppsrc/domain/test/chain/script.cppsrc/node/CMakeLists.txtsrc/node/include/kth/node/sync/reorg.hppsrc/node/src/sync/block_tasks.cppsrc/node/src/sync/orchestrator.cppsrc/node/src/sync/reorg.cppsrc/node/test/reorg_cycle.cpp
🚧 Files skipped from review as they are similar to previous changes (5)
- src/domain/src/chain/script.cpp
- src/blockchain/test/regtest_miner.hpp
- src/node/src/sync/block_tasks.cpp
- src/node/test/reorg_cycle.cpp
- src/node/CMakeLists.txt
e12a847 to
f8922dc
Compare
There was a problem hiding this comment.
🧹 Nitpick comments (3)
src/node/src/sync/reorg.cpp (1)
91-100: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winBound the barrier wait.
The loop exits only when the barrier is reached or
abortreturns true. A participant that registers and never parks keeps the pause raised, and every registered writer stays stopped. Add a deadline: after it expires, log the parked/registered counts and return a failed outcome instead of waiting further.♻️ Proposed deadline
+ constexpr auto barrier_timeout = std::chrono::seconds(60); + auto const deadline = std::chrono::steady_clock::now() + barrier_timeout; while ( ! chain.reorg_barrier_reached() && ! abort()) { + if (std::chrono::steady_clock::now() >= deadline) { + spdlog::error("[reorg] The barrier was not reached within {}s; the chain was not touched", + barrier_timeout.count()); + co_return reorg_outcome{}; + } ::asio::steady_timer wait_timer(executor); wait_timer.expires_after(std::chrono::milliseconds(50)); co_await wait_timer.async_wait(::asio::as_tuple(::asio::use_awaitable)); }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/node/src/sync/reorg.cpp` around lines 91 - 100, Bound the barrier-wait loop in the reorg coroutine so it cannot wait indefinitely when the barrier is never reached. Add a deadline alongside the existing abort handling; when it expires, log the current parked and registered counts, then return a failed reorg_outcome without touching the chain. Preserve the existing successful barrier and abort paths.src/database/src/data_base.cpp (1)
292-298: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick winDefine and enforce the degenerate inputs of the header-replacement API. The new replacement path accepts an empty
headerslist and astart_heightof 0. Neither value has a defined outcome: the removal range is expressed relative to the last supplied header, and height 0 holds genesis.
src/database/src/data_base.cpp#L292-L298: returnerror::successfor an emptyheaderslist, aspush_headers_batchdoes, and rejectstart_height == 0before the call tointernal_db_->replace_headers_from.src/database/include/kth/database/databases/internal_database.hpp#L113-L128: state the behavior for an emptyheaderslist in the contract, next to the existingstart_heightrequirement.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/database/src/data_base.cpp` around lines 292 - 298, Update data_base::replace_headers_from in src/database/src/data_base.cpp: return error::success immediately when headers is empty, and reject start_height == 0 before calling internal_db_->replace_headers_from. Update the contract near the existing start_height requirement in src/database/include/kth/database/databases/internal_database.hpp to explicitly document the empty-headers behavior.src/node/include/kth/node/sync/reorg.hpp (1)
55-90: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueMove the
execute_reorgdocumentation next to its declaration.Lines 55-69 describe
execute_reorg, butstruct reorg_outcomesits between them and the declaration at line 85. Line 70 starts thereorg_outcomedescription with no blank line, so the two blocks read as one. Move theexecute_reorgparagraphs to just above line 85 and keep thereorg_outcomeparagraph with the struct.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/node/include/kth/node/sync/reorg.hpp` around lines 55 - 90, Separate the documentation blocks in the reorg declarations: keep the “What a switch left behind...” comment immediately above struct reorg_outcome, and move the execute_reorg workflow and abort-behavior paragraphs to immediately above the execute_reorg declaration. Add the necessary blank-line separation so each comment clearly documents its corresponding symbol.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@src/database/src/data_base.cpp`:
- Around line 292-298: Update data_base::replace_headers_from in
src/database/src/data_base.cpp: return error::success immediately when headers
is empty, and reject start_height == 0 before calling
internal_db_->replace_headers_from. Update the contract near the existing
start_height requirement in
src/database/include/kth/database/databases/internal_database.hpp to explicitly
document the empty-headers behavior.
In `@src/node/include/kth/node/sync/reorg.hpp`:
- Around line 55-90: Separate the documentation blocks in the reorg
declarations: keep the “What a switch left behind...” comment immediately above
struct reorg_outcome, and move the execute_reorg workflow and abort-behavior
paragraphs to immediately above the execute_reorg declaration. Add the necessary
blank-line separation so each comment clearly documents its corresponding
symbol.
In `@src/node/src/sync/reorg.cpp`:
- Around line 91-100: Bound the barrier-wait loop in the reorg coroutine so it
cannot wait indefinitely when the barrier is never reached. Add a deadline
alongside the existing abort handling; when it expires, log the current parked
and registered counts, then return a failed reorg_outcome without touching the
chain. Preserve the existing successful barrier and abort paths.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: a5ab31d3-2702-4eb8-a3ad-b9caea67a255
⛔ Files ignored due to path filters (1)
src/database/include/kth/database/databases/header_database.ippis excluded by!**/*.ippand included bysrc/database/**
📒 Files selected for processing (17)
src/blockchain/include/kth/blockchain/interface/block_chain.hppsrc/blockchain/include/kth/blockchain/pools/header_organizer.hppsrc/blockchain/src/interface/block_chain.cppsrc/blockchain/src/pools/header_organizer.cppsrc/blockchain/test/header_organizer.cppsrc/blockchain/test/regtest_miner.hppsrc/database/include/kth/database/data_base.hppsrc/database/include/kth/database/databases/internal_database.hppsrc/database/src/data_base.cppsrc/domain/src/chain/script.cppsrc/domain/test/chain/script.cppsrc/node/CMakeLists.txtsrc/node/include/kth/node/sync/reorg.hppsrc/node/src/sync/block_tasks.cppsrc/node/src/sync/orchestrator.cppsrc/node/src/sync/reorg.cppsrc/node/test/reorg_cycle.cpp
🚧 Files skipped from review as they are similar to previous changes (8)
- src/node/CMakeLists.txt
- src/blockchain/test/header_organizer.cpp
- src/node/src/sync/block_tasks.cpp
- src/domain/src/chain/script.cpp
- src/blockchain/include/kth/blockchain/pools/header_organizer.hpp
- src/node/src/sync/orchestrator.cpp
- src/blockchain/src/pools/header_organizer.cpp
- src/blockchain/test/regtest_miner.hpp
f8922dc to
3165e07
Compare
Everything built for reorg support so far was verified in pieces. This stands up the whole cycle against the real stack: a trunk to height 100, a block at 101 that spends a coinbase matured on that trunk, and a competing branch of three blocks from 100 that outweighs it. The node connects the first chain, switches to the second, and connects that — through the same tasks the sync coordinator drives, against the same header index, flat-file store, undo files and UTXO-Z the node runs on. The blocks satisfy consensus (real proof of work, real merkle roots, a real signature over a real prevout), so the connect path runs full validation rather than the shortcuts a synthetic block slips through. Regtest makes that cheap: a hundred blocks are mined in milliseconds. What the test found, and what the reviews that followed found in the fixes, is one theme: the by-height view of the chain was written as if the chain only ever grew, and the tip could be moved from two places at once. Addressing blocks by height - The header table cannot be rewritten at a height. Headers are written with LMDB's APPEND, which assumes keys only ever grow, so after a switch the abandoned branch's header stays at that height forever — and every reader that addresses blocks by height (median time past, staleness, the RPC surface) answers from the branch the node left. The duplicate was reported and then swallowed as success. A collision now rewrites the entry; the sequential path keeps APPEND. - Rewriting a height left the displaced header's hash -> height entry behind, so a lookup by the abandoned block's hash resolved to its old height and returned whichever block now occupies it. The displaced entry is dropped before the overwrite; side-branch headers stay addressable through the header index, which is what actually represents them. - Nothing rewrote those heights after a switch. Doing so is part of the reorg sequence now, while the writers are still parked — the one moment nothing else reads those heights or appends past them. - More work does not mean more blocks: a branch mined at higher difficulty can outweigh a longer one, so the chain can end lower than it did and leave heights behind that are on no chain at all. Replacing the range, dropping that tail and setting the last-header height is one database transaction: written halfway, the table would name the new branch over part of the replaced range and the abandoned one over the rest, and nothing later would notice, since every height would still hold a header that parses. - block_storage_task and the header-persist task both walked the index by height. The index numbers entries in arrival order, so since side branches are stored an entry's index is not its height: the storage scan counted the abandoned branch's blocks as contiguous and pushed the validated tip one past where the chain reached, and the persist wrote whichever header sat at that index. Both resolve through the active chain now, as do the seven places that built a getheaders locator by casting a height to an index. Moving the tip - The organizer kept the abandoned branch as its tip. It decides fork versus extension by comparing against the tip it remembers, so left behind it reads every later batch as another fork and asks for a switch whose fork height no longer matches — header sync stops advancing for good. Adds adopt_tip, and execute_reorg to hold the sequence (park the writers, switch, rewrite the replaced heights, drop the tail, move the tip) in one place so the coordinator and the test drive the same thing. - That gave the tip a second writer. It is atomic, published under a mutex, and a batch only publishes if the tip is still the one it validated against — otherwise the switch's chain stands and the batch reports no progress. The cached tip hash is gone: it was a second copy of what the index already answers. - Header persistence takes part in the reorg barrier, like block storage and the UTXO build. Snapshotting the chain generation is not enough on its own — a switch can land between the check and the commit — but a registered, unparked writer holds the switch until it is done. And one that had nothing to do with reorgs: to_pay_public_key_hash_pattern_- unlocking emitted a leading OP_0. That is the dummy multisig needs; in P2PKH it survives to the end of the script and fails CLEANSTACK. The placeholder used for size estimates carried it too, so coin selection was estimating one byte over. Both shapes are pinned by unit tests now.
3165e07 to
11e7d4b
Compare
The write that makes a switch survivable is the one that re-describes the replaced heights. If it fails, the chain in memory and the chain on disk name different branches, nothing repairs that while the node runs, and a restart would come back on the abandoned branch with the UTXO set rewound below it. #578 added that path — reorg_outcome::fatal, on_fatal, full_node::notify_fatal, the executor's stop — and nothing exercised it. Reaching it needs the write to fail on demand. Corrupting a database to get there would test the corruption, not the handling, and filling LMDB's map would depend on page sizes and break on changes that have nothing to do with this. So execute_reorg takes the persister as a parameter. Not a test flag: the write is what decides whether the switch can be lived with, and a caller that runs a reorg has to say where it goes. The coordinator passes block_chain::replace_headers_from; the test passes one that refuses. The test then pins what happens on refusal: - the persister was reached at all (the case is not skipped elsewhere); - the switch itself succeeded — the UTXO set was rewound, the chain moved — and it is describing that on disk which failed; - the by-height table is untouched: height 13 still answers with A's block, so the transaction left nothing half-written; - the reorg is reported as fatal, and no blocks are connected for the new branch; - after a restart the node is on A, whole, with the validated tip and the UTXO-built height both back at the fork — so it re-downloads rather than trust a UTXO state that no longer matches the chain it came back on; - the heavier branch can be announced again and becomes a candidate again. Adds test/fatal_shutdown.cpp for the hinge above it: notify_fatal hands the reason to the owner once and verbatim, and a node with no handler is still safe to report to. What is NOT pinned is stated there too — that notify_fatal stops the node cannot be shown on a node that never started, since stopped() is already true before the call, and the executor's own stop path needs a running node with a network that no test stands up.
The write that makes a switch survivable is the one that re-describes the replaced heights. If it fails, the chain in memory and the chain on disk name different branches, nothing repairs that while the node runs, and a restart would come back on the abandoned branch with the UTXO set rewound below it. #578 added that path — reorg_outcome::fatal, on_fatal, full_node::notify_fatal, the executor's stop — and nothing exercised it. Reaching it needs the write to fail on demand. Corrupting a database to get there would test the corruption, not the handling, and filling LMDB's map would depend on page sizes and break on changes that have nothing to do with this. So execute_reorg takes the persister as a parameter. Not a test flag: the write is what decides whether the switch can be lived with, and a caller that runs a reorg has to say where it goes. The coordinator passes block_chain::replace_headers_from; the test passes one that refuses. The test then pins what happens on refusal: - the persister was reached at all (the case is not skipped elsewhere); - the switch itself succeeded — the UTXO set was rewound, the chain moved — and it is describing that on disk which failed; - the by-height table is untouched: height 13 still answers with A's block, so nothing changed it on the way out (not that the write is atomic — the persister here never reaches the database, so there is no transaction to have been left half-applied); - the reorg is reported as fatal, and no blocks are connected for the new branch; - after a restart the node is on A, whole, with the validated tip and the UTXO-built height both back at the fork — so it re-downloads rather than trust a UTXO state that no longer matches the chain it came back on; - the heavier branch can be announced again and becomes a candidate again. Adds test/fatal_shutdown.cpp for the hinge above it: notify_fatal hands the reason to the owner once and verbatim, and a node with no handler is still safe to report to. What is NOT pinned is stated there too — that notify_fatal stops the node cannot be shown on a node that never started, since stopped() is already true before the call, and the executor's own stop path needs a running node with a network that no test stands up.
…use (#580) The write that makes a switch survivable is the one that re-describes the replaced heights. If it fails, the chain in memory and the chain on disk name different branches, nothing repairs that while the node runs, and a restart would come back on the abandoned branch with the UTXO set rewound below it. #578 added that path — reorg_outcome::fatal, on_fatal, full_node::notify_fatal, the executor's stop — and nothing exercised it. Reaching it needs the write to fail on demand. Corrupting a database to get there would test the corruption, not the handling, and filling LMDB's map would depend on page sizes and break on changes that have nothing to do with this. So execute_reorg takes the persister as a parameter. Not a test flag: the write is what decides whether the switch can be lived with, and a caller that runs a reorg has to say where it goes. The coordinator passes block_chain::replace_headers_from; the test passes one that refuses. The test then pins what happens on refusal: - the persister was reached at all (the case is not skipped elsewhere); - the switch itself succeeded — the UTXO set was rewound, the chain moved — and it is describing that on disk which failed; - the by-height table is untouched: height 13 still answers with A's block, so nothing changed it on the way out (not that the write is atomic — the persister here never reaches the database, so there is no transaction to have been left half-applied); - the reorg is reported as fatal, and no blocks are connected for the new branch; - after a restart the node is on A, whole, with the validated tip and the UTXO-built height both back at the fork — so it re-downloads rather than trust a UTXO state that no longer matches the chain it came back on; - the heavier branch can be announced again and becomes a candidate again. Adds test/fatal_shutdown.cpp for the hinge above it: notify_fatal hands the reason to the owner once and verbatim, and a node with no handler is still safe to report to. What is NOT pinned is stated there too — that notify_fatal stops the node cannot be shown on a node that never started, since stopped() is already true before the call, and the executor's own stop path needs a running node with a network that no test stands up.
… end a run The header index is rebuilt at startup from the persisted by-height headers, so those decide which chain the node resumes. A switch that moved the chain in memory without rewriting them would look right for as long as the process lived, and come back up on the branch it had abandoned — with the UTXO set already rewound below it. #578 added the handling for that; nothing exercised it, in either direction. Adds chain_fixture::restart(), which drops the chain and brings it up again on the same directory: nothing in memory survives, so what comes back is whatever was written to disk. The switch that succeeds: a short cycle (trunk to 12, one block on A, three on B), switched, restarted. Heights 13-15 name B's blocks, the organizer's tip is B's head, the validated tip and the UTXO-built height both return at 15, and the UTXO set holds B's coinbases and not A's. The switch whose header write fails: reaching that needs the write to fail on demand. Corrupting a database to get there would test the corruption, not the handling, and filling LMDB's map would depend on page sizes and break on changes that have nothing to do with this. So execute_reorg takes the persister as a parameter — not a test flag: the write is what decides whether a switch can be lived with, and a caller that runs a reorg has to say where it goes. The coordinator passes block_chain::replace_headers_from; the test passes one that refuses. On refusal: the switch itself succeeded and it is describing that on disk which failed; the by-height table still answers with A's block, so nothing changed it on the way out (not that the write is atomic — that persister never reaches the database); the reorg is reported fatal; and after a restart the node is on A, whole, with the validated tip and the UTXO-built height both back at the fork, so it re-downloads rather than trust a UTXO state that no longer matches the chain it came back on. The heavier branch can then be announced again and becomes a candidate again. test/fatal_shutdown.cpp covers the hinge above that: notify_fatal hands the reason to the owner once and verbatim, and a node with no handler is still safe to report to. What is NOT covered is stated there — the coordinator acting on the flag, and the executor's own stop path, are wired by inspection only. Writing it turned up a defect. The organizer was not told how far blocks are validated at startup: nothing tells it until the first newly stored block, and deep-reorg parking measures rewind depth against that height — so a node that had just come up would treat any heavier branch as costing no rewind and follow it, however deep it forked. That is the same window in which finalization is not protecting either, since it needs headers older than the finalization delay. full_node::run_sync now reports the persisted validated height right after sync_tip(), the fixture does the same, and both tests assert it. And side branches do not survive a restart. The by-height table is the only persisted header store and it describes the active chain alone, so the rebuilt index has no entry for the abandoned branch at all. Its block bytes stay in the flat files, held by nothing, until a peer announces the branch again and its headers are re-downloaded. BCHN persists its whole block index and does remember. The node still converges either way; the test now says so out loud.
… end a run (#579) The header index is rebuilt at startup from the persisted by-height headers, so those decide which chain the node resumes. A switch that moved the chain in memory without rewriting them would look right for as long as the process lived, and come back up on the branch it had abandoned — with the UTXO set already rewound below it. #578 added the handling for that; nothing exercised it, in either direction. Adds chain_fixture::restart(), which drops the chain and brings it up again on the same directory: nothing in memory survives, so what comes back is whatever was written to disk. The switch that succeeds: a short cycle (trunk to 12, one block on A, three on B), switched, restarted. Heights 13-15 name B's blocks, the organizer's tip is B's head, the validated tip and the UTXO-built height both return at 15, and the UTXO set holds B's coinbases and not A's. The switch whose header write fails: reaching that needs the write to fail on demand. Corrupting a database to get there would test the corruption, not the handling, and filling LMDB's map would depend on page sizes and break on changes that have nothing to do with this. So execute_reorg takes the persister as a parameter — not a test flag: the write is what decides whether a switch can be lived with, and a caller that runs a reorg has to say where it goes. The coordinator passes block_chain::replace_headers_from; the test passes one that refuses. On refusal: the switch itself succeeded and it is describing that on disk which failed; the by-height table still answers with A's block, so nothing changed it on the way out (not that the write is atomic — that persister never reaches the database); the reorg is reported fatal; and after a restart the node is on A, whole, with the validated tip and the UTXO-built height both back at the fork, so it re-downloads rather than trust a UTXO state that no longer matches the chain it came back on. The heavier branch can then be announced again and becomes a candidate again. test/fatal_shutdown.cpp covers the hinge above that: notify_fatal hands the reason to the owner once and verbatim, and a node with no handler is still safe to report to. What is NOT covered is stated there — the coordinator acting on the flag, and the executor's own stop path, are wired by inspection only. Writing it turned up a defect. The organizer was not told how far blocks are validated at startup: nothing tells it until the first newly stored block, and deep-reorg parking measures rewind depth against that height — so a node that had just come up would treat any heavier branch as costing no rewind and follow it, however deep it forked. That is the same window in which finalization is not protecting either, since it needs headers older than the finalization delay. full_node::run_sync now reports the persisted validated height right after sync_tip(), the fixture does the same, and both tests assert it. And side branches do not survive a restart. The by-height table is the only persisted header store and it describes the active chain alone, so the rebuilt index has no entry for the abandoned branch at all. Its block bytes stay in the flat files, held by nothing, until a peer announces the branch again and its headers are re-downloaded. BCHN persists its whole block index and does remember. The node still converges either way; the test now says so out loud.
Everything built for reorg support so far was verified in pieces. This stands up the whole cycle against the real stack.
The scenario
Driven through the same tasks the sync coordinator drives (
block_storage_task,utxo_build_task,execute_reorg) against the same header index, flat-file block store, undo files and UTXO-Z the node runs on.The blocks satisfy consensus — real proof of work, real merkle roots, a real signature over a real prevout — so the connect path runs full validation rather than the shortcuts a synthetic block slips through. Regtest makes that cheap: a hundred blocks are mined in milliseconds.
test/regtest_miner.hppholds the miner; blocks are serialized and re-parsed aslight_blockso what reaches storage arrived the same way a block off the wire does.What it verifies
The spend really happened before the reorg (the coinbase it consumed is out of the set, the output it created is in) — without that the undo below would be undoing nothing and everything after it would pass vacuously. Then, after the switch: A's coinbase and its spend are gone from the UTXO set, the coinbase A spent is back at its original creation height (not at 101, which would misdate its maturity and its median time past), heights 101–103 name B's blocks, the validated tip and the UTXO-built height both reach 103, A's abandoned block is still on disk, and a header extending the new tip is read as a plain extension.
The five defects it found
All five are on the path a real reorg takes; none is reachable by a test that stops at the pieces.
1. The header table cannot be rewritten at a height. Headers are written with LMDB's
APPEND, which assumes keys only ever grow, so after a switch the abandoned branch's header stays at that height forever — and every reader that addresses blocks by height (median time past, staleness, the RPC surface) answers from the branch the node left. The duplicate was logged and then swallowed as success. A collision now rewrites the entry; the sequential path keepsAPPEND.2.
block_storage_taskwalked the header index by height. The index numbers entries in arrival order, so since side branches are stored (#570) an entry's index is not its height. The contiguous scan counted the abandoned branch's blocks and pushed the validated tip one past where the chain actually reached — visible in the test aslast_block_height= 104 on a 103-block chain. Resolved through the active chain now.3.
persist_headers_to_dbhad the same confusion, and persisted whichever header happened to sit at that index.4. The organizer kept the abandoned branch as its tip. It decides fork-versus-extension by comparing against the tip it remembers; left behind, it reads every later batch as another fork and asks for a switch whose fork height no longer matches — so
switch_to_branchrejects it and header sync stops advancing for good. Addsheader_organizer::adopt_tip, andnode::sync::execute_reorgto hold the sequence (park the writers at the barrier, switch, move the tip) in one place — which is also what lets the test drive the same thing the coordinator does instead of a re-implementation of it.5.
to_pay_public_key_hash_pattern_unlockingemitted a leadingOP_0. That is the dummy multisig needs; in P2PKH it survives to the end of the script and failsCLEANSTACK. The placeholder used for size estimates carried it too, so coin selection was estimating one byte over. No production caller builds real unlocking scripts with it today, which is why nothing had failed yet.Scope
The barrier is satisfied trivially here (no task is running at the moment of the switch) — its own behaviour is covered by the drain tests in #576. Compact UTXO mode is not exercised: it needs a separate build configuration, so it stays a follow-up.
Local:
kth_blockchain_test1919 assertions / 176 cases,kth_node_test616 / 110 (the cycle itself is 266 assertions),kth_domain_test1008351 / 3821,node-exebuilds.Summary by CodeRabbit
New Features
Bug Fixes
Tests