Skip to content

Commit 39b5bbc

Browse files
Merge #7869: fix: let startup block replay handle quorum commitments and asset unlocks
74dc3e5 test: replay blocks with quorum commitments and asset unlocks after a crash (pasta) f057817 fix: replay blocks under the BLS scheme they were connected with (pasta) 4585bee fix: give replayed blocks the chain view they were connected with (pasta) Pull request description: ## Issue being fixed or feature implemented When a node stops in the middle of a coins-database flush (crash, power loss, OOM kill), the next start rolls the unflushed blocks forward in `ReplayBlocks()`. That runs inside `LoadChainstate()` before `LoadChainTip()`, so the chain is still empty while `RollforwardBlock()` re-processes each block's special transactions and quorum commitments. Several of those checks read the chain, and two things go wrong on a chain with quorums: - A block containing a mined quorum commitment aborts the node: `IsMiningPhase()` asserts `nHeight <= active_chain.Height() + 1`, and the chain height is -1. The existing "skip when replaying" branches in `CQuorumBlockProcessor` do not cover that assertion, so the node crashes again on every start until `-reindex-chainstate`. - Asset-unlock and MNHF-signal transactions resolve their quorum against the chain, and with an empty chain they find none. Replay rejects the block (`bad-assetunlock-too-old-quorum`) and startup fails with "Unable to replay blocks. You will need to rebuild the database using -reindex-chainstate." Separately, `RollforwardBlock()` processes blocks under whatever BLS scheme the process-wide flag holds. At that point in startup it holds the legacy default, because `VerifyLoadedChainstate()` only sets it later. So the first replayed post-V19 block runs under the legacy scheme, and an asset unlock in that block fails signature verification (`bad-assetunlock-not-verified`) even with a chain in place. `ConnectBlock()` already sets the scheme for each block with `ScopedBLSLegacyScheme`. The replay path never got that guard. The replay range covers every block since the previous coins flush, often hours of blocks. Mainnet mines a non-null quorum commitment roughly every half hour, so in practice this recovery always fails on mainnet and needs a full `-reindex-chainstate`. That costs masternodes hours offline. ## What was done? - `ReplayBlocks()` sets the chain tip to the parent of each block before rolling it forward. Each replayed block then sees the same chain view `ConnectBlock()` saw when the block was first connected. The tip is moved to the parent before each block, not to the block after it, so the chain ends one block short of the new coins tip. `LoadChainTip()` then still runs its full path: it sets the tip and prunes the block-index candidates that `LoadBlockIndex()` added while the chain was empty. - `RollforwardBlock()` establishes the block's BLS scheme the same way `ConnectBlock()` does. The guard is not committed: every replayed block sets its own scheme on entry, and `VerifyLoadedChainstate()` sets the scheme for the loaded tip afterwards. Both changes only affect local crash recovery of blocks this node already fully validated. Block validation rules, serialization and the P2P protocol are unchanged. The CbTx best-ChainLock check is not involved: replay passes `fCheckCbTxMerkleRoots=false`, so the coinbase payload checks are skipped there. The `chainstate.m_chain.Tip() == nullptr` "skip when replaying" branches in `CQuorumBlockProcessor` are no longer reached during replay, because replay now runs with the same chain view as `ConnectBlock()`. They are left in place to keep this change small and can be removed in a follow-up. ## How Has This Been Tested? New step `test_replay_after_crash` at the end of `feature_asset_locks.py`, which already has the platform quorum, evo masternodes and asset unlock helpers. It crashes the regular node mid-flush twice and checks the replay each time. Each cycle restarts the node with `-dbcrashratio=1 -dbbatchsize=1`, mines without flushing, and forces a flush (`gettxoutsetinfo`) that crashes the node after its first partial batch. The node is then restarted without peers and must log `Rolling forward` for every expected block, come back at the same tip, and have the same UTXO set hash as the other node. - Cycle 1 mines a new platform quorum commitment and then an asset unlock (index 900) signed by a platform quorum. It also checks the new quorum is listed after the restart. The commitment block is replayed before the unlock. - Cycle 2 starts from a clean flush with the quorum already mature and mines only an asset unlock (index 901), so the unlock is the first and only block rolled forward. No earlier replayed block resets the BLS scheme, so this cycle is what independently covers the `RollforwardBlock()` scheme guard. After the second cycle the node is restarted normally and reconnected, so the step leaves the file as it found it. Results (macOS arm64, `--enable-debug`, full `feature_asset_locks.py` run, about 3 minutes): - With this PR: passes, with and without `--usecli`. - With the two `src/validation.cpp` fixes reverted (the PR base): cycle 1 fails at the restart, with the node aborting during startup on the `IsMiningPhase()` assertion (`Assertion failed: (nHeight <= active_chain.Height() + 1), function IsMiningPhase, file blockprocessor.cpp`). The commitment block is replayed before the asset unlock block, so the test reaches that crash first. - With only the chain-tip change reverted (BLS guard kept): cycle 1 fails with the same `IsMiningPhase()` assertion. - With only the BLS scheme guard removed (chain-tip change kept): cycle 1 still passes, because the post-V19 blocks replayed before the unlock reset the global scheme as a side effect. Cycle 2 fails: startup aborts with "Unable to replay blocks" and the log shows `RollforwardBlock(DASH): ProcessSpecialTxsInBlock for block ... failed with bad-assetunlock-not-verified`. Also ran `feature_dbcrash.py` and the unit suites `validation_chainstatemanager_tests`, `validation_chainstate_tests`, `validation_flush_tests`, `evo_cbtx_tests`, `evo_snapshot_tests`, plus `test/lint/lint-whitespace.py`. ## Breaking Changes None. ## Checklist: - [x] I have performed a self-review of my own code - [ ] I have made corresponding changes to the documentation - [ ] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) Top commit has no ACKs. Tree-SHA512: 2a281674e6d8e48ec66f276ba7cb65869dc54c83fa1580214fb2441e275716de75a0bb564481e3e8db476c2114e9095c7f5be9548032955add476bd7c9279a02
2 parents 4461504 + 74dc3e5 commit 39b5bbc

2 files changed

Lines changed: 55 additions & 0 deletions

File tree

‎src/validation.cpp‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5409,6 +5409,9 @@ bool Chainstate::RollforwardBlock(const CBlockIndex* pindex, CCoinsViewCache& in
54095409
BlockValidationState state;
54105410
const CAmount blockSubsidy = GetBlockSubsidy(pindex, m_chainman.GetConsensus());
54115411
MNListUpdates mnlist_updates;
5412+
// Use the scheme ConnectBlock used for this block instead of whatever replay inherited
5413+
ScopedBLSLegacyScheme bls_scheme_guard{
5414+
!DeploymentActiveAt(*pindex, m_chainman.GetConsensus(), Consensus::DEPLOYMENT_V19)};
54125415
if (!m_special_tx.ProcessSpecialTxsInBlock(*this, m_chain, block, pindex,
54135416
GetSpecialTxRules(pindex->pprev, m_chainman), inputs,
54145417
blockSubsidy, /*fJustCheck=*/false,
@@ -5507,6 +5510,11 @@ bool Chainstate::ReplayBlocks()
55075510

55085511
LogPrintf("Rolling forward %s (%i)\n", pindex.GetBlockHash().ToString(), nHeight);
55095512
uiInterface.ShowProgress(_("Replaying blocks…").translated, (int) ((nHeight - nForkHeight) * 100.0 / (pindexNew->nHeight - nForkHeight)) , false);
5513+
// Special transaction and quorum commitment checks read the chain, which
5514+
// LoadChainTip() has not set yet; give them the view ConnectBlock had.
5515+
// Stopping at the parent also keeps LoadChainTip() from skipping its
5516+
// candidate pruning.
5517+
m_chain.SetTip(*Assert(pindex.pprev));
55105518
if (!RollforwardBlock(&pindex, cache)) return false;
55115519
}
55125520

‎test/functional/feature_asset_locks.py‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,9 @@
55
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
66

77
import copy
8+
import http.client
89
import struct
10+
import subprocess
911
from decimal import Decimal
1012
from io import BytesIO
1113

@@ -46,6 +48,7 @@
4648
assert_equal,
4749
assert_greater_than,
4850
assert_greater_than_or_equal,
51+
assert_raises,
4952
assert_raises_rpc_error,
5053
softfork_active,
5154
)
@@ -354,6 +357,7 @@ def run_test(self):
354357
self.test_v24_fork(node_wallet, node, pubkey)
355358
self.test_asset_unlock_v2(node_wallet, node, pubkey)
356359
self.test_non_standard(node_wallet, node, pubkey)
360+
self.test_replay_after_crash(node_wallet, node, pubkey)
357361

358362

359363
def test_asset_locks(self, node_wallet, node, pubkey):
@@ -1438,6 +1442,49 @@ def test_non_standard(self, node_wallet, node, pubkey):
14381442
self.restart_node(1, self.extra_args[1])
14391443
self.connect_nodes(1, 0)
14401444

1445+
def test_replay_after_crash(self, node_wallet, node, pubkey):
1446+
self.log.info("Testing recovery from a crash mid-flush with a quorum commitment and an asset unlock to replay...")
1447+
# Every flush now dies after its first partial write, so nothing mined from here on is flushed
1448+
self.restart_with_crashing_flush()
1449+
1450+
self.mine_quorum_2_nodes()
1451+
quorum_hash = node_wallet.quorum("list", 1)["llmq_test_platform"][0]
1452+
commitment_block = node_wallet.quorum("info", llmq_type_test, quorum_hash)["minedBlock"]
1453+
txid = self.send_tx(self.create_assetunlock(900, COIN, pubkey))
1454+
unlock_block = self.generate(node_wallet, 1)[0]
1455+
assert txid in node_wallet.getblock(unlock_block)["tx"]
1456+
1457+
self.log.info("Restart without peers: the commitment block is replayed before the asset unlock block")
1458+
self.crash_and_replay(node_wallet, node, [commitment_block, unlock_block])
1459+
assert quorum_hash in node_wallet.quorum("list")["llmq_test_platform"]
1460+
1461+
self.log.info("Replay an asset unlock as the first block rolled forward, with the quorum already mature and flushed")
1462+
self.restart_node(0, self.extra_args[0])
1463+
self.restart_with_crashing_flush()
1464+
txid = self.send_tx(self.create_assetunlock(901, COIN, pubkey))
1465+
unlock_block = self.generate(node_wallet, 1)[0]
1466+
assert txid in node_wallet.getblock(unlock_block)["tx"]
1467+
self.crash_and_replay(node_wallet, node, [unlock_block])
1468+
1469+
self.restart_node(0, self.extra_args[0])
1470+
for i in range(1, len(self.nodes)):
1471+
self.connect_nodes(0, i)
1472+
1473+
def restart_with_crashing_flush(self):
1474+
self.restart_node(0, self.extra_args[0] + ["-dbcrashratio=1", "-dbbatchsize=1"])
1475+
for i in range(1, len(self.nodes)):
1476+
self.connect_nodes(0, i)
1477+
1478+
def crash_and_replay(self, node_wallet, node, replayed_blocks):
1479+
tip = node_wallet.getbestblockhash()
1480+
self.log.info("Force a flush, which crashes the node")
1481+
assert_raises((ConnectionError, http.client.CannotSendRequest, subprocess.CalledProcessError), node_wallet.gettxoutsetinfo)
1482+
self.wait_for_node_exit(0, timeout=10)
1483+
with node_wallet.assert_debug_log([f"Rolling forward {block}" for block in replayed_blocks]):
1484+
self.start_node(0, extra_args=self.extra_args[0] + ["-connect=0"])
1485+
assert_equal(node_wallet.getbestblockhash(), tip)
1486+
assert_equal(node_wallet.gettxoutsetinfo()["hash_serialized_2"], node.gettxoutsetinfo()["hash_serialized_2"])
1487+
14411488

14421489
if __name__ == '__main__':
14431490
AssetLocksTest().main()

0 commit comments

Comments
 (0)