Rewrite qa/rpc-tests as a python3 + pytest + asyncio suite - #290
Open
MorningLightMountain713 wants to merge 117 commits into
Open
Rewrite qa/rpc-tests as a python3 + pytest + asyncio suite#290MorningLightMountain713 wants to merge 117 commits into
MorningLightMountain713 wants to merge 117 commits into
Conversation
generate, getblocktemplate, and the BitcoinMiner thread dereferenced the CReserveScript returned by ScriptForMining without a null check. When no handler sets the script -- no wallet and no -mineraddress (e.g. -disablewallet on regtest) -- the shared_ptr is null and the ->reserveScript.size() check crashes the daemon. Add the upstream null guard so these paths report a clean error instead. generate is regtest-only and the primary beneficiary: regtest block production (including PON) no longer crashes under -disablewallet. The other two paths carry the identical latent dereference. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduce qa/pyproject.toml (uv-managed dependencies, ruff lint/format,
ty type checking) and qa/uv.lock so the RPC integration tests run under a
pinned python3 toolchain, with a short qa/README describing the entry point
and the in-progress python2 -> python3 migration.
Add qa/rpc-tests/generate_no_wallet.py covering the mining null
coinbase-script guard: a -disablewallet node returns RPC_INTERNAL_ERROR
("No coinbase script available") from generate instead of crashing the
daemon, and a node with a wallet still mints a block.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rewrite the three modules the RPC tests load on every run -- util.py, test_framework.py, authproxy.py -- as python3-only, fully type-annotated, f-string code, and fix the framework's worst runtime behaviours: - Fail fast on daemon death. start_node and the cached-chain bootstrap poll the process while waiting for RPC and raise immediately with the daemon's captured stderr, instead of blocking on -rpcwait until an external timeout (the "silent daemon death hangs forever" footgun). A fresh connection is used per poll so a refused connect during warmup does not wedge a reused http.client connection in the request-started state. - Capture each node's stderr to node_stderr.log; stdout goes to /dev/null so the metrics/console thread does not block on a non-tty file descriptor. - Teardown no longer hangs: wait_bitcoinds terminates any straggler past a grace period, and stop_nodes survives an already-dead node rather than masking the test's primary error. - argparse instead of optparse; logging instead of bare print. ty resolves the test_framework package via the rpc-tests import root declared in pyproject.toml. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduce an async pytest framework under rpc-tests/: - fluxtest.rpc.FluxRPC: an aiohttp JSON-RPC client. `await rpc.<method>(...)` returns the result with every JSON number parsed as Decimal, so satoshi amounts stay exact. - fluxtest.node.FluxNode: manages a regtest fluxd over asyncio (create_subprocess_exec), polling for RPC readiness and failing fast with the captured stderr if the daemon exits during startup. - conftest.py: a node_factory fixture that starts nodes and stops them after the test, plus a --fluxd option for the binary under test (asyncio_mode=auto). Port generate_no_wallet to two async pytest tests in test_generate_no_wallet.py and drop the class-based version. pytest collects only converted test_*.py modules, so the remaining legacy scripts are left untouched until they are migrated. Dependencies are uv-managed and pinned to current releases: aiohttp, pytest, pytest-asyncio, pyzmq, plyvel. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Port disablewallet.py to an async pytest test: a -disablewallet node still serves validateaddress (a mainnet t3 address is invalid on regtest, a testnet tm address is valid). Drop the python2 original. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add fluxtest.network (connect_nodes, connect_nodes_bi, sync_blocks, sync_mempools) and a FluxNode.mine helper that stamps increasing block times from a mocktime base. Two nodes given different bases build genuinely divergent chains -- regtest mining is otherwise deterministic and two nodes produce byte-identical chains, so a reorg test needs this to create competing forks. Convert invalidateblock to pytest: a node reorged onto a competing longer chain reorgs back to its original chain when that chain's block is invalidated (setBlockIndexCandidates is repopulated). Drop the python2 original. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A node will not download blocks for a chain whose tip is far older than its own clock, so a real-time node could not sync a mocktime-stamped chain. Start every node with its clock frozen near a shared base (offset by node index), and have mine() advance the block time ahead of the current tip (plus the index) so blocks stay valid after adopting a peer's chain and never deterministically reproduce an invalidated block. Restore invalidateblock's full three-node body (the won't-reorg-to-lower-work check), now that cross-node sync works. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add FluxNode.restart (stop the process, keep the datadir and RPC session, relaunch with optional new args). Convert reindex.py to pytest: mine 3 blocks, restart with -reindex -checkblockindex=1, and assert the chain rebuilds to the same height. Drop the python2 original. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PON shares a branch id with several earlier upgrades, so -nuparams cannot target it, and regtest hardwires PON on (activation height -1, which IsPONActive treats as always-active). That leaves regtest unable to mine PoW blocks -- so under PON the coinbase is redirected to the dev-fund address and a test wallet can never be funded by mining. Add a regtest-only -ponactivation=height option that sets the PON upgrade's activation height directly. Passing a high value runs regtest in PoW mode, where the coinbase pays the wallet, so it can be funded normally. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a funded_node fixture that runs a node in PoW mode (-ponactivation high) and mines past coinbase maturity, giving the wallet a spendable balance. test_wallet_send funds a node, sends to a second node, and asserts receipt. Drop the mine interval to 30s so a long chain's tip stays within the future-block window of a peer's frozen clock and can still be synced. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A coinbase spend enters the mempool only once the coinbase has reached COINBASE_MATURITY; the next one up is rejected as immature. Uses PoW-mode funding and spends the coinbase value minus a normal fee (Flux's coinbase is much larger than the original's hardcoded amount, which would otherwise trip the absurd-fee check before maturity). Drop the python2 original. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A node reorged onto a longer competing chain retains its abandoned chain as a valid fork, so getchaintips reports two tips (one active, one valid-fork with the expected branchlen). Rewritten as a two-node test in place of the original four-node cached-chain split. Drop the python2 original. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
nodehandling: setban/listbanned/clearbanned over a subnet, and disconnectnode followed by reconnect. mempool_resurrect: transactions mined into a block return to the mempool when that block is invalidated, and confirm again when re-mined. Drop the python2 originals. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
listtransactions: a send is recorded on the sender (negative amount) and a confirmed receive on the recipient. getblocktemplate: the template carries a required coinbasetxn, a 16-char noncerange and no coinbasevalue (needs a connected peer). Both drop the deprecated-account / zcash-hardcoded assertions of the originals. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
No txout proof exists for an unconfirmed transaction; once mined, a peer can fetch and verify proofs for one or several transactions in the block (verifytxoutproof returns the proven txids). Drop the python2 original. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
With the insight explorer enabled, getblockhashes(high, low) returns the block hashes whose time falls in the half-open range. Drop the python2 original. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rewrite the gettxoutsetinfo test to compute the expected total amount and txout count from the chain's own coinbase outputs, rather than hardcoding zcash's emission. This stays correct under Flux's reward schedule (subsidy plus foundation/exchange/swap fund outputs). Drop the python2 original. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Restarting with -zapwallettxes=1 drops unconfirmed wallet transactions while keeping confirmed ones. Drop the python2 original. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
httpbasics: the JSON-RPC HTTP endpoint answers and keeps the connection alive across requests. receivedby: listreceivedbyaddress/getreceivedbyaddress report a received amount with the right confirmation count and honour the minconf filter. Drop the python2 originals. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
signrawtransaction with explicit inputs and keys: a valid input signs to a complete transaction; an invalid and a missing input each produce a script verification error referring to the offending input. Drop the python2 original. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An await inside the all() generator expression made it an async generator, which all() cannot iterate, so sync_mempools raised TypeError on every call. Materialize each node's mempool set into a list first. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Flux regtest coinbases carry P2SH fund outputs (foundation, swap-pool) that the wallet lists in listunspent but cannot sign. Selecting outputs by index could pick one, yielding an empty scriptSig and a mandatory-script-verify-flag-failed rejection that surfaced nondeterministically as wallet keys are random per run. Filter to spendable outputs and assert the signature completes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rewrite the keypool/encryption test on the async framework: encrypt then restart, the keypool-exhausted error (-12) on the second address, refill, drain four change addresses, exhaust again. Removes the python2 original. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three-node *rawtransaction coverage: missing input, 2-of-2 and 2-of-3 multisig spendability, partial/complete signing, and input-sequence round-trip. The legacy +10 block-reward balance assertion is replaced with the per-height miner subsidy read from getblocksubsidy. Removes the python2 original. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Full fundrawtransaction matrix: input selection, change, fee parity vs sendtoaddress/sendmany, multisig outputs, spending a 2-of-2 over fundraw, the locked-wallet flow, ~19 small inputs, and OP_RETURN. The two zcash block-reward balance assertions are replaced with behavioural receipt checks. Removes the python2 original. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Coinbase-spend reorg scenarios (direct, indirect, grandchild) plus a block-height timelock, with both invalidateblock waves. Coinbase spends send value minus a tiny fee, read from the chain, to avoid Flux's absurd-fee rejection on its large coinbases. Removes the python2 original. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
importprivkey rescan carries a key's full UTXO history: a three-node rewrite (alice funder+miner, bob, charlie) verifying the imported key observes the same UTXO set and that import is idempotent. Removes the python2 original. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Port the decodescript / decoderawtransaction asm-decoding coverage (scriptSig and scriptPubKey for every standard type, plus the [ALL] / [NONE|ANYONECANPAY] sighash annotations). The legacy sighash subcases mutated a mininode CTransaction; here the single-input tx is rebuilt by hex splicing so no serializer is needed. No chain state required. Removes the python2 original. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rewrite to behaviour: two ordinary sends of specific coinbases are confirmed, an isolated node mines a conflicting double-spend on a longer chain, and after re-convergence the original sends report confirmations == -1 and balances follow the double-spend. Drops the zcash economics (fixed reward/starting balance), deprecated accounts, and the 4-node cached split. Removes the python2 original. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ACADIA gates whether createrawtransaction emits Overwintered transactions, whose signatures carry a consensus branch id. In regtest it defaults to no activation and shares branch id 0x76b809bb with several other upgrades, so -nuparams cannot single it out (the matcher stops at the first id match). This regtest-only flag sets ACADIA's activation height directly, mirroring -ponactivation, so consensus-branch-id signing can be exercised. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The node sanitizes a peer's subversion with SAFE_CHARS_DEFAULT, which on Flux allows alphanumerics and " .,;_/:?@()" but not "-" (upstream Bitcoin keeps "-"). So /python-mininode-tester:0.0.1/ is reported as /pythonmininodetester:0.0.1/. Assert that exact value rather than a substring. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The first mininode test: a peer announces 8, 16, 128 then 1024 block inventories with random hashes and counts the resulting getdata requests. Because the mininode never answers them, the in-flight slots stay full and the node stops requesting, so it never asks for a block twice and keeps the outstanding requests under the cap. Runs on the asyncio mininode, syncing with a ping and a short settle before each check. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Confirms the mininode CTransaction deserializes and re-serializes the daemon's raw transaction bytes (a Sapling v4 transaction) without change. Byte-exact round-tripping is what lets the P2P tests build, modify and re-sign transactions, so this guards that invariant directly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Drives the expiring-soon relay path with a mininode peer: a transaction whose expiry height is within the three-block threshold of the tip is dropped from every mempool; one accepted just outside the window is advertised and served over getdata; after node2 (isolated by a restart) mines one block and the others reorg onto it, that transaction is now expiring soon, so submitting it is refused and a getdata for it is answered with notfound rather than the transaction, while a fresh transaction is still served normally. Reframed for Flux: ACADIA active and PON off, the mininode connects above the regtest ACADIA protocol minimum, the spend is a whole coinbase less the fee, and the expiry heights track the tip (the three-block threshold matches upstream). The transaction is built, given an expiry height and signed through the mininode CTransaction. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A mininode sends a transaction whose expiry height equals the tip, so it has only just expired: the node rejects it but leaves the peer's ban score at zero (an honest peer can race the tip). After one more block clears the reject cache and the transaction is expired by more than a block, resending it raises the ban score to 10. Reframed for Flux: ACADIA active, PON off, the mininode connects above the regtest ACADIA protocol minimum, and the spend is a matured coinbase with its expiry set to the tip. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Upstream stages Overwinter then Sapling as two version boundaries; Flux activates both at the single ACADIA upgrade, so this collapses to one. Mininodes connect at a below-minimum version (Overwinter, 170003) and at the regtest ACADIA minimum (Sapling, 170006). Below the activation height all stay connected; once ACADIA activates, a message from each peer makes the node drop the below-minimum ones, a new below-minimum peer is rejected on connect with "Version must be 170006 or greater", and an at-minimum peer is still accepted. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The regtest ACADIA minimum peer protocol version is 170006, not 170007 (170007 is mainnet). The tests connect at 170021, which clears either, so only the comments were inaccurate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A node started with -nopeerbloomfilters -enforcenodebloom disconnects a peer that sends a filteradd but tolerates a harmless filterclear; a default node, which serves bloom filters, keeps the peer in both cases. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ports the pure-python equihash solver/verifier to python3 (the legacy module used python2 integer division throughout) and wires solve()/is_valid() into CBlock, so the mininode can build and submit regtest blocks. Regtest uses the n=48, k=5 parameters whose personalization is "ZcashPoW", matching the daemon; a real regtest block solves from scratch in about 0.15s. Also fixes the block hash: CBlockHeader.calc_sha256 hashed self.serialize(), which for a CBlock folded the transactions into the hash. It now hashes the header only, so the computed hash matches the daemon's. Regtest CheckProofOfWork returns true unconditionally, so is_valid()/solve() check the equihash solution rather than a hash target, which regtest does not enforce. A test mines a regtest block, confirms the solver validates the daemon's own solution and that the hashes match, then re-solves the header from scratch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A minimal CScript builder (fluxtest/script.py) and block helpers (fluxtest/blocktools.py: create_block, create_coinbase, create_transaction). create_coinbase pays the regtest founders reward (a fifth of the reward to the regtest dev-fund P2SH before the first halving) and carries the BIP34 height, so the coinbase is consensus-valid on Flux regtest. A test builds a block on the tip, solves its equihash and submits it: the daemon accepts it and the chain extends to the built block, end to end. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The regtest coinbase must pay the one-time exchange (3M) and foundation (2.5M) funding at height 10, and the recurring swap-pool funding (2.1M at heights 10, 20, 30, 40, 50). Add script_for_address (base58check decode to a P2PKH or P2SH script using the regtest version prefixes) and emit those outputs at the right heights, so a chain builds cleanly across them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ty could not infer the non-bytes branch was iterable. Annotate the items parameter accordingly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Port the unrequested-block acceptance test to the asyncio mininode. Both the whitelisted and non-whitelisted paths are exercised: a block extending the tip is accepted by both nodes; a competing same-work fork is kept as a header only unless the peer is whitelisted; a longer fork is stored but not connected while an intermediate block is missing, and an inv that marks the peer's chain availability prompts the node to request and connect it, reorging onto the fork. Blocks are built and equihash-solved by the mininode from a BASE height past the regtest difficulty-reset and eh_epoch ramp windows, spaced just over twice the target so each stays at min-difficulty. The spacing is kept tight enough that the ~290-block span stays under the 24h tip-age threshold, so the stuck node does not fall into initial block download (which would disable the block-download scheduler the reorg relies on). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Port the invalid-block-request test to the asyncio mininode and add the two
pieces it needs:
- fluxtest/comptool.py: a comparison harness that announces blocks and
transactions, serves them from an in-memory store when the node requests
them, and reads each node's resulting tip/mempool back over RPC. Serving on
request (rather than pushing) is what exercises the malleated-block case: a
block rejected for a duplicated transaction leaves a header with no valid
data, so the node re-requests it and the honest block sharing its hash is
served in its place.
- fluxtest/pow.py: the regtest difficulty rule, so the mininode can stamp each
block it builds with the nBits GetNextWorkRequired requires. On regtest this
reduces to powLimit everywhere except the three ZelHash ramp heights (71-73),
whose difficulty is a fixed, time-independent consequence of the ramp scale
factor overflowing modulo 2**256.
The test builds the whole chain itself -- the spendable coinbase, the 100
maturity blocks and the three test blocks -- and checks that a valid block is
accepted, a transaction-duplicated block is rejected while the honest block with
the same hash is still accepted, and an over-funded coinbase is rejected. The
peer is whitelisted so it survives the ban scored against the malleated block,
and mocktime tracks the tip so the node stays out of initial block download.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Port the CLTV (BIP65) and DERSIG (BIP66) tests to the asyncio mininode and the comparison harness. Flux enforces both rules unconditionally -- CLTV via the block script flags, strict-DER on every signature -- with no version-bits activation (IsSuperMajority has no callers), so these exercise enforcement, not the activation transition the upstream Bitcoin versions test; the Zcash-derived sources already drop that transition. A shared driver (_bip_enforcement.py) mines two wallet-owned coinbases past maturity and delivers one block carrying a valid wallet-signed coinbase spend (accepted) and one whose spend has been made rule-violating (rejected): a prepended -1 CHECKLOCKTIMEVERIFY for BIP65, a NUL byte padded into the DER signature for BIP66. The two blocks spend different coinbases so the invalid one fails for the rule under test rather than for a double spend. Only the test block is mininode-built; the coinbases are node-mined because the spend is wallet-signed. The peer is whitelisted to survive the ban scored against the invalid block, and the block's difficulty comes from the regtest difficulty port. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Port the addr-message rate-limiting test to the asyncio mininode. Flux carries the addr token-bucket limiter (getpeerinfo addr_processed / addr_rate_limited), so this is a real conversion rather than a drop. A sender peer and four receivers connect; the test checks that 1 + 69 addresses against a bucket refilled by 600s of mocktime are split 60 processed / 10 rate-limited, that a small addr message is relayed to other peers, and that an oversized (>1000) message is dropped wholesale while valid messages around it are relayed. The bucket stamps its last update with real time and refills against mocktime, so the node's clock is pinned to real time at connect and advanced from there. Also fix CAddress/msg_addr to carry the per-entry nTime that the addr wire format includes for the negotiated protocol version (without it the mininode misparses any addr the node sends). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
script_test announces blocks containing each script_valid/script_invalid vector and only checks that two nodes agree (every outcome is None), comparing a test binary against a reference binary. With Flux's single binary that agreement is trivially true. Its model also does not map to Flux: each vector specifies the script flags to apply and the test toggles P2SH by block time, whereas Flux validates blocks with a fixed flag set (P2SH and CLTV, plus always-on strict DER) that cannot be toggled, so a vector's valid/invalid expectation does not translate. The vectors themselves are already covered correctly by Flux's C++ unit test src/test/script_tests.cpp, which runs each one with its exact specified flags; the JSON data files are kept for it. End-to-end enforcement of scripts in real blocks is exercised by the converted bip65-cltv-p2p, bipdersig-p2p and invalidblockrequest tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The comparison harness (comptool, blockstore), the block-construction helpers (blocktools, script) and the proxy mock (socks5) under test_framework/ are replaced by fluxtest/ and the converted top-level socks5.py, and nothing imports them any more: the converted tests use fluxtest/, and the remaining python2 scripts pull only test_framework's authproxy/mininode/test_framework/util (plus equihash, which mininode needs). The rest of test_framework/ is removed once those last scripts are migrated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Port the mempool transaction-expiry test. Flux has no separate Overwinter epoch -- the wallet jumps from legacy v1 transactions (no expiry) straight to v4 Sapling transactions carrying nExpiryHeight at the single ACADIA upgrade -- so the upstream Sprout/Overwinter/Sapling staging collapses to one boundary crossing and every version assertion is against v4/892f2085. The scenarios: a reorg past the expiry height evicts a mined transaction and its mined dependent once they are returned to the mempool; a reorg below it returns them and keeps them; unmined transactions survive chain advances up to exactly their expiry height and are evicted one block past it, releasing the spent note back to the wallet; and a transaction within the expiring-soon margin of its expiry is refused entry to a fresh mempool (tx-expiring-soon) while entry exactly at the margin is allowed. Competing chains are mined on a second node while partitioned via restart (drops all peers), and the clock-catch-up helper used for that pattern is promoted from the anchorfork test into fluxtest.network.bump_clocks. Also remove tx_expiry_helper.py: it only ever served the P2P expiry tests (p2p_txexpiringsoon, p2p_txexpiry_dos), both already converted with their own fluxtest transaction building, and the mempool test never imported it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Port the mempool network-upgrade-boundary test to the single ACADIA boundary (zcash ran the same checks twice, at Sprout->Overwinter and Overwinter->Sapling). Mining the last pre-upgrade block drops every remaining legacy-branch transaction from the mempool (removeWithoutBranchId) after a subset was mined under -blockmaxsize; transactions created for the first post-upgrade block enter under the new branch and survive; invalidating the boundary block empties the mempool entirely (new-branch entries evicted, the disconnected block's legacy transactions re-validated for the post-upgrade height and refused -- the height-off-by-one zcash documents in the original); after reconsidering, a peer that kept the new-branch transactions mines them across the boundary. Two Flux reframes beyond the boundary collapse: z_sendmany refuses ANY shielded input or output (Sprout included) until ACADIA is active for the next block, so the pre-boundary shield zcash used becomes a refusal assertion plus a first-possible-moment shield built at the last pre-upgrade tip; and the getblockchaininfo upgrades map is unusable for status assertions because it is keyed by branch-id hex and ACADIA shares 0x76b809bb with PON (duplicate JSON keys), so the branch transition is asserted via consensus.chaintip/nextblock. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Port the wallet transaction-version test to the ACADIA jump. Flux's transaction builder has no reachable Overwinter stage (the v3/0x03c48270 branch of CreateNewContextualCMutableTransaction is dead code), so the wallet goes from legacy v1 transactions -- no version group, no expiry -- straight to v4 Sapling ones (0x892f2085, expiry = next block + txexpirydelta) once ACADIA is active for the next block. The same flows are asserted on both sides of the boundary: a plain send, a z_sendmany forwarding unconfirmed transparent funds, and (post-boundary only) a Sprout shield, since no shielded transaction can be created before activation. The upstream v2 Sprout-shield and Overwinter assertions therefore have no Flux equivalent. createrawtransaction's expiryheight parameter is checked at the boundary: refused outright before it, then bounded to [0, 500000000) and to heights past the expiring-soon margin of the next block. The upgrades map of getblockchaininfo is not asserted on (ACADIA and PON share branch id 0x76b809bb, giving the JSON object duplicate keys); the branch transition is read from consensus.chaintip/nextblock. Spending unconfirmed received funds polls the once-per-second wallet notifier first, and the shield runs before the phase's other sends so ordinary coin selection cannot consume the UTXO earmarked for it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Port the ZIP209 value-pool turnstile test; Flux keeps the whole mechanism (the experimental -developersetpoolsizezero override, the CreateNewBlock miner-side exclusion, and the ConnectBlock turnstile DoS), so both the Sprout and Sapling runs convert directly as one parametrized test driven through -acadiaactivation. A node relaunched with its in-memory pool sizes zeroed excludes its own unshielding transaction from block templates, rejects a peer's block carrying it while the rest of the network accepts, and -- relaunched without the override -- reads the real pool sizes from disk and accepts the block it previously rejected. The zcash original detected the rejection by grepping logs after a stop; the port waits deterministically for the block to appear as an invalid getchaintips entry, and reads debug.log only across the stops that were already part of the flow. The shield goes through a funded ordinary output with a zero fee so the pool values stay exact (10 shielded, 9 after the unshielding block). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Port the automated Sprout->Sapling wallet-migration test (ZIP 308). Flux keeps the whole engine -- the -migration/-migrationdestaddress init flags, the 500-block cycle (operation queued at 495 mod 500 targeting the next multiple of 500, transactions committed to the mempool at 499 mod 500, expiry 450 blocks past the target) and the z_getmigrationstatus bookkeeping -- and the z_setmigration toggle, deprecated once the FLUX rebrand upgrade activates, is still live on regtest where that upgrade never does, so the cycle converts directly. The upstream regtest Sapling address and key decode unchanged (same bech32 HRPs). One genuine divergence: Flux removed zcash's default HD account-0 destination (getMigrationDestAddress without the flag is a stub that reports failure), so a destination is mandatory. The upstream default-address run becomes an assertion that a flagless node refuses to enable (z_setmigration answers with the requirement message and z_getmigrationstatus omits destination_address), followed by the same migration cycle after relaunching that node with a destination and enabling purely over RPC. The migrated-amount bookkeeping is asserted exactly (sprout + sapling == 10 minus the 0.0001 fee) where the upstream's sum check was a vacuous assert_true; the 0-conf Sapling balance at 499 mod 500 polls the once-per-second wallet notifier. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Every test is now a converted pytest module and nothing imports test_framework, so delete the remaining package (authproxy, mininode, util, test_framework, bignum, netutil, equihash) -- each piece was replaced by the fluxtest framework as its dependents were converted. Rewrite the rpc-tests README for the pytest suite (the old one described the pull-tester harness, its cache/ directory and zcash-era amounts), refresh the stale pyproject comments, and fold in an import-order lint fix in test_p2p_acceptblock the tree-wide ruff pass surfaced. The full suite still collects (171 tests) and ruff/ty pass over the whole tree. The python2-era qa/pull-tester/rpc-tests.sh runner still lists the deleted scripts and is dead; it is left in place because removing it touches Makefile.am/configure.ac and how CI invokes the suite. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
qa/pull-tester/rpc-tests.sh listed only deleted python2 scripts and nothing invoked it (CI builds and packages binaries; no workflow ran the RPC tests), so remove the directory along with its configure-generated helpers, the AC_CONFIG_FILES entries, the EXTRA_DIST references and the stale .gitignore entries. Rename qa/zelcash to qa/flux, completing a rebrand the tree had already half done: full_test_suite.py and doc/unit-tests.md were already sed-rebranded to qa/flux/... paths years ago while the directory kept its old name, leaving the checksec stages pointing at a path that did not exist. Update the remaining references (EXTRA_DIST, the deb copyright file lists). full_test_suite.py gets the minimal changes needed to actually run again: python3 shebang and print calls, text-mode subprocess output for the checksec regex stages, and an rpc stage that invokes the pytest suite (uv run --project qa pytest qa/rpc-tests --fluxd src/fluxd -m 'not slow') in place of the deleted runner. The two python2 benchmark recipes (create_wallet_200k_utxos.py, create_benchmark_archive.py) are left as-is: they are archive-creation tooling for performance-measurements.sh, not part of the test harness. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Delete the four qa/flux scripts that no longer have a working system behind them. performance-measurements.sh required manually supplied benchmark archives matched against hardcoded zcash-era hashes and drove zelcash-named helpers; create_wallet_200k_utxos.py was the python2 recipe for regenerating its 200k-UTXO archive and imported the removed test_framework; create_benchmark_archive.py was python2, defaulted to a zcash mainnet block hash and wrote through plyvel against the bundled leveldb; test-depends-sources-mirror.py checked Flux's depends tarballs against zcash's download mirror. None are referenced outside historical zcash release notes. The zcbenchmark RPC the measurement script drove still exists in the daemon; if crypto micro-benchmarks are wanted again the natural home is a small test in the pytest suite rather than the archive workflow. checksec.sh stays (the full_test_suite security stages use it). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The full-suite gate tripped a latent race in this early conversion: node 1 creates the shielded transactions but node 0 mines the confirming blocks, and without a mempool sync in between node 0 can mine an empty block before the transaction relays -- the note never confirms and z_listunspent comes back empty. Sync the mempools before both cross-node mines (the same fix wallet_persistence needed). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Gate run 3 tripped test_p2p_txexpiringsoon: its mempool-request inv came back empty, which only happens if node 0's height advanced past the carefully placed expiry window -- meaning the deliberately isolated node 2's competing block arrived early. Isolation-by-restart was not airtight: a completed addnode-onetry handshake calls addrman.Good on the peer (main.cpp:7214), persisting it in peers.dat, and without -connect the open-connections thread falls through to addrman and can spontaneously redial it (throttled hard for non-default ports, hence the rarity). Every partition-by-restart test shared the hazard. Fix in the framework config: connect=0 keeps the open-connections thread off addrman entirely (explicit addnode-onetry wiring is unaffected), with listen=1 to override the listen=0 that setting -connect implies. Also apply the audited fix to test_wallet_listreceived: the one remaining cross-node mine race in the suite (node 1's shielding z_sendmany mined by node 0 with no mempool sync, then asserted at the default one-confirmation filter). A suite-wide audit of all 86 test files found no other instance; both of the file's cross-node mines now sync first. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The framework's new connect=0 isolation setting has a side effect on proxy-configured nodes: this daemon has no special case for the value and treats "0" as a hostname to dial, and with -proxy set the name goes to the SOCKS5 proxy, so the mock servers observe a steady trickle of CONNECT "0" requests interleaved with the traffic the tests drive. Skip that sentinel in the queue reader; the assertions on the real addnode targets are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The QA reference binary now carries the daemon-side -connect=0 special case (no automatic outbound connections), so proxy-configured nodes no longer dial the literal "0" through the mock SOCKS servers and there is no sentinel noise to skip. The suite from this commit on assumes a daemon with that behaviour. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requires #289 and #288 merged first. The suite drives regtest through flags they add (
-ponactivation,-acadiaactivation) and asserts behaviour they fix (-connect=0semantics, ZMQ/AMQP notifier behaviour, witness-cache survival across restart, PoW-default regtest for wallet funding). Running it against a daemon without them fails at node startup.What this is
The python2
qa/rpc-tests/regression suite rewritten as python3 + pytest + asyncio. Each test starts its own regtest nodes in pytest temp directories (unique sub-ephemeral ports, frozensetmocktimeclocks, automatic teardown) and talks to them over JSON-RPC or raw P2P.fluxtest/— the framework: aiohttp JSON-RPC client (Decimal-exact amounts), async node process management, network wiring and sync helpers, an asyncio mininode with a working regtest equihash solver, block construction, a block-delivery comparison harness, and the regtest difficulty rule.connect=0, so no node ever dials a peer on its own: every connection and partition in a test is explicit.Coverage relative to the legacy suite
Every legacy script was converted or explicitly dropped. Three drops:
script_test(its per-vector script-flag model does not map to the fixed validation flags; the vectors are covered by the C++script_tests, and block-level script enforcement by the BIP65/BIP66/invalid-block tests) andzcjoinsplit/zcjoinsplitdoublespend(deprecated raw joinsplit RPCs). Conversions encode Flux semantics rather than zcash's: the single ACADIA upgrade gate, PoW-mode wallet funding, and the premine/150-coinbase model.Removals
The python2
test_framework/package, theqa/pull-tester/runner (referenced only the deleted scripts; nothing in CI or the build invoked it), and the zelcash-era benchmark tooling are removed.qa/zelcashis renamedqa/flux, completing a rename its own contents already assumed.full_test_suite.pyruns under python3 with itsrpcstage invoking pytest. Build-system edits are packaging-only (EXTRA_DIST, two generated test-config files dropped fromconfigure.ac).Running
(or
--fluxd=/path/to/fluxd; the fourslow-marked tests run without the marker filter). CI is intentionally untouched by this PR.Validation
Full not-slow suite green on three complete runs plus the slow tests, on a daemon carrying #289 and #288.
🤖 Generated with Claude Code