feat: pin receipt balance reads to pass head block

[rouzwelt]

Receipt inventory backfill now pins each balanceOf read to the block its logs reach, not latest. The old behavior let a lagging RPC node read zero for a receipt that just arrived, skip it, and advance the checkpoint past its block permanently. The new behavior fails the pass if the node does not have that block yet, so the checkpoint stays and the next pass retries. At startup, a failed pass stops startup and the service restarts.

Closes RAI-2834

Contributes to RAI-2601
Contributes to RAI-1215

Live effect: prevents permanent receipt loss when RPC nodes lag · Risk: high (receipt tracking in money path; startup fails on slow RPC nodes instead of skipping data) · Ships: on merge · Blocks: —

Decisions

• Read at pass head block, not latest, so a node without that block fails the call instead of returning stale zero. The pass does not move the checkpoint, and the next pass retries.
• Reconciliation reads at pass head too. A burn after the pass head can settle in inventory before the read, briefly restoring shares the burn consumed. The next pass scans the burn's block and reconciles again.

Risks

• RPC nodes slower than log-serving nodes fail startup until they catch up. Limited to: the service restarts and retries every few seconds; RPCs handle the pass head within seconds in practice.
• A burn landing after the pass head but settling before reconciliation temporarily inflates inventory. Limited to: the next pass reconciles again; burns are rare.

Proof

• Integration test: discovery reads balance at pass head even when the receipt burns in a later block.
• Integration test: reconciliation reads balance at pass head and ignores a burn outside the pass range.
• Not verified: staging or production RPC lag behavior; manual rollback of a checkpoint that skipped a receipt under the old code.

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

[linear-code]

RAI-2834

RAI-1215

RAI-2601

[rouzwelt]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• feat: fork rehearsal of RKLB orchestrator cutover on a live Base fork #441
• feat: serialize backfill discovery writes to avoid optimistic concurrency races #440
• feat: pin receipt balance reads to pass head block #439 👈 (View in Graphite)
• feat: auto-rebroadcast retained burn when still mineable before refusing resume #438
• feat: recover exhausted orchestrator mints via own transaction classification #436
• feat: alert on orchestrator mints overdue for recipient authorization #424
• docs: expand orchestrator onboarding runbook for multi-chain cutover scope, release tag hygiene, and stuck-mint escalation #423
• feat: guard against duplicate (recipient, nonce) pairs across open mints #421
• feat: recover mined orchestrator mint from MintingFailed via its own tx receipt, never vault-direct confirm #427
• feat: orchestrator cutover leaves custody at bot wallet; rollback rediscovers receipts without confirm-custody #426
• feat: guard against double-burn when tx-free failure races an in-flight burn #419 : 1 other dependent PR (#420 )
• main

---

How to use the Graphite Merge Queue

Add the label add-to-gt-merge-queue to this PR to add it to the merge queue.

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

[rouzwelt]

@rain-marvin review

[rain-marvin]

🔎 Reviewing 3c63ab0, started by @rouzwelt. The review will appear here when it's done.

[rouzwelt]

@CodeRabbit review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[rain-marvin]

This PR pins every receipt balanceOf read in the backfill to an explicit block: the pass head, or a fresh eth_blockNumber taken after the log scan when that is later. Before, the reads used latest. A lagging read node could answer zero for a receipt that had just arrived, and the checkpoint then moved past its block for good. Now a node that does not have the block fails the pass. The checkpoint stays where it was, and the next pass retries. It also answers both earlier threads: the fresh head keeps the reads on recent state after a long scan, and it shrinks the window in which a settled burn can outrun a reading.

Overall the change is correct and does what it says. The checkpoint moves only after every pinned read succeeds, and both new Anvil tests check real chain behavior. No reviewer found a blocker. Three minor points remain. Two are about how often the stricter reads fail: reads at the newest block on a load-balanced RPC, and a single read block that gets old during a long balance phase. Both can make startup restart more often, but no funds are at risk. The third is that the new SPEC sentence promises more than the code does for the log scan.

claude-opus-5-5 · high · 11 min

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: ASSERTIVE
• Plan: Team
• Run ID: 0575264b-5e50-4b07-baab-2f9d02c7a4e5

📥 Commits

Reviewing files that changed from the base of the PR and between e0a1476 and 3c63ab0.

📒 Files selected for processing (2)

• SPEC.md
• src/receipt_inventory/backfill.rs

Included review availability: This review used your included allowance. 6 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 8 reviews per hour.

---

Walkthrough

Receipt backfill now selects an explicit read block: the scanned block when no logs were found, or the later of that block and a fetched chain head when logs were found. Discovery and reconciliation balance reads use this block. A failed read aborts the pass before checkpoint advancement. The specification describes this behavior, and tests cover an unavailable block and burns occurring before and after the pass head.

Priority: ➖ Normal

Severity of issue fixed: Medium

Merge Risk: 🟡 Moderate · up to 3c63a

A large backfill may repeatedly fail and prevent startup if its RPC node stops serving the selected block during reconciliation. Resolve that deployment risk before merging unless the RPC’s historical-state support is confirmed.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: pinning receipt balance reads to the pass head block.
Description check	✅ Passed	The description explains the backfill change, its effect on checkpoint handling, and the related risks and tests.
Docstring Coverage	✅ Passed	Docstring coverage is 85.71% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 21 functions across 1 files. (1 skipped: 1 …
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Commit to this branch
• Create a new PR

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

_{Comment @coderabbitai help to get the list of available commands.}

[rouzwelt]

@rain-marvin review

[rain-marvin]

🔎 Reviewing 8f8799d, started by @rouzwelt. The review will appear here when it's done.

[rain-marvin]

This PR pins every balanceOf read in the receipt backfill (discovery and reconciliation) to an explicit block: the larger of the block the logs reach and a fresh head less two blocks, fetched after the log scan. Before, the reads went to latest, so an RPC node behind the node that served the logs could read zero for a receipt that just arrived, skip it, and move the checkpoint past it for good. Now such a node fails the pass, the checkpoint stays, and the next pass retries. SPEC.md and two Anvil tests cover the new behavior, and the mock tests get the extra eth_blockNumber call.

Overall read: the change does what it says. The floor at the log head holds on every path, errors surface before the checkpoint moves, and the earlier threads (archive-state window, reconciler at latest, log-scan clamping, settled-burn window) are resolved as the code shows. One minor point: the two-block margin rarely takes effect, because the floor is a head fetched only moments earlier, so a one-block RPC skew can still fail a short pass. This costs availability, not funds. Nothing blocks the merge.

Panel: 9 lanes. The two flash 3.7 lanes ran on composer 2.5 because flash 3.7 was not logged in. Eight lanes found nothing new.

claude-opus-5-5 · high · 10 min

[rouzwelt]

@rain-marvin approve

[rain-marvin]

🔎 Verifying that my findings from 425af02 are addressed at 425af02, started by @rouzwelt. The review will appear here when it's done.

[rain-marvin]

This PR (RAI-2834) pins the receipt backfill's balanceOf reads to an explicit block that is never below the block the log scan reaches. A lagging RPC node then fails the pass and keeps the checkpoint, instead of reading zero and moving the checkpoint past a receipt for good. After a long scan, the reads move to a fresh head less READ_BLOCK_MARGIN.

The head is still 425af02, the commit of my last review, so no code changed since then and there is no new diff to check.

Earlier findings

• READ_BLOCK_MARGIN has no effect after a short scan (src/receipt_inventory/backfill.rs:257): resolved. The const doc, the inline comment and SPEC.md now say the margin applies only after a long scan, and that a backend behind the pass head fails the pass and the next pass or the restart retries. The checkpoint never moves past an unread receipt.

Threads resolved by hand

• Pinned read can restore shares that a settled burn consumed (comment 4178143745): addressed. read_block is now fetched after the log scan (backfill.rs:245-255), which was the fix offered, so the window is the margin plus the time to the reads, not the whole scan. reconcile_receipt documents the remaining window, and the next pass reconciles the receipt again.
• Pinned reads need archive state after a long scan (comment 4178143752): addressed. The reads use a fresh head (get_block_number() after the scan, floored at the pass head), so they stay inside a full node's recent-state window.

The other threads (reconciler reads at latest, one read block per pass, SPEC scope for the log scan, pooled-RPC skew) were resolved earlier with sound reasons that still hold at this head. Nothing open remains; the PR is good to merge.

claude-opus-5-5 · high · 35 s

[rouzwelt]

@CodeRabbit approve

[coderabbitai]

✅ Action performed

Comments resolved and changes approved.

[graphite-app]

Merge activity

• Oct 6, 4:11 PM UTC: rouzwelt added this pull request to the Graphite merge queue.
• Oct 6, 4:12 PM UTC: The Graphite merge queue removed this pull request due to downstack failures on PR #419.
• Oct 6, 4:12 PM UTC: The Graphite merge queue removed this pull request due to downstack failures on PR #419.
• Oct 6, 11:43 PM UTC: rouzwelt added this pull request to the Graphite merge queue.
• Oct 6, 11:44 PM UTC: CI is running for this pull request on a draft pull request (#455) due to your merge queue CI optimization settings.
• Oct 6, 11:44 PM UTC: Merged by the Graphite merge queue via draft PR: #455.