fix(v4): make ready-signal canonicalization a pure function of the sequenced bytes (V2)#1823
Merged
Merged
Conversation
…quenced bytes (V2) record_epoch_mpc_data_ready_signal canonicalized the persisted validated_peers using a weight closure that consulted the local validator_mpc_data_announcements table: a next-epoch joiner (zero current-committee weight) was KEPT only if present in that table, otherwise DROPPED. The table is populated for relayed joiner announcements only after verify_joiner_announcement passes against the installed JoinerPubkeyProvider — a ~5s Sui-poll product installed on wall-clock time (buffered until then). So two honest validators processing the SAME sequenced ready signal persisted DIFFERENT canonical validated_peers depending on their provider-install timing: one froze a joiner, the other excluded it → divergent next-epoch mpc_data / forked reconfiguration participant set. This is the exact taint freeze_mpc_data_if_first documents it avoids, laundered one step earlier. canonicalize_ready_signal_peers is now a pure function of the signal bytes: - KEEP every deduped (peer, hash) pair — including zero-weight joiners — instead of dropping non-committee peers via the announced-table lookup. A joiner is frozen only when a stake-quorum of signers attest it in the tally (compute_freeze_partition); kept non-committee names are inert downstream (never reach a stake-quorum → land in `excluded`, consulted only by committee-member key). - Measure quorum COVERAGE on current-committee weight only, so a sparse signal still can't push the freeze trigger. - Add a deterministic length cap (K × current committee size) against a byzantine signer padding garbage into durable storage; the deduped length is identical across honest validators, so the cap decision is itself consensus-deterministic. The call site no longer reads any local table on the consensus path. Diagnostics renamed non_committee_dropped → non_committee_kept (they are kept now, surfaced only for the byzantine warn). New OverCap outcome. Tests: joiner retained in Accept while coverage stays committee-only; a signal of only zero-weight peers → BelowQuorumCoverage; over-cap dropped (pure function of input length); existing dedup/coverage tests updated. Spec: validator-mpc-data-announcements.md — receive-time canonicalization added to the ready-signal semantics and to invariant 1 (pure function of the consensus sequence covers canonicalization, not just the freeze). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…; document the non-inert residue; chain-test the pipeline Review follow-ups on the ready-signal canonicalization: 1. The "likely byzantine signer" warn fired for EVERY honest signal in a churn epoch: honest emitters deliberately attest announced next-epoch joiners (zero current-committee weight), so non_committee_kept is the expected shape of an honest signal, not a byzantine tell. Split the anomaly logging: duplicates still warn (honest emitters dedup before broadcast); zero-weight names log at debug unless the kept set exceeds committee size (no honest joiner population looks like that). 2. The spec's "kept non-committee names are inert downstream" claim corrected: a garbage name cannot reach the frozen set and assembly reads by committee key only, but it DOES land durably in epoch_excluded_validators, inflates the excluded-validators gauge, and holds the full-coverage fast path open so the freeze fires via the grace path — one byzantine signer buys the grace latency for free, deterministically. Also documents the enlarged re-emit budget (up to cap - initial accepted strict-superset REPLACEs). 3. New unit test chains canonicalize -> compute_freeze_partition -> carry_forward_stable_mpc_data in one scenario: a quorum-attested zero-weight joiner freezes at its announced hash, a silent continuing member freezes at its prior-cert digest, a fresh attestation beats the prior digest, and a kept-but-never-quorum garbage name is excluded — the exact population the determinism claim covers, previously tested only per-stage. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
CI green at the main-merged head
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Track A consensus-determinism fix from the protocol-v4 gate list (see
dev-docs/plans/v4-activation-gate-list.md). Precondition for V1's freeze determinism.The fork
record_epoch_mpc_data_ready_signalcanonicalized the persistedvalidated_peerswith a weight closure that consulted the localvalidator_mpc_data_announcementstable: a next-epoch joiner (zero current-committee weight) was kept only if present in that table, otherwise dropped.That table is populated for a relayed joiner announcement only after
verify_joiner_announcementpasses against the installedJoinerPubkeyProvider— a ~5s Sui-poll product installed on wall-clock time (buffered until then). So two honest validators processing the same sequenced ready signal persisted different canonicalvalidated_peersdepending on their provider-install timing: validator A (provider installed) keeps joiner J and can freeze it; validator B (provider lagging) drops J and excludes it. Divergent frozen set → divergent next-epoch committee mpc_data → forked reconfiguration participant set / handoff cert. This is exactly the taintfreeze_mpc_data_if_firstdocuments it avoids ("we deliberately do NOT read the local announcement table here") — laundered one step earlier, at receive-time canonicalization.The fix — pure function of the signal bytes
canonicalize_ready_signal_peersno longer consults any local table:(peer, hash)pair — including zero-weight joiners — instead of dropping non-committee peers via theannounced-table lookup. A joiner is frozen only when a stake-quorum of signers attest it in the tally (compute_freeze_partition); kept non-committee names are inert downstream (a garbage name never reaches a stake-quorum → lands inexcluded, which is consulted only by committee-member key — verified).K × current committee size, K=4) bounds byzantine garbage padding; the deduped length is identical across honest validators, so the cap decision is itself consensus-deterministic.Diagnostics
non_committee_dropped→non_committee_kept; newOverCapoutcome.Tests
Joiner retained in
Acceptwhile coverage stays committee-only; a signal of only zero-weight peers →BelowQuorumCoverage; over-cap dropped as a pure function of input length; the existing dedup/coverage/mixed-padding tests updated for the keep-not-drop semantics. All green.Spec
validator-mpc-data-announcements.md: receive-time canonicalization added to the ready-signal semantics and to invariant 1 (the "pure function of the consensus sequence" property now explicitly covers canonicalization, not just the freeze tally).Validation note
Per CLAUDE.md's reconfiguration/epoch-boundary rule, the determinism property (two validators with divergent provider timing persist byte-identical signals and compute the identical freeze partition) is validated by the cluster suite — the fault-injection scenario (force the provider absent on one node) is the pre-merge gate.
🤖 Generated with Claude Code