Skip to content

Release 1.2.0#1787

Closed
omersadika wants to merge 191 commits into
mainfrom
dev
Closed

Release 1.2.0#1787
omersadika wants to merge 191 commits into
mainfrom
dev

Conversation

@omersadika

Copy link
Copy Markdown
Contributor

Release 1.2.0: merges everything on dev into main for the mainnet binary upgrade 1.1.8 -> 1.2.0 (~45 PRs).

Merge order: the companion prep PR #1786 (chore/release-1.2.0 -> dev) must land first — it syncs main's CI commits into dev (resolving the four conflicting workflow files in dev's favor) and carries the 1.2.0 version bump. Until it merges, this PR shows conflicts.

Upgrade-compatibility posture

What's in this release

Network-key lifecycle & reconfiguration

Off-chain validator metadata & committee

OCS — verified Sui reads over an untrusted relay

Signing & presign

Robustness fixes

Test & CI infrastructure

Pre-merge validation

A deep multi-agent code review of this full diff is running; findings will be posted as a comment on this PR. Required green before merge (on dev after the prep PR lands):

  • upgrade-test.yaml — scenarios v118_upgrade, v118_churn, cross_binary
  • test-cluster.yaml — full cluster suite
  • integration-tests-ci.yaml — Rust integration suite
  • ts-integration-tests.yaml — full TS suite against a localnet

🤖 Generated with Claude Code

omersadika and others added 30 commits August 14, 2025 10:56
Co-authored-by: dehed <david.lachmish@gmail.com>
Co-authored-by: Itay Levy <itaylevy134@gmail.com>
…ly when some validators receive the start event before receiving the network key (#1425)
…ive session's message before its start event (#1427)
@omersadika

Copy link
Copy Markdown
Contributor Author

Deep review of the 1.2.0 release diff

Produced by a 14-dimension multi-agent review of the full main...dev diff (~45 PRs, ~85k insertions): every subsystem reviewed independently, every blocker/major/minor finding adversarially verified by 1-2 independent refuters (110 agents total; 83 raw findings, 6 refuted, 77 surviving), then synthesized with cross-PR interaction analysis and a completeness audit.


ika 1.2.0 Release Review — origin/main...origin/dev (final report)

Scope: full release diff (~45 PRs), fourteen subsystem reviews, adversarially verified findings, plus targeted re-verification done for this report (chain-anchor tags, blending first-element in cryptography-private @ 32a27aa, workflow/scenario inventory on dev).

Bottom line: dev is NOT ready to merge to main as 1.2.0. Two confirmed blockers would take down the live network — one wedges mainnet/testnet in a network-wide crash loop at the first epoch reconfiguration, the other breaks consensus interop with 1.1.8 peers within seconds of the first upgraded validator restarting. Both have small, well-understood fixes. Two more must-fix items (a v3 behavior change in Schnorr presign timing, and the version bump the tag pipeline hard-requires) round out the gate list.


1. Final ranked findings

Duplicates merged across dimensions (noted per entry). Severity reflects verifier votes where they adjusted. GATES = must fix before dev merges to main; ROLLOUT = must be done before the mainnet tag/restart but can land as release-process work; v4-GATE / v5-GATE / OCS-GATE = must fix before that feature ever activates (ships dark in 1.2.0); FOLLOW-UP = fix on normal cadence.

1.1 Release-gating (GATES / ROLLOUT)

G1. BLOCKER (confirmed, chain-verified twice) — Deployed network keys are V1-tagged on chain; both reconfiguration input builders panic on them.
crates/ika-core/src/dwallet_mpc/crytographic_computation/mpc_computations/reconfiguration.rs:274 (v3 path) and :150 (v4 path). Merged: network-key-lifecycle blocker + its v4-companion major.
Mainnet and testnet network_dkg_public_output chunk 0 begins with BCS variant byte 0x00 = VersionedNetworkDkgOutput::V1 (read directly from both chains by two independent verifiers). 1.1.8 has a functional V1 arm that live mainnet exercises every epoch; this release replaced it with unreachable!("V1 network DKG outputs are no longer produced") in reconfiguration_bwd_compat_public_input — the arm selected at protocol v3. With workspace panic='abort', every 1.2.0 validator aborts at the first per-epoch reconfiguration session and crash-loops on replay: both live networks wedge, epochs can never close. The v4 generate_public_input has the identical unreachable!, and reconstruct_full_network_dkg_output fires only for (V2 anchor, V3 output), so the V2→V3 canonical migration can never fire for the real deployed keys — the flagship v3→v4 upgrade epoch would abort the same way. Root cause is a false premise ("deployed anchor is V2") recorded in dev-docs/plans/network-dkg-anchor-v2-to-v3-migration.md and the struct docs, contradicted by the tool comment (network_dkg.rs:847,:940) shipped in the same release.
Fix: reinstate a functional V1 arm in the bwd-compat builder mirroring 1.1.8 (decode V1 inner bytes, feed new_from_reconfiguration_output; confirm the constructor's accepted type with cryptography-private); decide a V1-anchor story for the v4 dispatcher and the migration (one-shot V1→V2 canonicalization at the mirror, or extend reconstruct_full_network_dkg_output + both dispatchers to accept V1); audit all 8 V1(_) => unreachable! sites (including dwallet-mpc-centralized-party/src/lib.rs:999, which panics in the shipped WASM SDK on real chain bytes); add an integration test feeding the deployed shape (V1 anchor + V2 reconfiguration output) through session_input_from_request at v3; correct the plan doc, handoff.md's "V2 origin" claim, and the struct docs. GATES.

G2. BLOCKER (confirmed twice, independently by two dimensions) — Ungated IdleStatusUpdate consensus emission at protocol v3 breaks 1.1.8 interop.
crates/ika-core/src/dwallet_mpc/dwallet_mpc_service.rs:662 (emission), vs 1.1.8's consensus_validator.rs whole-batch reject + consensus_output_api.rs:107 panic. Merged: consensus-messages-compat blocker + protocol-config-gating blocker (same defect).
send_status_update_to_consensus has no protocol gate; last_sent_idle_status starts None, so the first 20ms service iteration after every restart submits ConsensusTransactionKind::IdleStatusUpdate. 1.1.8 cannot decode the variant: its verify_batch rejects the entire block (silently ejecting each upgraded validator and its descendant blocks from old nodes' DAGs), and once upgraded stake reaches 2f+1 and such a block commits, every remaining 1.1.8 node panics deterministically and crash-loops. This breaks even the "atomic" swap's unavoidable seconds-to-minutes restart skew — it converts the documented MPC-level mixed-window degradation into a consensus-layer break. Every sibling new kind IS gated (EndOfPublishV2, announcements, GlobalPresignRequest, NOA handlers, and the same feature's own DB write/read streams); only this wire emission was missed. The repo's tests structurally cannot catch it (cross_binary's OLD binary is a same-branch v3-pin that decodes the variant; v118 swaps atomically).
Fix: one line — gate the branch on internal_presign_sessions_enabled() (symmetric with the write gate at authority_per_epoch_store.rs:5126 and read gate at dwallet_mpc_service.rs:1084). Then prove it: rerun the v118 rehearsal with a deliberately staggered swap and confirm 1.1.8 stragglers keep running (per the test-testing playbook — log evidence, not exit code). GATES.

G3. MAJOR (confirmed) — schnorr_presign_second_round_delay (8 rounds) applies to external Schnorr-family presigns at v3, where 1.1.8 used zero delay.
crates/ika-core/src/dwallet_mpc/crytographic_computation/mpc_computations/presign.rs:115 fed from ika-protocol-config/src/lib.rs:643 (base config, visible v1..3).
Taproot/EdDSA/Schnorrkel presigns are externally requestable at v3. During any mixed 1.1.8/1.2.0 window, upgraded validators advance presign round 2 eight leader rounds later with a potentially larger message set — different participant sets → byte-divergent outputs → output-quorum split (this codebase's known false-malicious/stall failure class). Even post-swap it is a v3 behavior change (+~2s latency) violating the "v3 byte-identical to 1.1.8" requirement. Note this is also the actual residual of the disputed take-first presign finding (see D1): the blending itself is byte-safe; the delay is what diverges the cohorts.
Fix: version-gate — None in base config, Some(8) in the v4 arm (consumer treats None as no delay), or branch in PresignAdvanceRequestByProtocol::try_new on a v4 predicate. Third-round delay needs no gate (VSS-only, unreachable at v3). GATES.

G4. MAJOR (confirmed) — Release cannot be tagged: workspace version is 1.1.9 and release.yaml hard-fails the tag/version match.
Cargo.toml:32 vs .github/workflows/release.yaml:91-93. Pushing release/mainnet-1.2.0 fails the first job; the workflow_dispatch path rejects bare versions, so there is no alternate route.
Fix: include version = "1.2.0" (+ lockfile refresh) in the release PR to main before tagging. ROLLOUT (trivial, but hard-blocks the tag).

G5. MAJOR (confirmed twice) — No automated coverage of the exact rollout-day combination: 1.2.0 binary + 1.1.8-era old-style config.
crates/ika-swarm-config/src/node_config_builder.rs:188 (all builders emit new-style configs). Every harness on dev runs the new binary on sui_data_source/gRPC; nothing constructs a sui-rpc-url-only config for it — yet that is what every mainnet node will run, and this release touched the legacy path (SuiSyncer re-gating, Option-alized senders). Compounding it (related finding, verifier-downgraded to minor but operationally load-bearing): ika-config/src/node.rs:406 silently flips old-style notifier/fullnode configs from JSON-RPC to gRPC at the same URL, and the notifier is the sole submitter of checkpoints and advance_epoch — if its endpoint doesn't serve Sui gRPC, boot fails and epoch advance stalls network-wide.
Fix: before tagging, run at least one v118-style scenario where the NEW binary boots from an old-style YAML in Validator and Notifier modes and crosses an epoch boundary (add a with_legacy_sui_rpc_config() builder knob); add a rollout-notes preflight ("notifier sui-rpc-url endpoint must serve Sui gRPC") and a boot-time error message naming the requirement. ROLLOUT.

G6. MINOR (confirmed) — Systemic pattern behind G2/G3: all ~45 new config values injected into the base config (visible at v1..3), contra the file's own "never modify a pre-existing protocol version" invariant.
crates/ika-protocol-config/src/lib.rs:628+. Safe only where every consumer is v4-gated — the manual audit missed exactly two consumers (G2's idle threshold, G3's delay). Side effect: 1.2.0's v3 protocol-config digest differs from 1.1.8's (benign for the v4 vote as implemented, but forfeits the digest's drift-detection purpose).
Fix: in the same PR as G3, move new values to the 4 => arm as None-in-base (the macro getters' unwrap then mechanically enforces "consumer is version-gated"); keep bls_checkpoints with a one-line "false-branch = 1.1.8 behavior" justification. GATES (as the pattern fix riding G3), pattern adoption itself FOLLOW-UP.

1.2 Must fix before protocol v4 activates (ships dark in 1.2.0)

V1. MAJOR (confirmed 4x) — Freeze carry-forward silently degrades to announce-only on a missing/unreadable prior-epoch handoff cert; async, non-retrying joiner bootstrap lets the freeze fork the frozen set.
crates/ika-core/src/authority/authority_per_epoch_store.rs:3363. Merged: epoch-lifecycle-handoff major + specs-vs-code major (same defect, spec-invariant angle).
prior_epoch_mpc_data_digests() returns an empty map on Ok(None) AND on read Err (warn + degrade), and freeze_mpc_data_if_first writes the frozen set once, irreversibly. The barrier claim in its comment covers only continuing validators; a joiner's bootstrap is tokio::spawn'd fire-and-forget with a 300s give-up (ika-node/src/lib.rs:2397, 2550-2561). Divergent frozen sets → byte-divergent reconfiguration outputs → false-malicious conviction or reconfiguration quorum failure (wedged close). Directly contradicts validator-mpc-data-announcements.md invariant 1 and the handoff spec's read-error-must-retry rule (which adopt_cert_verified_keys implements correctly — this is the one consumer that degrades).
Fix: block epoch entry (or consensus processing) for joiners until the E-1 cert is verified locally with infinite Unavailable retry; make cert read errors in the freeze path fail-stop (Err → commit replays); keep the empty-map path only for chain-true no-cert epochs. v4-GATE.

V2. MAJOR (confirmed) — Ready-signal canonicalization filters against the wall-clock-timed local announcements table; recorded freeze inputs diverge for relayed joiners.
authority_per_epoch_store.rs:3210. Two honest validators processing the same sequenced signal record different canonical validated_peers depending on whether their JoinerPubkeyProvider (a ~5s Sui poll product) had installed. The freeze block itself refuses to read the announcements table for exactly this reason — the same dependency is laundered one step earlier. Fix: keep unknown zero-weight pairs in the persisted canonical signal (deterministically bounded), or verify relayed joiner announcements against a consensus-deterministic pubkey source and never buffer on the consensus path. v4-GATE.

V3. MAJOR (confirmed) — Restart after own EndOfPublishV2 sequenced permanently loses the expected-attestation install; post-restart peer signatures buffer forever, close round diverges.
crates/ika-core/src/epoch_tasks/handoff_signature_sender.rs:246. The early return on has_recorded_end_of_publish_vote (durable) runs before build/install; expected_handoff_attestation is in-memory-only; buffered-quorum adoption counts only the post-restart buffer. A rolling restart near epoch end — exactly what a release does — can put >f validators in this state: no quorum on any close round, final checkpoint never certifies. Fix: run snapshot-ready check + build + install before the early return (install is idempotent), or persist the expected attestation per-epoch and re-install at store open. v4-GATE.

V4. MAJOR (confirmed; one verifier: minor) — v4 close's handoff-cert gate is not a pure function of the consensus sequence; a divergent-attestation validator closes at the grace×4 backstop and forks its local final checkpoints.
authority_per_epoch_store.rs:4128 (the code comment admits it). Bounded to ≤f stake by quorum math, so the network closes — but the divergent validator's fault-tolerance is silently spent. Fix: tally sequenced EndOfPublishV2 bundles per attestation digest unconditionally (verifiable from sequenced bytes + registration-fixed keys), so cert-quorum detection is sequence-pure; a diverging validator then fails at the barrier, not as a checkpoint fork. v4-GATE.

V5. MAJOR (confirmed; one verifier: minor) — One unverifiable validator_info aborts whole consensus-key batches: joiner-provider install and prior-committee chain fallback fail wholesale.
crates/ika-core/src/sui_connector/pubkey_provider_updater.rs:297 and :103. Move validates key lengths only; Rust verify() enforces more (consensus≠network pubkey, BLS point validity), so one crafted registration passing Move suppresses the entire joiner-announcement relay pipeline for an epoch. Fix: skip-and-log the unverifiable member (matches the established sui_syncer per-validator drop pattern); per-member skip is sound for the prior committee because cert verification needs only quorum stake. v4-GATE.

V6. MINOR (confirmed; verifier-downgraded from major) — Process-static NetworkKeyId mapping wedges cert-gated adoption for any non-seeded key after a restart.
crates/ika-core/src/network_key_id_mapping.rs:42. Adoption requires the mapping, the mapping requires instantiation, instantiation requires adoption — no release path after restart for keys outside the two hardcoded SEED entries (every localnet/CI cluster, any future new key). Integration tests hide it by calling register manually. Fix: derive the content-derived id on demand from the locally-held blob (the print_deployed_network_key_ids tool already does this derivation) or persist registrations in a perpetual table; at minimum make "unmapped + cert has DKG items" defer-and-retry instead of routing to the digest-mismatch skip. v4-GATE.

V7. MINOR (confirmed; both verifiers downgraded from major) — Committee gained a bcs mid-struct field (consensus_keys); 1.1.8-written committee_map records are undecodable by 1.2.0.
crates/ika-types/src/committee.rs:104. Unreachable at v3 today (no production reader of old records), but a latent landmine: RocksDbStore::get_committee has an .unwrap() panic path (storage.rs:118), and at the v3→v4 boundary the prior-epoch local-anchor read .ok().flatten()s the decode failure into "never persisted", silently forfeiting the designed local trust-anchor path at exactly the transition epoch. Fix: legacy-layout fallback decode (empty consensus_keys) or a versioned envelope; replace the unwrap with propagation; log decode-failure distinctly from absent. v4-GATE (before the boundary epoch).

V8. MINOR (confirmed) — Internal-presign refill gate has no release path on a permanently failed session; MINOR — NOA-signing-key selection tie-breaks on HashMap iteration order.
mpc_manager.rs:1585 and :1471. Frozen pool → global-presign starvation → undershoot wedge; and equal dkg_at_epoch keys can desynchronize the shared internal-presign sequence counter (consensus-visible session ids). Fix: reconcile counters on terminal failure (or track failures explicitly); min_by_key(|(id, k)| (k.dkg_at_epoch, *id)). v4-GATE.

V9. MINOR (confirmed) — VSS derivation .ok()? black hole; stale handoff_signatures rows counted across a mid-epoch attestation re-install; prior-committee consensus keys mis-keyed under boundary key rotation.
network_dkg.rs:430 (silent None conflates "pre-V3 key" with real derivation failure → untraceable WaitingForNetworkKey on every VSS sign); authority_per_epoch_store.rs:3119 (replay drops rows from the aggregator but not the table, so the close gate can count a mixed set); pubkey_provider_updater.rs:108 (chain-fallback map keyed by post-rotation names/keys — rotated signers' stake silently stops counting in cert verification). Fixes: as per findings (Result-with-NotApplicable + one error! per key; batch-delete non-verifying rows in the install replay; key the map by snapshot-derived names). v4-GATE / FOLLOW-UP.

1.3 Before the NOA/v5 flag ever ships (not deployed at all)

N1. MAJOR (confirmed) — NOA store status-transition panics assume local lockstep with quorum; the documented restart-replay recovery path deterministically panics.
crates/ika-core/src/noa_checkpoints/local_store.rs:262. Replay applies all round resolutions before draining the sign-output channel, so mark_finalized on a None-status entry → panic → crash loop; threshold-signing also lets 2f+1 fast validators legitimately outrun a lagging one live. Fix: make quorum-driven transitions monotone (quorum is authoritative: Finalized from any state incl. None; retry tolerates None/RetryPending); reserve panics for genuinely impossible transitions. v5-GATE.

N2. MINOR (confirmed; verifiers downgraded from major) — all_finalized_flag reads "done" during the initial signing window; end-of-publish can close the epoch with an in-flight NOA checkpoint. noa_checkpoints/handler.rs:266. Fix: done = entries.is_empty() && no buffered messages || all Finalized. v5-GATE.

N3. MINOR (confirmed) — remaining NOA items, batched: no domain separation in signable bytes (ika-types/src/noa_checkpoint.rs:213 — include KIND_NAME + counterparty chain in the tuple, state the invariant for the real TransactionData construction); sign output lost on full try_send channel (mpc_manager.rs:2742 — buffer and retry); failure-quorum consumed before dispatchability check (dwallet_mpc_service.rs:1254 — verifier-downgraded to refactor; move the dedup insert to dispatch time); observation maps accept attacker-fabricated keys, never pruned (mpc_manager.rs:1273 — epoch/round bounds + prune on finalization); multi-hundred-KB Debug info-log of certified checkpoints (handler.rs:129, unverified — drop the field). Plus the acknowledged-scaffolding state (observation hardcoded None, LogOnly submitter, placeholder tx bytes): record the enablement checklist next to the commented v5 block. v5-GATE.

1.4 OCS opt-in verified-reads paths (config-shape gated; not active on mainnet configs)

O1. MAJOR (confirmed) — Direct-node fold caches object bytes not bound to the proven object ref: malicious-fullnode cache poisoning, sticky forged state, permanent read wedge.
crates/ika-core/src/sui_connector/push_worker.rs:362. Violates spec invariant 8; the artifacts digest binds effects only, output_objects is entirely unchecked; never-downgrade + persistence make the forgery sticky and poison the high-water. Fix (one line): .find(|o| o.compute_object_reference() == object_ref) (also fixes the benign multi-tx intermediate-version case); skip-and-warn when no output matches. Cheap enough to take in this release even though OCS is dark. OCS-GATE (recommend: take now).

O2. MAJOR (confirmed) — Changeset currency gate silently dead on any pruned (i.e. real) chain, sui_connector/setup.rs:379 (bootstrap_from near genesis × relay prune-floor empty pages × Ok(0)-means-caught-up); which masks O3. MAJOR — ChangesetIndex built without the fold filter (setup.rs:372, with_fold_filter has zero production callers; multi-GB index at mainnet scale once O2 is fixed); and O4. MAJOR — unbounded byzantine bag-walk pagination (bag_event_pump.rs:297, a malicious relay spins one tick forever / grows memory unboundedly). Fixes: re-anchor from_seq on repeated empty pages (server reports its floor) + dormancy log; wire the known stable id set into with_fold_filter; cap the walk at expected_size × slack + page ceiling + dedup. OCS-GATE.

O5. MINOR (confirmed/unverified) — OCS hardening batch: pusher permanently skips a checkpoint on any transient fetch error at debug level (push_worker.rs:175 — distinguish NotFound from Network, hold cursor, warn); forged non-monotonic effects panic the pusher via upstream assert! under panic=abort (push_worker.rs:337 — pre-validate or build the map yourself); archive cold-bootstrap one-shot latch on transient failure (ocs_verifier.rs:245, unverified — latch only on success/determinate failure); VerifiedSuiTransport flattens NotFound/determinate proof failures into Network (verified_transport.rs:83, unverified — pass ReaderError::Transport through). OCS-GATE / FOLLOW-UP.

1.5 Test-infra: fix before a green run is used as upgrade evidence

T1. MAJOR (confirmed) — Malicious-cross-binary detection is satisfiable by the faulty validator's self-report; honest 1.1.8 binaries don't even export the gauge. crates/ika-upgrade-test/tests/malicious_cross_binary.rs:123. Fix: scrape only named honest validators, fail loudly when no scraped validator exposes the gauge, or assert on committee exclusion. ROLLOUT-evidence.

T2. MAJOR (confirmed) — Sign-flow integration tests never check SignOutput.rejected; a network rejecting every signature stays green. dwallet_mpc/integration_tests/sign.rs:145,275,760. Fix: assert!(!sign_output.rejected) (+ verify signature bytes); same for the partial-signature output. ROLLOUT-evidence — do this before the pre-merge suite runs, it is nearly free.

T3. MAJOR (confirmed) — v118 "pre-activation-window" probe is not pinned to protocol v3; timing drift silently degrades the mainnet-deadlock rehearsal into an ordinary v4 pool test. crates/ika-upgrade-test/tests/v118_upgrade.rs:190. Fix: assert protocol ≤3 immediately before and after the probe workload. ROLLOUT-evidence — must fix before the mandatory pre-merge v118 runs (Section 5), or their green is not evidence for the property this release most depends on.

T4. MINOR (confirmed) — test hygiene batch: removal test asserts nothing about removal and skip(1)s a HashMap-ordered node (ika-test-cluster/tests/joiner.rs:193); validator_restart.rs contains no restart and its pool test exercises only the in-memory mock (:335); is_running() never try_wait()s so post-health crashes pass unnoticed (ika-upgrade-test/src/process.rs:273); churn cert assertion accepts 1 cert across 5 epochs × 7 nodes (joiner.rs:811, unverified). FOLLOW-UP.

1.6 SDK / docs / CI hygiene

S1. MAJOR (confirmed; gates next npm publish, not the validator release) — SignatureAlgorithm.SchnorrkelSubstrateSchnorrkel rename (key AND runtime string) with no version bump/changeset/alias; npm already has 0.4.1 with the old name. sdk/typescript/src/client/types.ts:225. Fix: bump to 0.5.0, changelog + migration note, optional @deprecated alias key with the new string value. Fold in the stale SchnorrkelSubstrate at README:349. FOLLOW-UP (hard-gates next SDK publish).

S2. MINOR (confirmed) — SDK/docs batch: default poll timeout silently 30s→10min for all *InParticularState waiters (ika-client.ts:1361 — changelog it); README documents a 120s runner default (actual 1200s) and recommends --timeout 300, below the SDK's own 600/900s budgets (README.md:530); DoubleSHA256-for-secp256r1 claimed in docs, rejected by both TS and Rust tables (README.md:347, types.ts:169,211); phantom lockfile importer sdk/signature-mpc-wasm/pkg (pnpm-lock.yaml:103 — regenerate + negative workspace glob). FOLLOW-UP.

S3. MINOR (confirmed) — CI/docs batch: main-only target-cpu=native RUSTFLAGS silently absent from dev's workflow rewrites — dev's measured budgets suggest it's unneeded, but decide consciously and note it in the merge PR (ts-integration-tests.yaml:131); metric ratchet nags to prune the six format!-built pruner names its own header says to keep (scripts/check-metric-names.sh:62); four internal crypto-migration working docs committed under public docs/ instead of dev-docs/plans/ — with a developer-machine path and private-repo revs in the public repo (move them, ideally before the merge fires deploy-docs); simtest.yaml points at a deleted "## Simtest" CLAUDE.md section; CLAUDE.md says Rust 1.93, pin is 1.94 (unverified). FOLLOW-UP (move the docs/ files with the release PR).

S4. MINOR (confirmed) — Spec-staleness batch (one docs PR): merged: both handoff deferred-close drift findings + the remaining specs-vs-code minors. Update handoff.md for the full v4 close predicate (all-voted short-circuit, handoff-cert-quorum gate, grace×4 backstop + barrier-wedge warning; non-pure-gate note), the withhold-EOP-not-omit-item rule, invariant 2's overbroad "never suppresses votes" wording, and the stale pubkey-provider buffering clause; validator-mpc-data-announcements.md for the phantom "needs-data" freeze condition and the strict-superset/coverage-floor ready-signal semantics + batch-write attribution; ocs-verified-sui-reads.md invariant 3 (operator-pinned anchor is gone); fast-schnorr-vss.md external-reachability table. FOLLOW-UP (same-PR-as-behavior rule says: land with the release).

1.7 Disputed / refuted — recorded so they aren't re-litigated

D1. DISPUTED→effectively refuted — external presign presigns.first() byte-identity. presign.rs:480. Re-verified for this report: pinned cryptography-private 32a27aa mpc/src/blending.rs:87 constructs the first "blended" element as the plain unblended aggregated contribution, so take-first is byte-identical to 1.1.8's output by construction. The genuine mixed-window divergence in this code path is G3 (the round-2 delay), which is confirmed and gating. No separate action beyond G3.

D2. DISPUTED — validator accepts v4/v5-only consensus kinds at any protocol version. consensus_validator.rs:83. The accept/reject asymmetry vs 1.1.8 is real; the claimed v3 side-processing was partially wrong. After G2's fix the practical exposure collapses. Worth doing as defense-in-depth (reject kinds whose introducing flag is off — deterministic, epoch-uniform) so a future premature emission surfaces as the emitter's blocks being refused rather than as old-binary crashes. FOLLOW-UP.

Refuted-on-verification suspicions retained in the record (no action): bootstrap Rejected does halt fail-closed; buffered-quorum HashMap pick is deterministic under ≤f byzantine; stake-aggregator eviction change is local-only and v3-replay-safe; V2-digest hydration cannot un-flip the V3 migration (overlay resolves the flipped mirror).


2. Cross-PR interaction analysis

Risks that exist only because of combinations of changes in this release:

  1. Crypto-bump migration × v3/v4 dispatcher split (G1). The V2→V3 canonical-migration work (feat(dwallet-mpc): V2->V3 canonical network DKG migration #1758, feat(dwallet-mpc): content-derived NetworkKeyId for the cross-epoch handoff #1773) wrote its planning doc as if the deployed anchors were already V2, and the bwd-compat/main dispatcher split then deleted the V1 arm on that premise — while a tool comment added in the same release states the opposite (correct) fact. Neither PR alone is fatal; the interaction removed 1.1.8's only code path for the real chain bytes. Lesson embedded in the fix: the migration's own preconditions must be verified against chain state, not against the plan doc.

  2. Internal-presign feature × gating discipline × upgrade-harness design (G2). The feature's DB streams were carefully double-gated; the wire emission was missed; and two other PRs made the miss invisible — cross_binary's OLD binary became a same-branch v3-pin (which decodes the new variant), and v118 was specified as an atomic swap. Three individually-reasonable choices compose into an untestable consensus break. The fix must therefore include the staggered-swap rehearsal, not just the one-line gate.

  3. Carry-forward freeze (fix(reconfiguration): carry forward stable mpc_data so a late member doesn't wedge the next epoch (#1742) #1741) × async joiner bootstrap × cert-gated adoption × content-derived NetworkKeyId (V1/V2/V6). The freeze fix moved the announcement dependency into the prior-epoch cert — correct — but three different wall-clock-fed local installs (joiner bootstrap task, JoinerPubkeyProvider, expected-attestation install) now feed consensus-consumed state (frozen set, canonical ready signals, signature rows). Meanwhile the content-derived NetworkKeyId (feat(dwallet-mpc): content-derived NetworkKeyId for the cross-epoch handoff #1773) is held in a process-static map whose only production writer is instantiation, which the cert-gated adoption path (fix(dwallet-mpc): network-key adoption gates — no committee-unapproved parameter set may go live #1735 lineage) depends on. Each PR narrowed one race and opened a dependency on another's timing. The common repair shape: consensus-consumed state must be a function of the sequence plus durable, barrier-guaranteed local artifacts — never of "has my background poll finished".

  4. Consensus-keys-as-committee-data × committee persistence × OCS committee ratchet (V7). Carrying consensus keys on Committee (the PR this branch was for) is the right pivot, but it interacts with the perpetual committee_map (1.1.8 records undecodable) and with the v3→v4 boundary local-anchor read that the OCS/genesis-rooted-trust work introduced — the .ok().flatten() silently converts the decode break into "no local anchor", pushing the joiner-bootstrap trust root onto a chain read at exactly the epoch it matters.

  5. Changeset currency gate (OCS: verified Sui reads over an untrusted relay + peer-only validators #1744) × genesis-rooted bootstrap (OCS: replace the trusted anchor with a genesis-rooted committee chain #1774) (O2→O3). The currency receiver bootstraps from the oldest verifiable committee summary; the genesis-rooted work made that summary sit near chain genesis; the relay pages break at the prune floor. Result: the whole currency subsystem is dormant on any real chain — and its dormancy masks the unwired fold-filter OOM that fires the moment someone fixes the bootstrap. Fix them together or the second becomes a production incident.

  6. Base-config value injection × manual v4-gating audit (G6→G2, G3). ~45 new values made visible at v1..3, with safety resting on a by-hand "every consumer is v4-gated" audit that missed exactly two consumers — both of which became release-gating findings. The pattern is the bug; the two escapes are its instances.

  7. SDK timeout redesign × pre-existing README runner section (S2). The README section landed before the timeout redesign and now actively recommends settings that reinstate the failure mode the redesign eliminated.

  8. Dev workflow rewrites × main-only codegen fix (S3). Dev's CI workflows evolved from a pre-fix copy of main's; the merge silently drops main's target-cpu=native pair without anyone having decided to.


3. Top refactor suggestions (value ÷ effort, descending)

  1. SuiSyncer::run ingestion enum (sui_syncer.rs:96-176): replace run_legacy_event_ingestion: bool + two Option senders with enum EventIngestion { Legacy { new_requests_sender, uncompleted_requests_sender }, BagEventPump } constructed at the single decision point in sui_connector/mod.rs. Kills a runtime .expect() on a validator task, drops two args at the clippy arg-count ceiling, makes the invariant compiler-checked.
  2. Delete dead ConsensusNetworkKeyData (ika-types/src/messages_dwallet_mpc.rs:182): zero uses; a serde wire-type inviting revival of the mechanism the handoff cert replaced. Also re-check the Hash derive on DWalletNetworkEncryptionKeyData added for it.
  3. NOA v5 enablement checklist next to the commented flag block (ika-protocol-config/src/lib.rs:737): one comment listing the hard prerequisites (real SuiChainObservation, real submitter + TransactionData bytes, N1/N2 fixes) so the flag cannot be flipped without them — currently discoverable only via three FIXMEs in three files.
  4. Collapse PubkeyProviderUpdater fn-pointer plumbing (pubkey_provider_updater.rs:40): one instantiation exists post-pivot (Committee is itself the provider); inline selector/installer, drop label.
  5. ocs_currency cap honesty (ocs_currency.rs:883): rewrite the "no cap" test to exercise the actual 256-entry cap, and add a Dropped outcome for the cap-full branch that currently returns Queued while logging "dropping".
  6. bls_checkpoints simplification (lib.rs:730): delete the redundant v4 re-assignment; either drop the flag until the cutover version exists or extend the v5 template to show noa_checkpoints=true && bls_checkpoints=false.
  7. Runner-script arg guards + trap print_summary INT TERM (run-integration-tests-sequential.sh:45).
  8. Dockerfile.runtime jemalloc cleanup (docker/ika-node/Dockerfile.runtime:14): apply the sibling Dockerfile's fix — drop libjemalloc-dev, dead ENV LD_PRELOAD, dead RUN; the shipped mainnet image currently embeds a comment this release proved false.
  9. Test-cluster doc hygiene (ika-test-cluster/src/lib.rs:808,197): re-unite the split doc comment; rename current_epoch_from_chaincurrent_node_epoch (it reads node memory, and the harness itself documents chain-vs-node lag as a trap).
  10. One-line uniqueness comment on quorum_attestation_in_buffer (handoff_cert.rs:368): record the ≤f-byzantine argument for why HashMap iteration order can't select between two quorums, so the next reviewer doesn't re-derive it.

4. Release verdict

Not ready to merge. Required before dev → main:

Must-fix code changes (GATES):

  1. G1 — reinstate functional V1 anchor handling on the v3 bwd-compat path (mainnet/testnet wedge), plus at minimum a decided-and-tracked V1 story for the v4 dispatcher/migration and the unreachable! audit (the v4 half may technically ship dark, but it's the same fix session and the same false premise; the plan doc/spec corrections go in the same PR).
  2. G2 — gate IdleStatusUpdate emission on internal_presign_sessions_enabled().
  3. G3 (+G6 pattern) — version-gate schnorr_presign_second_round_delay so v3 = 1.1.8 exactly; move new config values out of the base config while there.
  4. New regression tests riding the fixes: deployed-shape (V1 anchor + V2 reconfiguration output) through session_input_from_request at v3; a staggered-swap interop check for G2. Fix T3 (pin the v118 pre-activation probe to protocol ≤3) and ideally T2 (!rejected asserts) first, so the mandated green runs below actually witness the properties they exist for.

Release-process requirements (ROLLOUT):

  • G4Cargo.toml → 1.2.0 in the release PR to main (tag validation hard-fails otherwise).
  • G5 — one rehearsal of the new binary on an old-style (sui-rpc-url-only) config in Validator and Notifier modes across an epoch boundary; release-notes preflight for the notifier gRPC endpoint requirement.
  • Move the four internal migration docs out of public docs/ (S3) with the release PR.
  • Release notes must state the operational swap plan explicitly (post-G2 the consensus layer survives a rolling window, but the harness's own docs still scope MPC wire-compat to same-branch builds — mixed windows should stay short).

CI suites that MUST be green on the fixed dev head before merge (workflow file → scenario/filter, all present on dev):

Workflow What must run
ci.yaml default push gates (fmt/clippy/build, ika-protocol-config snapshot tests — will catch the G3/G6 config change)
upgrade-test.yaml v118_upgrade and v118_churn (literal mainnet-v1.1.8 old binary — the direct rehearsal of this rollout; with T3 fixed so the pre-activation probe is pinned), cross_binary (rolling swap + churn), plus smoke and workload; additionally one staggered v118-style run as the G2 proof (1.1.8 stragglers keep running while 1.2.0 nodes emit)
test-cluster.yaml full nextest run — must include protocol_version_transition, restart_mid_grace, joiner, off_chain_metadata, ocs_verifier, epoch_boundary_presign_traffic, multi_network_key_dkg
integration-tests-ci.yaml full Rust integration job incl. dwallet_mpc::integration_tests (with the new V1-anchor test and T2's tightened sign asserts)
ts-integration-tests.yaml full TS SDK suite against the localnet
simtest.yaml test_swarm_reaches_epoch_2 (as configured)

(scheduled-all-suites.yaml only dispatches — verdicts must be read per-suite. release.yaml needs no change; it is byte-identical on both branches and works once G4 lands. No contract upgrade ships: deployed_contracts/{mainnet,testnet} are byte-identical between main and dev.)

After merge, tracked follow-ups by activation gate: V1–V9 before any protocol-v4 activation vote; O1–O5 before any OCS opt-in deployment (take O1's one-line fix now); N1–N3 before the v5/NOA flag; S1 hard-gates the next npm publish of the SDK; T-batch before the next time these suites are cited as upgrade evidence.

The underlying engineering in this release is largely disciplined — the verified-reads trust chain, the epoch-close lock gating, the batch-loop and silent-skip hygiene, and the deterministic-tally fixes all held up under adversarial review. The two blockers are both single-point gating omissions against a correctly-gated backdrop, which is exactly why they are fixable in days: the patterns to copy are already in the same files.

Appendix: completeness audit (files outside the 14 review scopes)

One flagged question is already resolved: bls_checkpoints: true in the v3 snapshot is behavior-preservingmainnet-v1.1.8 creates pending system checkpoints unconditionally (authority_per_epoch_store.rs:1083 at the tag); dev gates the identical logic on the flag with true in the base config, so v3 behavior matches 1.1.8 exactly.

Remaining checks the review scopes did not own:

  • Run/verify green v118_upgrade + cross_binary + malicious_cross_binary on the release SHA; per the test-testing playbook, confirm they'd actually fail on a v3 output divergence (not vacuous-pass).
  • Diff bls_checkpoints handling: what does 1.1.8 do at v3 (flag absent) vs 1.2.0 at v3 (flag true)? If any wire/signature format branches on it, this blocks the rolling restart.
  • Grep for readers of the new v3-snapshot presign-pool/delay params and confirm each is behind a v4 (or NOA/internal-presign) gate.
  • Audit all ZeroRng construction sites (3 crates) and confirm none is reachable from user-facing DKG/sign flows or the WASM exports.
  • Verify VersionedCentralizedDKGPublicOutput::V1 still parses a pre-existing mainnet user DKG output (fixture test) after the Secp256k1ECDSAProtocolSecp256k1DKGProtocol type switch.
  • Confirm no production build path (docker/ika-node, release CI) enables the test-testing feature.
  • Confirm workspace version gets bumped to 1.2.0 before tagging.
  • Run scripts/check-metric-names.sh result review: new metrics in dwallet_mpc_metrics.rs vs allowlist vs production-alerts playbook.
  • Sanity-check .cargo/config.toml [profile] default = "release" is honored by the pinned cargo (1.93) and doesn't change CI artifact profiles.

All other changed files fall inside the 14 dimension scopes; with the items above, coverage is complete.
Cross-cutting risks no single path-scoped reviewer owns (from the audit): the crypto-library swap (inkrypto @ abd7f010cryptography-private @ 32a27aa) changes the underlying MPC implementation for v3 paths too — the v118/cross_binary rehearsals are the only whole-tree byte-compat evidence, so they must run green on the exact release SHA with the live-mainnet key version; tokio 1.44→1.52 exact-pin bump; new ZeroRng (CryptoRng impl that yields zeros) must stay confined to the NOA sign-DKG emulation; dwallet-mpc-centralized-party's VersionedCentralizedDKGPublicOutput::V1 deserialization type switch needs a fixture test against a real mainnet user DKG output; and the simulator profile flipped debug-assertions/overflow-checks OFF.

🤖 Generated with Claude Code

omersadika and others added 2 commits July 2, 2026 09:23
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
# Conflicts:
#	.github/workflows/integration-tests-ci.yaml
#	.github/workflows/test-cluster.yaml
#	.github/workflows/ts-integration-tests.yaml
#	.github/workflows/upgrade-test.yaml
…doption deadlock (#1789)

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
omersadika and others added 5 commits July 2, 2026 15:02
…they claim (#1790)

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
#1791)

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…yle YAMLs for every role (#1792)

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…ead fork-residue metrics (#1793)

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
… dev (#1794)

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…tor_info fails to decode (#1795)

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
omersadika and others added 7 commits July 4, 2026 21:30
… reconfiguration path (G1) (#1798)

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
… (G2+G3) (#1797)

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…main (v4) reconfiguration path + one-time V1->V3 anchor migration (#1799)

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
#1796)

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
)

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…cryptography-private

Point the seven crypto crates (mpc, proof, class_groups, commitment,
twopc_mpc, group, homomorphic_encryption) at dwallet-labs/inkrypto @ 0572d6c
(latest main) rather than cryptography-private @ 32a27aa. inkrypto main mirrors
the latest cryptography-private file-by-file, so the full workspace builds
unchanged against it — no source changes required (dev already carries the
Blake2b256 HashScheme handling).

Repoint both workspace locks (root Cargo.lock and the excluded
sdk/ika-wasm/Cargo.lock) to inkrypto, and update the CI/Docker
url.<ssh>.insteadOf rewrites (setup-ssh action, release workflow, all three
Dockerfiles) plus the release-doc deploy-key note so private-repo cloning
targets inkrypto. NOTE: the CI GH_DEPLOY_KEY must be granted read access to
dwallet-labs/inkrypto for CI/Docker builds to fetch it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@omersadika

Copy link
Copy Markdown
Contributor Author

Closing — superseded. dev's content was landed on main as a clean, linear, merge-free history via #1803 (then main was flattened to drop the merge commit). main now matches dev's tree exactly. Merging this PR would re-introduce dev's full tangled history (15k commits + the old-Sui roots), which is what the flatten deliberately removed. The dev branch is left intact.

@omersadika omersadika closed this Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants