Skip to content

fix(drive): read stored group actions without the proof decoding budget - #5006

Merged
QuantumExplorer merged 1 commit into
v4.2-devfrom
claude/dreamy-pascal-3fbffa
Sep 26, 2026
Merged

QuantumExplorer merged 1 commit into
v4.2-devfrom
claude/dreamy-pascal-3fbffa

Conversation

@QuantumExplorer

@QuantumExplorer QuantumExplorer commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

Issue being fixed or feature implemented

#4629 (first released in v4.2.0-beta.1, in no v4.1.x) gave GroupAction #[platform_serialize(limit = 100000)] to bound decoding of group actions from proofs. Its comment reasoned that no valid stored action can exceed the limit, because a state transition is capped at the same 100,000 bytes. That does not hold, for two reasons:

  1. bincode's limit counts memory claimed, not bytes read. A map, a set or a vector of anything but bytes claims len * size_of::<T>() from its length prefix before it reads an element; strings and byte vectors claim their length. A token conventions localization entry claims 80 bytes and encodes in as few as 13.
  2. The derive applies the same limit to the trusted decode, so Drive's reads of the actions it stored itself hit it too.

A conventions change with 1,250 valid localizations encodes in 16,325 bytes but claims just over 100,000 at the map's length prefix. Encoding does not check the limit, so the proposal is accepted and the action is stored. Every read of it then fails:

  • a co-signer's transition loads the stored action in the token base transition transformer, and the decode error comes back as an internal error, so the action can never collect its second signature;
  • fetch_action_infos (the getGroupActions query) and verify_action_infos_in_contract (client proof verification) fail for any page that includes it.

v4.1.x binaries decode these actions without any limit, so they and the 4.2 binaries also treated such a co-signature differently.

What was done?

  • Drive reads its own stored actions without a budget, as v4.1.x did. fetch_active_action_info v0 (both functions) and fetch_action_infos v0 now call deserialize_from_bytes_trusted_no_limit. These are the only GroupAction decodes outside tests apart from the client proof path. Stored actions were validated when written and are bounded by the transition that proposed them (max_state_transition_size, 20,480 bytes).

  • The proof budget is sized from memory claims. The untrusted decode keeps a budget, raised from 100,000 to 262,144, and the attribute comment now gives the real reasoning. Per container reachable from a storable GroupActionEvent, on a 64-bit target, packed into 20,480 encoded bytes:

    Container Claimed per entry Fewest encoded bytes per entry Most entries in 20,480 bytes Claimed at the length prefix
    Conventions localizations, BTreeMap<String, TokenConfigurationLocalization> 80 13 (2-letter code, two 3-letter forms, 3 length prefixes, tag, flag) 1,569 125,520
    Pricing schedule SetPrices, BTreeMap<u64, u64> 16 2 for the first 251 keys, 4 after 5,227 83,632
    Distribution function Stepwise, BTreeMap<u64, u64> 16 2 for the first 251 keys, 4 after 5,225 83,600
    Notes and other strings 1 per byte 1 per byte 20,375 bytes 20,375
    Identifiers, [u8; 32] 32 32

    No event a group can store holds an identifier list or any other container. The fixed claims outside the container (the action's two ids and the like) come to under 100 bytes, so the worst storable action claims about 125,600 bytes, and 262,144 (256 KiB) leaves more than twice that.

  • Why the number is a literal and not in SystemLimits: the derive parses limit as an integer literal and passes it to with_limit::<N>(), a const generic, so it cannot come from a version table, and after this change no consensus path reads it. A test packs each container shape to the largest max_state_transition_size in PLATFORM_VERSIONS, so raising that limit without raising this budget fails the test.

  • New shared fixture get_token_conventions_with_localizations_fixture(n) in dpp's tests::fixtures.

  • Not done: rewording the limit error. The derive reports MaxEncodedBytesReachedError { max_size_kbytes: <limit>, size_hit: <input length> }, displayed as "Payload reached a 100000KB limit", which reads as a byte count. That display text also reaches SerializedObjectParsingError messages on the state transition decode path, so changing it is not local to this fix; it is left for a separate change.

Before / after

A token whose conventions change needs both members of a two-member group. Member 1 proposes conventions with 1,250 localizations ("en" plus two-letter codes, each with forms "abc" / "abc", all accepted by validate_localizations). The proposal is about 16.5 KB, it is accepted, and the stored action is 16,325 bytes.

Before:

member 2 co-signs        -> InternalError("storage: protocol: Payload reached a 100000KB limit")
getGroupActions (ACTIVE) -> fails for every page that includes the action
proof verification       -> MaxEncodedBytesReachedError { max_size_kbytes: 100000, size_hit: 16325 }

After:

member 2 co-signs        -> SuccessfulExecution; the token's conventions hold the 1,250 localizations
getGroupActions (ACTIVE) -> returns the action
proof verification       -> returns the action

In-place changes to shipped generations

Edited generation Protocol versions that select it Why consensus cannot change there
Drive::fetch_active_action_info_v0 and fetch_active_action_info_and_add_operations_v0 Every protocol version, 1 to 14 (DRIVE_GROUP_METHOD_VERSIONS_V1 is the only group method table) Same decoder on the same bytes; the only difference is that the budget check is gone, and that check can only turn a decoded value into an error. For every action the budget accepted, the output is identical. The actions it refused are decoded by every released binary for protocol versions 1 to 13 (v4.1.x), because the budget came in with #4629 without a version gate and first shipped in v4.2.0-beta.1. So the edit restores the released behaviour at every shipped protocol version, and protocol version 14 is unreleased. A test reads such an action at protocol version 13 and at the latest.
Drive::fetch_action_infos_v0 and fetch_action_infos_and_add_operations_v0 Same Same argument. This path serves the getGroupActions query; block execution does not call it.

The GroupAction budget itself is unversioned and, after this change, only client proof verification reads it.

Networks: testnet runs drive 4.1.0 at protocol version 13 (seeds 1 to 5 checked 2026-09-26), which has no budget here. The moutai devnet runs 4.2.0-beta.4 at protocol version 14. It holds 28 contracts and none defines a group, so it stores no group action and nothing there is waiting on this fix.

Breaking Changes

None, and no ! in the title. ! is for changes nodes could disagree on if rolled out unevenly. This change makes 4.2 decode stored group actions exactly as every released v4.1.x binary does at every protocol version they run, so a network mixing those binaries with this one agrees on every co-signature. The only binaries it differs from are the 4.2 betas, which run protocol version 14 only on devnets that are re-cut each release. The client budget only grows: any proof that verified before still verifies.

Other decode limits (checked, not changed)

  • Limits compared between v4.1.2 and v4.2-dev. Every platform_serialize(... limit ...) attribute is unchanged except the two fix(dpp): bound untrusted length prefixes on the proof-verification decode path #4629 added: GroupAction (this PR) and AssetLockValue (15,000). AssetLockValue is also decoded under its budget during block execution (fetch_asset_lock_outpoint_info). That budget holds, because its containers claim about what they encode: a byte vector for the script, and a vector of 32-byte tags capped by max_asset_lock_usage_attempts (16). The existing largest_valid_value_round_trips_under_limit test covers a 10,000-byte script with the maximum tags. The direct with_limit::<N>() calls added since v4.1.2 are all client side (proof verifier, SDK, wallet), and the contract decode limit (CONTRACT_DESERIALIZATION_LIMIT, 15,000) is unchanged and used only in tests.
  • Caution for whoever fixes the first-attribute derive bug. The derive reads only the first platform_serialize attribute, so the limit = 100000 on StateTransition, VotePoll and ContestedDocumentResourceVotePoll has never applied: each carries #[platform_serialize(unversioned)] first. That limit counts memory in the same way. The 16,465-byte co-signing transition in the new drive-abci test claims more than 100,000 bytes and is refused (LimitExceeded) when decoded under a 100,000 StateTransition budget. Applying that limit as written would refuse valid transitions well under 20 KiB, such as a contract create or a token config update with many localizations, so it needs sizing from memory claims first, as done here for GroupAction.

How Has This Been Tested?

  • dpp, group::group_action::deserialize_limit_tests:
    • should_decode_a_conventions_change_with_1250_localizations: valid localizations, 16,325 bytes, refused under the previous budget; decodes untrusted, trusted, and trusted without a limit.
    • should_decode_the_largest_storable_action_of_each_container_shape: localizations, SetPrices, Stepwise and a note, each packed to max_state_transition_size, decode with every decoder. Of these, only the localizations shape was refused under the previous budget.
    • should_refuse_a_localizations_length_prefix_beyond_the_budget: a map length prefix of 1,000,000 with no entries behind it is refused by the untrusted decode. The existing 8 GB note prefix test still passes.
  • drive: should_fetch_a_stored_conventions_change_with_1250_localizations (at protocol version 13 and at the latest), should_list_a_stored_conventions_change_with_1250_localizations, should_verify_a_proof_holding_a_conventions_change_with_1250_localizations.
  • drive-abci: should_let_a_group_co_sign_a_conventions_change_with_1250_localizations. Member 1 proposes and the stored bytes equal the proposed action's encoding; member 2's co-signature succeeds and the token's conventions are the 1,250 localizations.
  • With the fix reverted (budget back to 100,000, Drive back to the budgeted decode), every new decode test fails: dpp on the untrusted decode, drive with MaxEncodedBytesReachedError { max_size_kbytes: 100000, size_hit: 16325 }, and drive-abci with InternalError("storage: protocol: Payload reached a 100000KB limit") on the co-signature.
  • cargo test -p dpp --all-features --lib group:: (32 passed), cargo test -p drive --lib -- drive::group verify::group (111 passed), cargo test -p drive-abci --lib -- token_config_update_tests group_queries (98 passed).
  • cargo clippy -p dpp -p drive -p drive-abci --all-features --all-targets -- -D warnings and cargo fmt --all -- --check are clean.

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have added "!" to the title and described breaking changes in the corresponding section if my code contains any
  • I have made corresponding changes to the documentation if needed
  • If I added or changed GroveDB structure, I described it in the area's structure.rs, regenerated grovedb-structure.json, and checked the structure viewer link posted on this pull request

For repository code-owners and collaborators only

  • I have assigned this pull request to a milestone

🤖 Generated with Claude Code

PR Hygiene · 9cb905e

  • Bots — coderabbitai not yet · thepastaclaw not yet — /skip-bots proceeds without the ones not yet reported
  • Self-review — post /self-reviewed once the bots are done
  • Within your 5 open PRs
  • Build failed
  • Approvals — you own every area touched; none needed

When every box is checked the PR Hygiene check passes and this can merge.

GroupAction's `limit = 100000` (#4629) was sized as if bincode counted
bytes read. It counts memory claimed: a map claims len * size_of::<T>()
at its length prefix, and a conventions localization entry claims 80
bytes against as few as 13 encoded. The derive also applies the limit
to the trusted decode, so a valid stored conventions change with 1,250
localizations (16,325 bytes) could not be read back by Drive: its
co-signature ended in an internal error and the group actions query
and proof verification failed on any page holding it.

Drive now reads its own stored actions with
deserialize_from_bytes_trusted_no_limit, as every v4.1 binary does at
every protocol version it runs. The untrusted proof budget is sized from
the worst memory claim a 20,480-byte transition can produce (the
localizations map, about 125,600 bytes) and set to 262,144.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 25, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Next included review available in 17 seconds.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository: dashpay/platform/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: c03bc638-baf7-454e-a5a5-84db7f341e72

📥 Commits

Reviewing files that changed from the base of the PR and between 7829c84 and 9cb905e.

📒 Files selected for processing (7)
  • packages/rs-dpp/src/group/group_action/mod.rs
  • packages/rs-dpp/src/tests/fixtures/get_token_conventions_fixture.rs
  • packages/rs-dpp/src/tests/fixtures/mod.rs
  • packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/token/config_update/mod.rs
  • packages/rs-drive/src/drive/group/fetch/fetch_action_infos/v0/mod.rs
  • packages/rs-drive/src/drive/group/fetch/fetch_active_action_info/v0/mod.rs
  • packages/rs-drive/src/verify/group/verify_active_action_infos/v0/mod.rs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added this to the v4.2.0 milestone Sep 25, 2026
@github-actions github-actions Bot added the waiting-bots Waiting for the review bots to report on this head label Sep 25, 2026
@thepastaclaw

thepastaclaw commented Sep 25, 2026 •

Copy link
Copy Markdown
Collaborator

🔍 Review in progress — actively reviewing now (commit 9cb905e) · triage: normal

@QuantumExplorer QuantumExplorer left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed

@QuantumExplorer
QuantumExplorer merged commit cca6c0f into v4.2-dev Sep 26, 2026
32 of 34 checks passed
@QuantumExplorer
QuantumExplorer deleted the claude/dreamy-pascal-3fbffa branch September 26, 2026 00:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

waiting-bots Waiting for the review bots to report on this head

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants