Repository navigation
feat(coordinator): added BlobSubmitterV2 interface and implementation - #3673
Conversation
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
2819ca2 to
f0d25e8
Compare
7ffc48c to
4eb7c81
Compare
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
jonesho
changed the title
jonesho
changed the title
bde12ce to
1f361c5
Compare
39831fa to
0850d3c
Compare
|
Hi @jonesho what's this PR about? What's his motivation or issue/problem that is trying to solve? |
@fluentcrafter this was the PR to address the new blob submitter for RISC-V proving system, however the implementation is obsolete given that it was created before the "flexible blobbing proposal", will mark this as draft now, and resume once we have the L1 contract changes from the contract team, will ping them again. |
…ream Stub for building the [4-byte-len BE][lz4(truncated_conflation)] segment per conflation. LZ4 implementation deferred; stub unblocks the coordinator. Signed-off-by: Gaurav Ahuja <gauravahuja9@gmail.com>
…omain types StreamPosition captures the last sealed chunk's end block, DRH, and byte offset — used for cold-start recovery via BlobsRepositoryV2. The DRH fold function is stubbed pending keccak256 implementation. Signed-off-by: Gaurav Ahuja <gauravahuja9@gmail.com>
…RISC-V rollup Repurpose the blobs table for rollup proof records (no DB migration): - BlobRecordV2: parentShnarf→parentDataRollingHash, endShnarf→dataRollingHash, totalBatchesCount→totalConflationsCount; add proofHash and endOffset fields - BlobData: blobHash→chunkHash, compressedData→blobBytes, batchesCount→conflationsCount - BlobsInfo JSON: matching key renames + new endOffset field - BlobsPostgresDaoV2: write proofHash to blob_hash column; read it back in parseRecord - BlobsPostgresDaoG TODO: blob_hash repurposed, remove from drop list Signed-off-by: Gaurav Ahuja <gauravahuja9@gmail.com>
…ndler RollupProofPoller polls up to proofPollsPerTick (default 20) in-flight rollup proof requests per tick, modelled on ExecutionProofGeneratingCoordinator.action(). RollupProofHandler is a fun interface delivered with the poller; context carries all fields needed to assemble BlobRecordV2 when the proof response arrives. RollupProofPersistenceHandler implements RollupProofHandler and writes to BlobsRepositoryV2. Signed-off-by: Gaurav Ahuja <gauravahuja9@gmail.com>
…C-V rollup proofs Combined chunk assembly and proof submission coordinator. Ingests conflated batches, builds the continuous DA byte stream, seals BLOB_BYTES_LENGTH chunks, queries BatchesRepository to gate submission on all N conflations being proven, then submits a RollupProofRequestV1 and hands the proof index to RollupProofPoller. BlockEncoder and chunkHasher are injected to keep web3j and RLP off coordinator/core's classpath. The l2Executions hash field carries TODO pending the follow-on PR that adds proof_index_hash to the batches table. Signed-off-by: Gaurav Ahuja <gauravahuja9@gmail.com>
…ngCoordinator - Add findBatchesByBlockRange to BatchesDao/Repository/PostgresBatchesRepository/ RetryingBatchesPostgresDao to retrieve proofIndexHash per conflation at proof-submission time; add integration tests for the new query - Resolve TODO in buildRequest: fetch proofIndexHash from batches table and populate BlockIntervalProofIndex.hash for each l2Executions entry - Convert DataRollingHashCalculator and ConflationSegmentBuilder from objects to fun interfaces, wired as constructor parameters in RollupProofGeneratingCoordinator - Remove self-recursive trySubmitRollupProof call; the polling loop handles subsequent windows without the latency optimisation - Revert totalConflationsCount/conflationsCount field renames back to totalBatchesCount/batchesCount Signed-off-by: Gaurav Ahuja <gauravahuja9@gmail.com>
Cover the key behaviours of the coordinator: - out-of-order conflation rejection - no submission when fewer than N conflations accumulated - no submission when not all conflations are proven (null or partial) - proof request content when all N conflations are proven - correct context passed to RollupProofPoller after submission - window state reset after successful submission Signed-off-by: Gaurav Ahuja <gauravahuja9@gmail.com>
Link issue #3658 to track dropping blob_hash and expected_shnarf columns and renaming blob_compression_proof to blobs_info after riscv migration. Remove inline schema comments now tracked in the issue. Signed-off-by: Gaurav Ahuja <gauravahuja9@gmail.com>
…or BlobSubmitterV2 implementation and simplified BlobSubmittedEvent Signed-off-by: jonesho <jones.ho@consensys.net>
Signed-off-by: jonesho <jones.ho@consensys.net>
Signed-off-by: jonesho <jones.ho@consensys.net>
Signed-off-by: jonesho <jones.ho@consensys.net>
Signed-off-by: jonesho <jones.ho@consensys.net>
…b submitter V2 Signed-off-by: jonesho <jones.ho@consensys.net>
Signed-off-by: jonesho <jones.ho@consensys.net>
This reverts commit 3fe9a92. Signed-off-by: jonesho <jones.ho@consensys.net>
…nto coordinator/feat-blob-submitter-v2-for-risc-v
Signed-off-by: jonesho <jones.ho@consensys.net>
4997dc4 to
e3d027f
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit e3d027f. Configure here.
Signed-off-by: jonesho <jones.ho@consensys.net>
Signed-off-by: jonesho <jones.ho@consensys.net>
Signed-off-by: jonesho <jones.ho@consensys.net>
| // transactionHash = transactionHash.toByteArray(), // or decodeHex()? | ||
| // ) | ||
| // blobSubmittedEventConsumer.accept(blobSubmittedEvent) | ||
| // } |
There was a problem hiding this comment.
@gauravahuja same here, I don't have a concrete idea on the event structure to be dispatched as well as the consumer, I will leave this for you when integrating the submitter with conflationAppV2
There was a problem hiding this comment.
I you look at how the blobSubmittedEvent is used in pre riscv you will notice that is is mostly used for metric tracking. It also does not use all the fields from the BlobSubmittedEvent. So the data class BlobSubmittedEvent can be simplified and used here too.
The other thing is that Conflation App is not releated to blob or aggregation submission. The L1RelayingApp owns them. We will need to either reuse the existing BlobSubmission and AggregationFinalization coordinators or create new ones.
There was a problem hiding this comment.
@gauravahuja I revised and simplified the BlobSubmittedEvent to keep only the in-use fields.
For risc-v, one caveat is that BlobSubmittedEvent will be fired only when all the blobs in a BlobRecordV2 were submitted that might be done by multiple blob-carrying transactions (my understanding is that a BlobRecordV2 is 1 to 1 to a rollup proof, and the number of blobs involved in a rollup proof can be more than 6 (i.e. max # of blobs per transaction on L1)), and hence we might need multiple blob transactions for a BlobRecordV2.
Also note that, in risc-v, we no longer have concrete block range information per blob, all we have and need during blob submission is the resultant parentDataRollingHash and storedDataRollingHash of the submitting blobs, and that's why I skipped emitting BlobSubmittedEvent per blob transaction. Let me if that makes sense to you :)
There was a problem hiding this comment.
That makes sense, but then we need to decide how do we handle the case of more than 6 blobs per rollup and how do we filter out already submitted blobs. This will be important for BlobSubmissionCoordinatorV2
| // submissionTimestamp = clock.now(), | ||
| // transactionHash = transactionHash.toByteArray(), | ||
| // ) | ||
| // aggregationSubmittedEventConsumer.accept(aggregationSubmittedEvent) |
There was a problem hiding this comment.
@gauravahuja same here, I don't have a concrete idea on the event structure to be dispatched as well as the consumer, I will leave this for you when integrating the submitter with conflationAppV2
There was a problem hiding this comment.
Similar comment as for the BlobSubmissionEvent. We only use a few fields from FinalizationSubmittedEvent and others can be removed. This way you can reuse the same dataclass here.
There was a problem hiding this comment.
@gauravahuja Similarily, I revise and simplify the AggregationSubmittedEvent to keep only the in-use fields.
Signed-off-by: jonesho <jones.ho@consensys.net>
gauravahuja
left a comment
gauravahuja
left a comment
There was a problem hiding this comment.
The PR looks good to me but there are still open questions about the BlobSubmisisonCoordinator and AggregationFinalizationCoordinator. These two will be integrateing the BlobSubmitter and AggregationSubmitter.
3 participants
…or BlobSubmitterV2 implementation and simplified BlobSubmittedEvent
This PR implements issue(s) #
Checklist
PR.
Note
High Risk
Changes L1 submission/finalization encoding and domain events that drive coordinator progress tracking; mistakes could break blob posting, finalization, or metrics on production rollup/validium paths.
Overview
This PR aligns the coordinator and contract bindings with a data rolling hash (and DA offset) model on Lineth Rollup V9 and Validium V3, replacing the previous shnarf/blob-hash submission and finalization shapes.
L1 contract surface:
LinethRollupV9-rc.abiis regenerated sosubmitBlobstakesparentDataRollingHashandstoredDataRollingHash,finalizeBlocksusesFinalizationDataV5withparentDataRollingHash,endDataRollingHash, andstartOffset/endOffset, and init/events referencedataRollingHashProviderinstead ofshnarfProvider. Local deploy scripts usedataRollingHashProvider; a newdeployValidiumV3Stubtarget deploys a no-op Validium V3 stub for dev.Coordinator: Web3j builders and clients encode the new rollup and validium calls (
acceptDataRollingHash,finalizeBlocksV9on validium).submitBlobsV9gains anonlyEthCallpath for preflight-only blob submission.BlobSubmitterV2submitsBlobRecordV2chunks (rollup EIP-4844 and validium V2 implementations).BlobSubmittedEventandFinalizationSubmittedEventare slimmed to end-block number/timestamp plus submission time (no tx hash, shnarf, or nested proof in the event). Trackers andAggregationSubmitterfollow those fields.Reviewed by Cursor Bugbot for commit 78d5744. Bugbot is set up for automated code reviews on this repo. Configure here.