feat(subgraph): track payment-active state for cheaper anon-retrieval queries#582
Draft
dennis-tra wants to merge 1 commit into
Draft
feat(subgraph): track payment-active state for cheaper anon-retrieval queries#582dennis-tra wants to merge 1 commit into
dennis-tra wants to merge 1 commit into
Conversation
… queries Add `DataSet.isPaymentActive: Boolean!` and maintain it from FWSS events plus a per-block handler, so anon-retrieval can filter expired PDP payments at the subgraph layer instead of pulling candidates and re-checking client-side. How it is maintained: - `PDPPaymentTerminated` with future endEpoch schedules the dataset into a `PendingPaymentTermination` bucket keyed by epoch; if endEpoch is already past, the flag flips immediately. - A new `handleBlock` attached to the FWSS data source loads the bucket for the current block (O(1) load returning null in the steady state) and flips matching datasets to `isPaymentActive: false`. - `ServiceTerminated` also flips the flag and unschedules from the bucket. - Re-extension (a new PDPPaymentTerminated after a prior epoch elapsed) sets the flag back to true and reschedules. Backwards compatible: `pdpPaymentEndEpoch` stays on the schema unchanged and is still populated. Existing clients comparing it to the current epoch keep working; new clients can use the cheaper `isPaymentActive` filter instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The subgraph part of #579
The backend part is here probe-lab#12 and depends on #487 to land first.
Why
In the pending retrieval anon implementation why draw candidate pieces by running multiple attempts and then hoping that we draw a pieces whose payment hasn't terminated yet. The reason for this is that we'd need to pass the current epoch to the query to filter the result set server side which isn't available. Secondly, we'll get served the current epoch together with the query result but cannot make use of that field during the querying process so we can only apply the filter condition after we got served the data.
What
In this PR we add a callback to the subgraph code that fires on every block (
handleBlock) which "cleans up" all datasets that have expired at that epoch by setting a newisPaymentActiveflag to false. The rest of the code is just bookkeeping of that flag as a response to various events.How it is maintained:
PDPPaymentTerminatedwith future endEpoch schedules the dataset into aPendingPaymentTerminationbucket keyed by epoch; if endEpoch is already past, the flag flips immediately.handleBlockattached to the FWSS data source loads the bucket for the current block (O(1) load returning null in the steady state) and flips matching datasets toisPaymentActive: false.ServiceTerminatedalso flips the flag and unschedules from the bucket.Backwards compatible:
pdpPaymentEndEpochstays on the schema unchanged and is still populated. Existing clients comparing it to the current epoch keep working; new clients can use the cheaperisPaymentActivefilter instead.Deployment
Deployed new version of this subgraph to:
calibration: https://api.goldsky.com/api/public/project_cmppqidloyxtu01tgc9b3e3w4/subgraphs/dealbot-calibration/0.1.0/gnmainnet: https://api.goldsky.com/api/public/project_cmppqidloyxtu01tgc9b3e3w4/subgraphs/dealbot-mainnet/0.1.0/gnI've also reset the version to
0.1.0because I've deployed them into a new Goldsky account which I'll plan to hand over to FilOZ.