Repository navigation
Conversation
Expose metalake maintenance action sample and finish endpoints with OpenAPI, wired through the in-job pipeline and recommenders. Co-authored-by: Cursor <cursoragent@cursor.com>
…basic Keep sample/finish DTOs, HK2 binding, and error mapping inside the TMS plugin, and publish the callback OpenAPI as its own spec. Co-authored-by: Cursor <cursoragent@cursor.com>
…cyMetaService Drop MetalakeNames after A1 removed it. Sample/finish checks the path metalake against the action policy. Co-authored-by: Cursor <cursoragent@cursor.com>
…e id Match OccupancyGate: look up metalake via TableMetaService, not the current policy. Policy deletion must not make a live action look like it belongs to no metalake. Co-authored-by: Cursor <cursoragent@cursor.com>
…ntenance-api Share TMS callback DTOs via :maintenance:maintenance-api so the plugin and built-in jobs can depend on one module without coupling through client-java. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
…ommon Wire DTOs are shared implementation types, not an SPI surface; align the module name with Gravitino's *-common convention. Co-authored-by: Cursor <cursoragent@cursor.com>
…napi.yaml Subdirectory specs must use ../openapi.yaml for shared components; ./openapi.yaml unresolved refs failed :docs:lintMaintenanceOpenAPI. Co-authored-by: Cursor <cursoragent@cursor.com>
Drop tableId/policyId from the sample request body; the server already has them on the in-flight action bound to jobId. Co-authored-by: Cursor <cursoragent@cursor.com>
…rt API Spark decides continue/stop locally; occupancy reclaims on job terminal state. The only Spark-facing callback persists snapshot metrics. Co-authored-by: Cursor <cursoragent@cursor.com>
…tables Drop UNIQUE(job_id), key metrics/before-snapshot by tableId, and finish all action rows for a job together so a single Spark run can cover a table batch. Co-authored-by: Cursor <cursoragent@cursor.com>
…able Expose metrics/before and metrics/after with optional finish to close the soft slot by (jobId, tableId, policyId). Drop job-terminal occupancy reclaim. Co-authored-by: Cursor <cursoragent@cursor.com>
…metrics
Jobs that do not sample can POST .../actions/{jobId}/finish with tableId
(and optional afterSnapshotId) to release occupancy per table/policy.
Co-authored-by: Cursor <cursoragent@cursor.com>
…ubmit bind Callbacks that arrive before bindJobId CAS the sole unbound claim for the table onto the path jobId; submit-side bind is idempotent for the same id. Also moves metrics finish to a query param and drops MaintenanceActionOperation. Co-authored-by: Cursor <cursoragent@cursor.com>
One Spark job can bind multiple soft slots on the same table; require policyId on callback DTOs and look up/bind/record by (jobId, tableId, policyId) so metrics and finish attribute to the correct action row. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
lasdf1234
force-pushed
the
feature/tms-a2-rest-sample-finish
branch
from
October 23, 2026 12:48
65b8672 to
ecf7487
Compare
Code Coverage Report
Files
|
lasdf1234
changed the title
This was referenced Oct 24, 2026
lasdf1234
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Second slice of the in-job track (A2). Stacked on A1 (#14054); targets
branch-tms-scheduler.metrics/before,metrics/after, andfinish(TableMaintenanceActionOperations)MaintenanceMetricsService, binder, and REST utils (aligned with idp-basic style):maintenance:maintenance-commondocs/open-api/maintenance/(jobId, tableId, policyId)bindJobIdTableMetaService.getMetalakeNameByTableIdWhy are the changes needed?
In-job Spark jobs need HTTP hooks to record before/after snapshot metrics and to release soft occupancy per table (with or without metrics). Wire DTOs belong in a thin shared
*-commonmodule so the TMS plugin and built-in jobs can share them without coupling throughclient-java.Fix: #14205
Part of #13760. Depends on #14054.
Does this PR introduce any user-facing change?
Yes — maintenance action metrics/finish REST (plugin OpenAPI):
POST /metalakes/{metalake}/maintenance/actions/{jobId}/metrics/beforePOST /metalakes/{metalake}/maintenance/actions/{jobId}/metrics/afterPOST /metalakes/{metalake}/maintenance/actions/{jobId}/finishHow was this patch tested?
./gradlew :maintenance:maintenance-common:compileJava :plugins:tms:compileJava :plugins:tms:compileTestJava -PskipITs./gradlew :plugins:tms:test -PskipITs -PskipDockerTests=true --tests org.apache.gravitino.maintenance.web.rest.TestTableMaintenanceActionOperations