Skip to content

[#14020] feat(tms): Built-in TMS Iceberg maintenance Spark job - #14122

Closed
lasdf1234 wants to merge 4 commits into
apache:branch-tms-schedulerfrom
lasdf1234:feature/tms-b8-tms-spark-job
Closed

lasdf1234 wants to merge 4 commits into
apache:branch-tms-schedulerfrom
lasdf1234:feature/tms-b8-tms-spark-job

Conversation

@lasdf1234

@lasdf1234 lasdf1234 commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

What changes were proposed in this pull request?

Standalone B8 slice on branch-tms-scheduler (depends on A2 #14121 for the sample/finish REST surface).

  • Built-in TmsIcebergMaintenanceJob (builtin-tms-iceberg-maintenance) with S0 sample/gate, Iceberg procedure dispatch (single type or commit-chain), and S1 finish
  • Spark callbacks go through MaintenanceActionClient / typed request-response DTOs in client-java (JSON-compatible with the plugin server DTOs; client-java does not depend on plugins/tms)
  • Register the template via BuiltInJobTemplateProvider

Why are the changes needed?

Cron and commit submit pipelines need a single built-in Spark entry that performs in-job S0 sample/recommend, runs Iceberg maintenance procedures, and closes soft slots via finish (S1).

Fix: #14020

Depends on #14121. Also related to #13760.

Does this PR introduce any user-facing change?

No new public REST in this PR. Sample/finish server API is in #14121.
Adds a built-in job template name for TMS Iceberg maintenance.

How was this patch tested?

./gradlew :clients:client-java:compileJava :maintenance:jobs:compileJava :maintenance:jobs:compileTestJava -PskipITs

./gradlew :maintenance:jobs:test -PskipITs -PskipDockerTests=true

lasdf1234 and others added 4 commits October 22, 2026 10:21
…e name

Shared constant for the jobs module and TMS plugin so the built-in
template name cannot drift.

Co-authored-by: Cursor <cursoragent@cursor.com>
Add builtin-tms-iceberg-maintenance template with S0 sample/gate, procedure
dispatch (single or commit-chain), and S1 finish.

Co-authored-by: Cursor <cursoragent@cursor.com>
…of common

Align with A2: server DTOs belong in plugins/tms. client-java cannot
depend on the plugin, so Spark callback request/response types live next
to MaintenanceActionClient and stay JSON-compatible with the plugin DTOs.

Co-authored-by: Cursor <cursoragent@cursor.com>
…e other builtins

Drop JobTemplateProvider.BUILTIN_TMS_ICEBERG_MAINTENANCE; use
BUILTIN_NAME_PREFIX + "tms-iceberg-maintenance" on TmsIcebergMaintenanceJob.

Co-authored-by: Cursor <cursoragent@cursor.com>
@lasdf1234 lasdf1234 self-assigned this Oct 22, 2026
@lasdf1234 lasdf1234 closed this Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant