Skip to content

TS SDK: add task groups to the native Dag surface - #73439

Merged
pierrejeambrun merged 1 commit into
apache:mainfrom
jason810496:feature/ts-sdk/task-groups
Sep 30, 2026
Merged

pierrejeambrun merged 1 commit into
apache:mainfrom
jason810496:feature/ts-sdk/task-groups

Conversation

@jason810496

@jason810496 jason810496 commented Sep 21, 2026 •

Copy link
Copy Markdown
Member

TS SDK: add task groups to the native Dag surface

Why

ts-sdk/adr/0002 decision 5 keeps Python's grouping semantics: a scope offering the same task and taskGroup methods as the Dag, prefixing each task id, and standing at either end of an edge as group1 >> group2 does.

How

  • dag.taskGroup(groupId, options?) opens that scope and prefixes the id of everything declared in it, as prefix_group_id does. { prefixGroupId: false } keeps the ids as written.
  • A group id is checked as Python's validate_group_key does: at most 200 characters, letters, digits, dashes and underscores only.
  • Node is the base type both a task and a group satisfy, so before and after take either. Python has the same in DAGNode; the Go SDK calls it airflow.Node.
  • Tasks and groups share one id namespace, as they do in Python. The separator cannot appear in an id of either kind: a dotted task id used to be accepted verbatim, giving a task whose id carries a prefix while the Dag holds no such group.
const staging = dag.taskGroup("staging");
staging.task("stage_rows", stageRows)(); // task id "staging.stage_rows"

const checks = dag.taskGroup("checks", { prefixGroupId: false });
checks.task("nulls", checkNulls)(); // task id "nulls"

What

  • TaskGroupRef, TaskGroupOptions, Node, and TaskGroupRecord as what the Dag records.
  • An edge endpoint is checked by identity for a group as it already was for a task; matching on the id alone accepted a group from another Dag object and retargeted the edge locally.

Was generative AI tooling used to co-author this PR?

@pierrejeambrun pierrejeambrun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two small nits, overall looking good.

Comment thread ts-sdk/src/sdk/dag.ts Outdated
Comment thread ts-sdk/src/sdk/dag.ts Outdated
@jason810496
jason810496 marked this pull request as draft September 30, 2026 01:40
@jason810496
jason810496 force-pushed the feature/ts-sdk/task-groups branch from 45ea18a to 87f2734 Compare September 30, 2026 02:27
@jason810496
jason810496 force-pushed the feature/ts-sdk/task-groups branch from 87f2734 to 159e4da Compare September 30, 2026 02:31
@jason810496
jason810496 marked this pull request as ready for review September 30, 2026 02:38

@jason810496 jason810496 left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review.

Comment thread ts-sdk/src/sdk/dag.ts Outdated
Comment thread ts-sdk/src/sdk/dag.ts Outdated
@jason810496
jason810496 force-pushed the feature/ts-sdk/task-groups branch from 159e4da to 0490d68 Compare September 30, 2026 05:12

@pierrejeambrun pierrejeambrun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks

@pierrejeambrun
pierrejeambrun merged commit 53ca2a4 into apache:main Sep 30, 2026
90 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants