Skip to content

TS SDK: declare task dependencies by calling tasks - #73435

Merged
jason810496 merged 1 commit into
apache:mainfrom
jason810496:feature/ts-sdk/native-dag-authoring-syntax
Sep 23, 2026
Merged

jason810496 merged 1 commit into
apache:mainfrom
jason810496:feature/ts-sdk/native-dag-authoring-syntax

Conversation

@jason810496

@jason810496 jason810496 commented Sep 21, 2026 •

Copy link
Copy Markdown
Member

Why

ts-sdk/adr/0002-native-dag-interface.md decides that the call graph is the task graph: dag.task(...) hands back a factory, and calling it places the task and supplies its arguments. Before this, wiring travelled in an inputs option nothing read.

How

  • dag.task(taskId, handler) returns a TaskFactory. Calling it records the task's place and its inputs; each argument takes an upstream reference or a literal JSON value.
const extract = dag.task("extract", async (): Promise<number> => 42);
const transform = dag.task("transform", async (rows: number, region: string) => rows);
const load = dag.task("load", async (total: number) => {});

load(transform(extract(), "us"));
  • Arguments are passed in the order the handler declares them. A handler that declares a single object of named arguments can also be called with that object, which is the shape Python TaskFlow uses for load(transformed=transform(...)).
  • A reference passed by position is checked against the argument's own type, which is what tells the two call shapes apart when a handler declares a single argument.
  • A reference has to be the argument itself. One buried in an array or an object draws no edge, so it is rejected rather than recorded as data.
  • Every task has to be called exactly once, so none is silently left out of the graph. A TaskRef exists only once its producing call has returned, which is what makes a cycle through arguments unwritable.
  • A TaskRef carries its handler's return type, so a construct that needs a particular one can ask for it.
  • Bundle keeps the Dags declared in TypeScript and the task handlers for Python-declared Dags in separate maps, since one dag_id belongs to one of them and never both.

Was generative AI tooling used to co-author this PR?

@pierrejeambrun pierrejeambrun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGMT.

Just one nit.

Comment thread ts-sdk/src/sdk/bundle.ts Outdated
@jason810496
jason810496 force-pushed the feature/ts-sdk/native-dag-authoring-syntax branch from 1c0a934 to 508da6f Compare September 22, 2026 13:10
@jason810496
jason810496 merged commit b6538c3 into apache:main Sep 23, 2026
89 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants