Skip to content

Commit 4166797

Browse files
committed
TS SDK: add task groups to the native Dag surface
1 parent 909171b commit 4166797

5 files changed

Lines changed: 517 additions & 39 deletions

File tree

‎airflow-core/docs/authoring-and-scheduling/language-sdks/typescript.rst‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -323,6 +323,27 @@ already exists changes nothing. Each returns the reference it was called on, so
323323
Pass a value as an argument when the downstream task needs it, and use ``before`` or ``after`` when
324324
it only needs to run in order.
325325

326+
Task groups
327+
~~~~~~~~~~~
328+
329+
``dag.taskGroup(groupId)`` opens a scope with the same ``task`` and ``taskGroup`` methods as the Dag,
330+
prefixing the id of everything declared in it, as Python's ``prefix_group_id`` does:
331+
332+
.. code-block:: typescript
333+
334+
const staging = dag.taskGroup("staging");
335+
staging.task("stage_rows", stageRows)(); // task id "staging.stage_rows"
336+
staging.taskGroup("checks").task("nulls", checkNulls)(); // "staging.checks.nulls"
337+
338+
staging.before(loaded); // staging >> loaded
339+
340+
A group is an edge endpoint in its own right, so ``before`` and ``after`` order a whole group against
341+
a task or against another group.
342+
343+
Tasks and groups share one id namespace, as they do in Python, so a Dag cannot hold both a task and a
344+
group called ``staging``. A ``.`` is what separates a group from what it holds, so it cannot appear in
345+
an id of either.
346+
326347
``new Dag`` and ``dag.task`` both take a trailing spec of Airflow options:
327348
``{ schedule: "@daily", tags: ["etl"] }`` for the Dag, ``{ retries: 2, retryDelay: 30 }`` for a task.
328349

‎ts-sdk/src/index.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,9 @@ export type { ArgNameMap } from "./sdk/arg-names.js";
3030
export type { Registerable } from "./sdk/bundle.js";
3131
export type {
3232
DagSpec,
33+
Node,
3334
TaskFactory,
35+
TaskGroupRef,
3436
TaskInput,
3537
TaskInputs,
3638
TaskOptions,

0 commit comments

Comments
 (0)