TS SDK: generate Dag and task options from the serialization schema

A Dag authored in TypeScript has to describe itself with the same options
Airflow's scheduler reads, and that vocabulary lives in airflow-core's Dag
serialization schema. Hand-listing those options in the SDK would leave two
copies to keep aligned by hand, and the copy the SDK ships would quietly rot
every time core adds, renames, or drops a key.

Deriving them instead makes the drift visible: a new scalar key shows up in the
regenerated diff or fails generation until someone says why the SDK omits it,
and a curated entry that outlives the key it names fails the same way. The
serialization version is the one thing the schema cannot supply, so a core bump
now fails a test in airflow-core rather than producing Dags stamped with a
version core no longer writes.

This follows the policy the Go SDK's TaskSpec generator set and ADR-0008
records, so the three language SDKs expose the same Dag surface for the same
documented reasons.

Author: jason810496

.pre-commit-config.yaml

@@ -308,6 +308,16 @@ repos:
         additional_dependencies: ['PyYAML>=6.0', 'rich>=13.6.0']
         pass_filenames: false
         require_serial: true
+      - id: sync-ts-sdk-dag-schema
+        name: Sync TypeScript SDK Dag serialization schema with airflow-core
+        description: "Copy airflow-core's serialization schema when TS SDK's vendored dag-schema.json drifts"
+        entry: ./ts-sdk/scripts/ci/prek/sync_dag_schema.py
+        language: python
+        pass_filenames: false
+        files: >
+          (?x)
+          ^airflow-core/src/airflow/serialization/schema\.json$|
+          ^ts-sdk/schema/dag-schema\.json$
       - id: check-go-version-in-sync
         name: Check Go toolchain version is consistent across build files
         entry: ./scripts/ci/prek/check_go_version_in_sync.py

airflow-core/docs/authoring-and-scheduling/language-sdks/typescript.rst

@@ -243,9 +243,9 @@ task instance.
 Declaring a Dag in TypeScript
 -----------------------------
 
-A ``Dag`` is declared on this side rather than in Python: its tasks and the edges between them are
-written in TypeScript. The surface is still growing, so a Dag declared this way is not served to
-Airflow yet.
+A ``Dag`` is declared on this side rather than in Python: its schedule, its tasks, their options and
+the edges between them are all written in TypeScript. The surface is still growing, so a Dag declared
+this way is not served to Airflow yet.
 
 ``dag.task(taskId, handler)`` returns a *factory*. Calling it places the task in the Dag and supplies the
 handler's arguments, so the call graph is the task graph:
@@ -277,7 +277,11 @@ argument itself: one buried inside an array or an object is a literal, and draws
 Every task has to be called exactly once. An uncalled task fails when the Dag is read, so none can be
 left out of the graph by accident.
 
-``new Dag`` and ``dag.task`` also take a trailing ``spec`` object that is not used yet; do not set it.
+``new Dag`` and ``dag.task`` both take a trailing spec carrying the options Python would otherwise own:
+``{ schedule: "@daily", tags: ["etl"] }`` for the Dag, ``{ retries: 2, retryDelay: 30 }`` for a task.
+Both are generated from Airflow's own Dag serialization schema, so the field names are the schema's keys
+in camelCase, every field is optional, and a misspelled one is a compile error. The specs are recorded
+but not serialized yet.
 
 Writing tasks
 -------------

airflow-core/tests/unit/serialization/test_ts_sdk_serialization_version.py

@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from __future__ import annotations
+
+import re
+
+import pytest
+
+from airflow.serialization.serialized_objects import DagSerialization
+
+from tests_common.pytest_plugin import AIRFLOW_ROOT_PATH
+
+GENERATED_FIELDS_PATH = AIRFLOW_ROOT_PATH / "ts-sdk" / "src" / "generated" / "dag-schema-fields.ts"
+
+SERIALIZATION_VERSION_PATTERN = re.compile(
+    r"^export const SERIALIZATION_VERSION = (?P<version>\d+);$", re.MULTILINE
+)
+
+
+@pytest.mark.skipif(
+    not GENERATED_FIELDS_PATH.is_file(),
+    reason="TypeScript SDK sources are absent (source distribution build)",
+)
+def test_ts_sdk_pins_the_current_serialization_version():
+    """
+    The schema constrains ``__version`` only to a positive integer, so the TypeScript SDK
+    pins it by hand. Without this check, bumping the serializer would leave the SDK emitting
+    Dags stamped with a version core no longer writes.
+    """
+    match = SERIALIZATION_VERSION_PATTERN.search(GENERATED_FIELDS_PATH.read_text(encoding="utf-8"))
+    assert match is not None, f"No SERIALIZATION_VERSION export found in {GENERATED_FIELDS_PATH}"
+
+    assert int(match.group("version")) == DagSerialization.SERIALIZER_VERSION, (
+        f"{GENERATED_FIELDS_PATH.relative_to(AIRFLOW_ROOT_PATH)} pins serialization version "
+        f"{match.group('version')}, but DagSerialization.SERIALIZER_VERSION is "
+        f"{DagSerialization.SERIALIZER_VERSION}. Update SERIALIZATION_VERSION in "
+        "ts-sdk/scripts/generate-dag-schema.mjs, re-run `pnpm run generate:dag-schema`, and make "
+        "sure the SDK still emits Dags core can read."
+    )

ts-sdk/.pre-commit-config.yaml

@@ -49,6 +49,18 @@ repos:
         additional_dependencies: ['pnpm@10.28.1']
         pass_filenames: false
         require_serial: true
+      - id: check-ts-sdk-dag-schema
+        name: Check TypeScript SDK Dag schema fields are up to date
+        entry: ./scripts/ci/prek/check_dag_schema.py
+        language: node
+        files: >
+          (?x)
+          ^src/generated/dag-schema-fields\.ts$|
+          ^schema/dag-schema\.json$|
+          ^scripts/generate-dag-schema\.mjs$
+        additional_dependencies: ['pnpm@10.28.1']
+        pass_filenames: false
+        require_serial: true
       - id: check-ts-sdk-docs-package-version-in-sync
         name: Check ts-sdk/docs package.json versions match ts-sdk/package.json
         entry: ../scripts/ci/prek/check_ts_sdk_docs_package_version_in_sync.py

ts-sdk/package.json

@@ -51,7 +51,8 @@
     "build": "pnpm run clean && tsc -p tsconfig.build.json",
     "verify:package": "node scripts/verify-package.mjs",
     "prepack": "pnpm run build",
-    "generate:supervisor": "node scripts/generate-supervisor.mjs"
+    "generate:supervisor": "node scripts/generate-supervisor.mjs",
+    "generate:dag-schema": "node scripts/generate-dag-schema.mjs"
   },
   "keywords": [
     "airflow",