Repository navigation
Parse native TypeScript Dags with the Node coordinator - #73844
Merged
jason810496 merged 3 commits intoOct 1, 2026
Merged
jason810496 merged 3 commits into
jason810496 merged 3 commits into
Conversation
jason810496
added this pull request to stack #73848
September 28, 2026 13:20
This was referenced Sep 28, 2026
This was referenced Sep 28, 2026
jason810496
force-pushed
the
jason/lang-sdk-e2e/04-java-dag-importer
branch
from
September 28, 2026 15:57
4619343 to
a80facc
Compare
jason810496
force-pushed
the
jason/lang-sdk-e2e/05-node-dag-importer
branch
from
September 28, 2026 15:57
b441444 to
3ccfffc
Compare
jason810496
force-pushed
the
jason/lang-sdk-e2e/04-java-dag-importer
branch
from
September 29, 2026 01:25
a80facc to
d62246f
Compare
jason810496
force-pushed
the
jason/lang-sdk-e2e/05-node-dag-importer
branch
from
September 29, 2026 01:26
3ccfffc to
a9d2d19
Compare
jason810496
force-pushed
the
jason/lang-sdk-e2e/04-java-dag-importer
branch
from
September 29, 2026 01:29
d62246f to
f8acd1d
Compare
jason810496
force-pushed
the
jason/lang-sdk-e2e/05-node-dag-importer
branch
from
September 29, 2026 01:29
a9d2d19 to
1705d22
Compare
1 task done
jason810496
removed this pull request from stack #73848
October 1, 2026 03:02
jason810496
force-pushed
the
jason/lang-sdk-e2e/04-java-dag-importer
branch
from
October 1, 2026 03:03
f8acd1d to
ceb6b6a
Compare
jason810496
force-pushed
the
jason/lang-sdk-e2e/05-node-dag-importer
branch
from
October 1, 2026 03:03
1705d22 to
7443b46
Compare
1 task done
jason810496
added this pull request to stack #74005
October 1, 2026 03:04
jason810496
force-pushed
the
jason/lang-sdk-e2e/04-java-dag-importer
branch
from
October 1, 2026 06:20
ceb6b6a to
ca832b6
Compare
jason810496
force-pushed
the
jason/lang-sdk-e2e/05-node-dag-importer
branch
from
October 1, 2026 06:20
7443b46 to
a8c1db7
Compare
jason810496
force-pushed
the
jason/lang-sdk-e2e/04-java-dag-importer
branch
from
October 1, 2026 12:41
ca832b6 to
5e8a705
Compare
jason810496
force-pushed
the
jason/lang-sdk-e2e/05-node-dag-importer
branch
from
October 1, 2026 12:41
a8c1db7 to
8e95f87
Compare
A NodeCoordinator configured without bundles_root now hands out a Dag importer, so the Dag bundles it serves have their *.min.mjs bundles parsed by running them with node. Only files that start with the airflow-ts-pack layout header are parsed. The Dag source shown for such a Dag is the bundle's entry module, as the bundle embeds it, or a short notice when there is none to show.
Describe the coordinator configuration that parses native Dags, what the Dag processor needs for it, what the Code view shows, the cost of handler-only bundles, and that cluster policies and some CLI commands do not apply to a native Dag.
jason810496
removed this pull request from stack #74005
October 1, 2026 13:18
jason810496
force-pushed
the
jason/lang-sdk-e2e/05-node-dag-importer
branch
from
October 1, 2026 13:18
8e95f87 to
fe68315
Compare
jason810496
force-pushed
the
jason/lang-sdk-e2e/04-java-dag-importer
branch
from
October 1, 2026 13:18
5e8a705 to
8ffd19e
Compare
Base automatically changed from
jason/lang-sdk-e2e/04-java-dag-importer
to
jason/lang-sdk-e2e/02-importer-registry
October 1, 2026 13:18
This was referenced Oct 1, 2026
Member
Author
|
Replaced by #74037. GitHub marked this PR as merged into a stack branch when the stack was reordered; nothing was merged into |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack (bottom to top): #73841, #74004, #73842, #73843, #73844, #73845, #73846, #73847
Part of the native Dag e2e stack, related to #71929. Builds on the Java importer layer (new commits only). Merge after #73442 and #73445, which make the TypeScript runtime answer the Dag-parsing request and add
triggerDagRun.Why
The earlier layers let a coordinator parse native Dags with its own runtime, and
JavaCoordinatoropts in. This makesNodeCoordinatoropt in, so Airflow parses the Dags declared in packed*.min.mjsTypeScript bundles.What changes
A
NodeCoordinatorwithoutbundles_rootnow parses the TypeScript bundles in the Dag bundles it serves, and the same entry runs their tasks:.min.mjsand start with the headerairflow-ts-packwrites are parsed, andsafe_modedoes not change this check. A file that cannot be read is kept, so its parse reports the error.node <bundle>with the supervisor schema version from the bundle metadata. A bundle that fails its integrity check gets an import error.bundles_rootstill parses nothing, as before.typescript.rstdocuments the setup, that the Dag processor needs Node.js, that a Dag with a cycle is an import error, and the Code view.Decision left open by #71929
#71929 does not settle what
get_source_codereturns for a native Dag. Here the Code view shows the bundle's entry module for each of its Dags, since a Dag's source is stored per file. A short notice replaces a source that cannot be read.Limitations
TaskHandlerobjects is parsed too when it sits in a served Dag bundle, because its metadata does not say whether it declares Dags. Each parse launchesnodeand finds no Dags, so the docs suggest keeping such bundles underbundles_root.dag_policyandtask_policydo not run on a native Dag.airflow dags test,tasks test,tasks renderandtasks listrefuse it, andairflow dags reserializedoes not store the Dags of*.min.mjsbundles. The docs say so.typescript.rststill describe only stub Dags. This stack leaves them, since Update the TypeScript SDK docs for Dags declared in TypeScript #73875 rewrites that page.How to test
The native e2e layer at the top of the stack, above the SDK layers, parses a native TypeScript Dag end to end and checks its source and a triggered run.
Was generative AI tooling used to co-author this PR?