Go SDK: run TriggerDagRun tasks in the Go runtime by jason810496 · Pull Request #74417 · apache/airflow

jason810496 · 2026-10-07T16:13:29Z

related: Go SDK: embed each native Dag's source file in the bundle #74342, Cover native Go Dags in the go_sdk e2e test #74411 (e2e), TS SDK: support logicalDate and runAfter in triggerDagRun #74414 (the same options for the TypeScript SDK)

Why

A Go airflow.TriggerDagRun task was meant to run as TriggerDagRunOperator on a Python worker, but a Python worker refuses a task whose Dag file is a Go bundle, so the task could never run. As ADR-0008 decides, each Lang SDK runtime runs the trigger task itself, as the TypeScript SDK already does.

How

The Go runtime finds a trigger task of a registered Dag and runs it the way TriggerDagRunOperator does:
- checks whether the target Dag is paused, when asked;
- pushes the run link, triggers the run, and pushes trigger_run_id;
- skips or fails without retry when the run already exists;
- waits for the run by polling, or defers to the standard provider's DagStateTrigger in the Python triggerer when Deferrable is set, then resumes in the Go runtime.
A trigger task takes DagSpec.Queue unless its TaskSpec sets one, so it is routed to the Go runtime.
Like the TypeScript SDK, the arguments are sent as written, with no Jinja rendering.

dag := airflow.Dag("go_native_trigger", airflow.DagSpec{Queue: "golang"})
dag.Task(airflow.TriggerDagRun(airflow.TriggerDagRunSpec{
	DagID:             "report",
	LogicalDate:       time.Date(2026, 10, 1, 0, 0, 0, 0, time.UTC), // was a string; zero means now
	Conf:              map[string]any{"triggered_by": "go_native_trigger"},
	WaitForCompletion: true,
	Deferrable:        &deferrable,
}), airflow.TaskSpec{TaskID: "trigger_report"})

What

pkg/execution: the trigger runner, and the TriggerDagRun, GetDagRunState and GetDag calls it makes.
TriggerDagRunSpec.LogicalDate is a time.Time; the shared conformance case and go_native.json follow.
The trigger task takes the Dag's queue; serialize.go and the generated spec docs follow.
Docs on TriggerDagRun: who runs it, its queue, Deferrable, and no templating.

Was generative AI tooling used to co-author this PR?

Yes, with help of Claude Code Opus 5.5 following the guidelines

A native Java Dag could not start another Dag's run. Airflow's TriggerDagRunOperator is Python, and a Java bundle has no Python Dag file to declare one in, so a Java Dag that fans out to another Dag needed a Python counterpart for that one task. A `TriggerDagRun` passed to `dag.task` declares a task that has no Java body to run; the Java runtime runs it as `TriggerDagRunOperator` does. Its settings are named as Python names the operator's parameters: ```java dag.task("trigger_downstream", new TriggerDagRun("downstream_etl") .config("wait_for_completion", true) .config("conf", Map.of("rows", 2))); ``` An annotated Dag declares one from an ordinary `@Builder.Task` method that returns a `TriggerDagRun`. The serialized Dag has to carry what is triggered, so that method runs when the Dag is built rather than when the task runs, and therefore takes no parameters. The task serializes as Python's own `TriggerDagRunOperator`: its template fields, its UI colour and extra link, and a `dag_dependencies` entry, so the Airflow UI draws the dependency between the two Dags. Python writes only an operator's template fields and keeps the rest in the Dag file; a Java Dag has no such file, so every setting the author made is written too. The runtime records the triggered run's link and ID as XComs, skips or fails on a run that already exists, and waits for completion either by polling or, with `deferrable`, by deferring to the Python triggerer's DagStateTrigger and resuming from the event it fires. That makes this the first Java runtime to emit DeferTask and a skipped TaskState. `scripts/ci/lang_sdk_serialization/test_dags.yaml` now gives the shared trigger Dag a real logical date instead of a Jinja template, because the SDK takes one as a date-time. The Go SDK is making the same change in #74417; whichever lands second drops that one line, and until then the Go serialization conformance check reports that single difference.

A native Java Dag could not start another Dag's run. Airflow's TriggerDagRunOperator is Python, and a Java bundle has no Python Dag file to declare one in, so a Java Dag that fans out to another Dag needed a Python counterpart for that one task. A `TriggerDagRun` passed to `dag.task` declares a task that has no Java body to run; the Java runtime runs it as `TriggerDagRunOperator` does. Its settings are named as Python names the operator's parameters: ```java dag.task("trigger_downstream", new TriggerDagRun("downstream_etl") .config("wait_for_completion", true) .config("conf", Map.of("rows", 2))); ``` An annotated Dag declares one from an ordinary `@Builder.Task` method that returns a `TriggerDagRun`. The serialized Dag has to carry what is triggered, so that method runs when the Dag is built rather than when the task runs, and therefore takes no parameters. The task serializes as Python's own `TriggerDagRunOperator`: its template fields, its UI colour and extra link, and a `dag_dependencies` entry, so the Airflow UI draws the dependency between the two Dags. Python writes only an operator's template fields and keeps the rest in the Dag file; a Java Dag has no such file, so every setting the author made is written too. The runtime records the triggered run's link and ID as XComs, skips or fails on a run that already exists, and waits for completion either by polling or, with `deferrable`, by deferring to the Python triggerer's DagStateTrigger and resuming from the event it fires. That makes this the first Java runtime to emit DeferTask and a skipped TaskState. `scripts/ci/lang_sdk_serialization/test_dags.yaml` now gives the shared trigger Dag a real logical date instead of a Jinja template, because the SDK takes one as a date-time. The Go SDK is making the same change in #74417; whichever lands second drops that one line, and until then the Go serialization conformance check reports that single difference. (cherry picked from commit af569b2)

A native Java Dag could not start another Dag's run. Airflow's TriggerDagRunOperator is Python, and a Java bundle has no Python Dag file to declare one in, so a Java Dag that fans out to another Dag needed a Python counterpart for that one task. A `TriggerDagRun` passed to `dag.task` declares a task that has no Java body to run; the Java runtime runs it as `TriggerDagRunOperator` does. Its settings are named as Python names the operator's parameters: ```java dag.task("trigger_downstream", new TriggerDagRun("downstream_etl") .config("wait_for_completion", true) .config("conf", Map.of("rows", 2))); ``` An annotated Dag declares one from an ordinary `@Builder.Task` method that returns a `TriggerDagRun`. The serialized Dag has to carry what is triggered, so that method runs when the Dag is built rather than when the task runs, and therefore takes no parameters. The task serializes as Python's own `TriggerDagRunOperator`: its template fields, its UI colour and extra link, and a `dag_dependencies` entry, so the Airflow UI draws the dependency between the two Dags. Python writes only an operator's template fields and keeps the rest in the Dag file; a Java Dag has no such file, so every setting the author made is written too. The runtime records the triggered run's link and ID as XComs, skips or fails on a run that already exists, and waits for completion either by polling or, with `deferrable`, by deferring to the Python triggerer's DagStateTrigger and resuming from the event it fires. That makes this the first Java runtime to emit DeferTask and a skipped TaskState. `scripts/ci/lang_sdk_serialization/test_dags.yaml` now gives the shared trigger Dag a real logical date instead of a Jinja template, because the SDK takes one as a date-time. The Go SDK is making the same change in #74417; whichever lands second drops that one line, and until then the Go serialization conformance check reports that single difference.

FrankYang0529

Overall LGTM. Should we also update skipped and deferred in go-sdk/capabilities.yaml?

* Java SDK: Trigger another Dag's run from a native Java Dag A native Java Dag could not start another Dag's run. Airflow's TriggerDagRunOperator is Python, and a Java bundle has no Python Dag file to declare one in, so a Java Dag that fans out to another Dag needed a Python counterpart for that one task. A `TriggerDagRun` passed to `dag.task` declares a task that has no Java body to run; the Java runtime runs it as `TriggerDagRunOperator` does. Its settings are named as Python names the operator's parameters: ```java dag.task("trigger_downstream", new TriggerDagRun("downstream_etl") .config("wait_for_completion", true) .config("conf", Map.of("rows", 2))); ``` An annotated Dag declares one from an ordinary `@Builder.Task` method that returns a `TriggerDagRun`. The serialized Dag has to carry what is triggered, so that method runs when the Dag is built rather than when the task runs, and therefore takes no parameters. The task serializes as Python's own `TriggerDagRunOperator`: its template fields, its UI colour and extra link, and a `dag_dependencies` entry, so the Airflow UI draws the dependency between the two Dags. Python writes only an operator's template fields and keeps the rest in the Dag file; a Java Dag has no such file, so every setting the author made is written too. The runtime records the triggered run's link and ID as XComs, skips or fails on a run that already exists, and waits for completion either by polling or, with `deferrable`, by deferring to the Python triggerer's DagStateTrigger and resuming from the event it fires. That makes this the first Java runtime to emit DeferTask and a skipped TaskState. `scripts/ci/lang_sdk_serialization/test_dags.yaml` now gives the shared trigger Dag a real logical date instead of a Jinja template, because the SDK takes one as a date-time. The Go SDK is making the same change in #74417; whichever lands second drops that one line, and until then the Go serialization conformance check reports that single difference. * Java SDK: Point both example triggers at a Dag that triggers nothing * Java SDK: Say why a switch cannot choose a task that triggers a Dag run * Java SDK: Snapshot a TriggerDagRun when its task is declared * Java SDK: Check a TriggerDagRun setting the way every other config key is checked * Java SDK: Describe what a trigger task pushes and how it names a run * Java SDK: Correct which runtimes already run trigger tasks * Java SDK: Format a trigger task's date-times in one place * Java SDK: Reject a trigger method that throws a checked exception * Java SDK: Give the trigger fixture a logical date every SDK can write * Java SDK: Run a trigger task through runTask in tests * Java SDK: Check a trigger task polls the run it started

A TriggerDagRun task triggers a Dag run, reads its state and checks whether the Dag is paused. The runtime has no client for these three requests yet. An existing run_id comes back as a result instead of an error, because the task decides whether that skips or fails it.

The Go runtime runs a TriggerDagRun task itself and renders no templates, so a string cannot hold a date it must compute. A time.Time lets the runtime send the date as it is, and the zero value keeps the Python default of now. The conformance case and the Go fixture move to a literal datetime.

A task from airflow.TriggerDagRun had no Task in the bundle, so a Go Dag needed a Python worker to run it. The bundle now returns a TriggerTask, and RunTask runs it the way TriggerDagRunOperator does: the same run_id and date rules, the link and trigger_run_id XComs, polling, and a deferral to DagStateTrigger that resumes in the same runtime. It reads its config from the environment and renders no templates.

The Go runtime runs a TriggerDagRun task now, so it belongs on the coordinator that the Dag queue maps to. Before, the task kept no queue and went to a Python worker. A queue in the TaskSpec still wins, as for any other task, and a Dag with no queue leaves the task without one.

Say how the task names its run, what Deferrable needs, and that the arguments are not templated and OpenLineage parent info is not added.

Mark the skipped and deferred states supported, since a TriggerDagRun task now emits both. Co-Authored-By: Claude <noreply@anthropic.com>

boring-cyborg Bot added area:DAG-processing area:dev-tools area:go-sdk kind:documentation labels Oct 7, 2026

jason810496 mentioned this pull request Oct 7, 2026

Cover native Go Dags in the go_sdk e2e test #74411

Merged

1 task done

jason810496 force-pushed the feature/go-sdk/trigger-dag-run-runtime branch 2 times, most recently from 25343ee to d73904c Compare October 8, 2026 02:40

jason810496 marked this pull request as ready for review October 8, 2026 04:04

jason810496 requested review from amoghrajesh, ashb, bugraoz93, gopidesupavan, jscheffl and potiuk as code owners October 8, 2026 04:04

jason810496 removed the area:DAG-processing label Oct 8, 2026

jason810496 requested review from guan404ming and henry3260 October 8, 2026 04:06

jason810496 self-assigned this Oct 8, 2026

jason810496 mentioned this pull request Oct 8, 2026

Java SDK: Trigger another Dag's run from a native Java Dag #74440

Merged

1 task done

guan404ming approved these changes Oct 8, 2026

View reviewed changes

phanikumv reviewed Oct 8, 2026

View reviewed changes

Comment thread go-sdk/airflow/trigger_dag_run.go

FrankYang0529 approved these changes Oct 8, 2026

View reviewed changes

jason810496 added 4 commits October 9, 2026 07:09

jason810496 and others added 2 commits October 9, 2026 07:09

Go SDK: document how the runtime runs a TriggerDagRun task

6c99272

Say how the task names its run, what Deferrable needs, and that the arguments are not templated and OpenLineage parent info is not added.

Go SDK: reject a templated DagID or RunID of TriggerDagRun

5a3b530

Mark the skipped and deferred states supported, since a TriggerDagRun task now emits both. Co-Authored-By: Claude <noreply@anthropic.com>

jason810496 force-pushed the feature/go-sdk/trigger-dag-run-runtime branch from 4e8b042 to 5a3b530 Compare October 9, 2026 07:10

jason810496 merged commit 30a3a6d into apache:main Oct 9, 2026
185 of 186 checks passed

jason810496 mentioned this pull request Oct 9, 2026

[DO NOT MERGE] Mirror of Java SDK #73596, #71189, #71190 for the native Dag stack #73846

Closed

1 task done

Davda-James mentioned this pull request Oct 10, 2026

ts-sdk: support for logicalDate and runAfter in triggerDagRun #74543

Open

Navigation Menu

Repository navigation

Go SDK: run TriggerDagRun tasks in the Go runtime - #74417

Go SDK: run TriggerDagRun tasks in the Go runtime#74417
jason810496 merged 6 commits into
apache:mainfrom
jason810496:feature/go-sdk/trigger-dag-run-runtime

jason810496 commented Oct 7, 2026 •

edited

Loading

Uh oh!

Uh oh!

FrankYang0529 left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

jason810496 commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

How

What

Was generative AI tooling used to co-author this PR?

Uh oh!

Uh oh!

FrankYang0529 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jason810496 commented Oct 7, 2026 •

edited

Loading