Skip to content

Java SDK: Draw a Java-authored Dag's edges with before and after - #73594

Merged
jason810496 merged 2 commits into
apache:mainfrom
jason810496:feature/java-sdk-native-dag-edges
Sep 26, 2026
Merged

jason810496 merged 2 commits into
apache:mainfrom
jason810496:feature/java-sdk-native-dag-edges

Conversation

@jason810496

@jason810496 jason810496 commented Sep 23, 2026 •

Copy link
Copy Markdown
Member

Merge order:

  1. Java SDK: Honor TaskFlow arg bindings sent by the supervisor #71188 — Honor TaskFlow arg bindings sent by the supervisor
  2. Java SDK: Draw a Java-authored Dag's edges with before and after #73594 — Draw a Java-authored Dag's edges with before and after (current one)
  3. Java SDK: Generate Dag and task configuration from the serialization schema #73595 — Generate Dag and task configuration from the serialization schema
  4. Java SDK: Resolve a task's arguments from the Dag's own wiring #73596 — Resolve a task's arguments from the Dag's own wiring
  5. Java SDK: Declare a Dag's task graph with @Builder.Deps #71189 — Declare a Dag's task graph with @Builder.Deps
  6. Java SDK: Serialize native Dags to DagSerialization v3 #71190 — Serialize native Dags to DagSerialization v3

Every PR targets main because GitHub cannot base a pull request on a branch that exists only on a fork, so these diffs are cumulative — the compare link above shows only this layer. #69757 and #71057, which the stack was originally built on, are merged.

Why

A Dag authored in Java had no way to say what runs after what. Without a Python Dag file to hold the graph every task was a root, so a Dag that Java owns end to end could not be expressed at all. This is the graph half of ADR-0002.

Example

var dag = new DagDef("java_etl");

var extract = dag.task("extract", Extract.class);
var transform = dag.task("transform", Transform.class);
var load = dag.task("load", Load.class);

transform.after(extract).before(load);

How

  • dag.task(...) registers a task as it creates it and hands back the handle, so there is no second addTask to forget.
  • before and after are Java's spelling of Python's >> and <<, named to match the TypeScript SDK and to leave then free for the conditional control flow that reads as a then-branch. They are the only way to draw an edge, so one graph has one spelling however it was built.
  • Both are variadic, so a.before(b, c) fans out and d.after(b, c) fans in, and both return their own receiver: a fan-out has no single next task to hand back.
  • Flow.of(a, b).before(c, d), from org.apache.airflow.sdk.Deps.Flow, draws every edge between two sets in one call.
  • A Dag is checked as it is registered with a Bundle: an upstream that belongs to another Dag, and a cycle anywhere in the graph, both fail there rather than at the first task run. A Dag that fails the check is not added to the bundle.
  • example/.../nativedag/InterfaceExample.java is a Dag with no Python counterpart.

Was generative AI tooling used to co-author this PR?

A Dag authored in Java had no way to say what runs after what. Without a
Python Dag file to hold the graph, every task was a root, so a Dag that
Java owns end to end could not be expressed at all.

The verbs are before and after, matching the TypeScript SDK, and leaving
then free for the conditional control flow that reads as a then-branch.
They are the only way to draw an edge, so one graph has one spelling
however it was built.
@uranusjr
uranusjr force-pushed the feature/java-sdk-native-dag-edges branch from d84462c to 9c1af34 Compare September 26, 2026 01:25
@uranusjr

Copy link
Copy Markdown
Member

Since the dependent PR has been merged, I took the liberty to rebase this against main.

@uranusjr uranusjr left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a couple of notes on implementation choices. This now looks good to me. (Pending CI and a final look from Jason.)

@uranusjr
uranusjr marked this pull request as ready for review September 26, 2026 09:50

@jason810496 jason810496 left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update, TP, LTGM.

@jason810496
jason810496 merged commit 77a1953 into apache:main Sep 26, 2026
176 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants