Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -537,16 +537,15 @@ calls with no arguments.
Native Java Dags
----------------

A Dag can also be authored entirely in Java, with no Python stub file: the ``DagDef`` and
``TaskDef`` objects hold the tasks, and Java declares the graph.
A Dag can also be authored entirely in Java: the ``DagDef`` and ``TaskDef`` objects hold the
tasks, and Java declares the graph.

Building the Dag in Java
~~~~~~~~~~~~~~~~~~~~~~~~

``dag.task(...)`` registers a task as it creates it and hands back a handle, so there is no second
``addTask`` call to forget. ``before`` and ``after`` draw every edge on this surface — Python's
``a >> b`` and ``b << a`` — and the task body moves the data itself, by reading the upstream's XCom
through ``Client``:
``addTask`` call to forget. ``before`` and ``after`` draw every edge on this surface, and the
task body moves the data itself, by reading the upstream's XCom through ``Client``:

.. code-block:: java

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,8 @@

import org.apache.airflow.sdk.*;

// A Dag defined entirely in Java, interface-style: no Python stub file
// describes it. dag.task registers a task as it creates it and hands back the
// handle, and `before`/`after` wire the graph -- Java's spelling of `>>` and `<<`.
// A Dag defined entirely in Java, interface-style. dag.task registers a task as
// it creates it and hands back the handle, and `before`/`after` wire the graph.
public class InterfaceExample {
private static final System.Logger log = System.getLogger(InterfaceExample.class.getName());

Expand Down
18 changes: 8 additions & 10 deletions java-sdk/sdk/src/main/kotlin/org/apache/airflow/sdk/Deps.kt
Original file line number Diff line number Diff line change
Expand Up @@ -24,20 +24,19 @@ interface Deps {
/**
* A point in the task graph: one task, or a set of them.
*
* [Flow] is Java's spelling of Python's `>>` and `<<`, for a dependency
* where nothing flows but the ordering. An edge that carries a value is
* declared by passing the upstream's handle instead.
* [Flow] declares a dependency where nothing flows but the ordering. An
* edge that carries a value is declared by passing the upstream's handle
* instead.
*/
interface Flow {
/** The tasks at this point in the flow. */
fun nodes(): List<TaskDef>

/**
* Runs the tasks here before each of [next], carrying no value — Java's
* spelling of Python's `>>`.
* Runs the tasks here before each of [next], carrying no value.
*
* ```java
* loaded.before(cleaned, notified); // load >> [cleanup, notify]
* loaded.before(cleaned, notified); // cleanup and notify both wait for load
* ```
*
* Variadic, so one call fans out, and it returns its own receiver: a
Expand All @@ -54,11 +53,10 @@ interface Deps {
}

/**
* Runs the tasks here after each of [previous], carrying no value —
* Python's `<<`.
* Runs the tasks here after each of [previous], carrying no value.
*
* ```java
* cleaned.after(loaded, transformed); // [load, transform] >> cleanup
* cleaned.after(loaded, transformed); // cleanup waits for load and transform
* ```
*
* @param previous Tasks that run before the ones here.
Expand All @@ -76,7 +74,7 @@ interface Deps {
* every edge between two sets:
*
* ```java
* Flow.of(a, b).before(c, d); // [a, b] >> [c, d]
* Flow.of(a, b).before(c, d); // c and d both wait for a and b
* ```
*/
@JvmStatic
Expand Down
Loading