Skip to content

Commit 2e45493

Browse files
committed
fixup! Add user-facing docs for the new Task Loops feature.
Reviewers asked what happens when the gate condition returns something other than a bool, whether the context name `loop` is safe, and what a loop that ended without running means for downstream tasks. Each answer is a rule the page did not state: only an actual bool is accepted, `loop` is a context parameter inside a loop body and is shadowed by Jinja's own `loop` inside a for block, and the trigger rule that lets a downstream task run after a loop that ended early, or a gate marked successful by hand, is now shown instead of implied.
1 parent 3384f87 commit 2e45493

1 file changed

Lines changed: 27 additions & 5 deletions

File tree

  • airflow-core/docs/authoring-and-scheduling

‎airflow-core/docs/authoring-and-scheduling/loops.rst‎

Lines changed: 27 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -44,16 +44,20 @@ Define the body with ``@task_group`` and call ``.loop()`` on the decorated
4444
function. Supply a positive integer ``max_iterations`` to limit the number of
4545
iterations. An optional ``until`` callable decides when to stop early.
4646

47-
Airflow evaluates ``until`` in a gate task downstream of the body's terminal task:
47+
Airflow evaluates ``until`` in a gate task downstream of the body's terminal task.
48+
It must return a ``bool``:
4849

4950
* ``True`` means stop: the condition has been met.
5051
* ``False`` means continue, provided another iteration is allowed.
5152

53+
Any other return value, including the ``None`` that a forgotten ``return`` produces, will result in
54+
the gate task failing, and the loop not continuing.
55+
5256
Airflow creates the next iteration when the gate says to continue.
5357
It does not create all ``max_iterations`` up front:
5458
if the loop stops after four iterations, you see those four, without a tail
55-
of skipped iterations. For a fixed-count loop, the gate continues until the
56-
count is reached.
59+
of skipped iterations. A fixed-count loop runs exactly ``max_iterations``
60+
iterations.
5761

5862
This example improves an estimate of the square root of two until the error
5963
is small enough:
@@ -67,6 +71,13 @@ is small enough:
6771
through ``loop.previous``. The gate appears as a task in the loop, named after
6872
the condition function: ``refine.accurate_enough`` in this example.
6973

74+
Tasks inside a loop body receive ``loop`` as a context parameter, so a parameter with that name in a
75+
loop body must be keyword-only and cannot have a default other than ``None``.
76+
77+
In a templated field, a Jinja ``{% for %}`` block defines its own ``loop`` that hides this one for
78+
the length of the block. Read the Airflow value before the block and use that inside it, for example
79+
``{% set iteration = loop.index %}``.
80+
7081
When ``until`` has no usable name, the gate is named ``__loop_gate`` instead.
7182
That covers a lambda, a ``functools.partial`` and a callable object. A gate name
7283
that matches a task in the loop body gets a ``__1`` suffix, as with any other
@@ -224,14 +235,25 @@ Skipping the body's terminal task also skips the gate under its
224235
``all_success`` trigger rule, so no next iteration is created. Downstream
225236
tasks with the default ``all_success`` rule are skipped too; give a downstream
226237
task ``trigger_rule="none_failed"`` if it should still run when the loop ends
227-
without running.
238+
without running:
239+
240+
.. code-block:: python
241+
242+
@task(trigger_rule="none_failed")
243+
def finished(): ...
244+
245+
246+
refinement >> finished()
247+
228248
If some branches may be skipped but the loop should continue, give the final
229249
combining task a trigger rule that permits those skips and have it return the
230250
iteration's result.
231251

232252
Manually marking a gate successful completes it without evaluating ``until``
233253
or creating another iteration. This is an explicit override of normal gate
234-
execution. Any later iterations retained after a selective clear remain unchanged.
254+
execution. Downstream tasks then run as they would after any successful gate: they cannot tell a
255+
gate that was marked successful by hand from one that stopped because ``until`` returned ``True``.
256+
Any later iterations retained after a selective clear remain unchanged.
235257

236258
Iterations and execution history
237259
================================

0 commit comments

Comments
 (0)