Repository navigation
Share Dag cycle detection between the Task SDK and core - #74004
Merged
jason810496 merged 0 commit intoOct 1, 2026
Conversation
jason810496
added this pull request to stack #74005
October 1, 2026 03:04
This was referenced Oct 1, 2026
jason810496
force-pushed
the
jason/lang-sdk-e2e/02-importer-registry
branch
from
October 1, 2026 12:41
57c9a34 to
a5d284b
Compare
jason810496
force-pushed
the
jason/lang-sdk-e2e/02b-shared-dag-cycle-detection
branch
from
October 1, 2026 12:41
e14a584 to
7caeacb
Compare
jason810496
removed this pull request from stack #74005
October 1, 2026 13:17
jason810496
force-pushed
the
jason/lang-sdk-e2e/02-importer-registry
branch
from
October 1, 2026 13:18
a5d284b to
c24e6fa
Compare
jason810496
force-pushed
the
jason/lang-sdk-e2e/02b-shared-dag-cycle-detection
branch
from
October 1, 2026 13:18
7caeacb to
5c20182
Compare
jason810496
deleted the
jason/lang-sdk-e2e/02b-shared-dag-cycle-detection
branch
October 1, 2026 13:18
jason810496
restored the
jason/lang-sdk-e2e/02b-shared-dag-cycle-detection
branch
October 1, 2026 13:36
This was referenced Oct 1, 2026
Member
Author
|
Replaced by #74034. GitHub marked this PR as merged into a stack branch when the stack was reordered; nothing was merged into |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack (bottom to top): #73841, #74004, #73842, #73843, #73844, #73845, #73846, #73847
Why
Cycle detection only runs for Dags that go through the Python SDK's
DAGobject. A Dag from another language SDK reaches core already serialized, so nothing rejects a cycle in it.How
Core cannot reuse
DAG.check_cycleas it is: it is a method on the authoring class and reads task objects, while core holds serialized data it should not hydrate just to run this check.shared/dagnode, whichtask-sdkandairflow-corealready depend on.AirflowDagCycleExceptionlives in the Task SDK, and importing it here would invert the dependency, so each caller raises its own error.What
detect_cycle(node_ids, downstream_of)inshared/dagnode/src/airflow_shared/dagnode/cycle.py, with its unit tests.DAG.check_cycle()delegates to it and keeps its message,Cycle detected in Dag: <dag_id>. Faulty task: <task_id>.No behavior change: the walk still follows downstream edges only and stays iterative, and the existing
TestCycleTestertests pass unchanged. No newsfragment, since nothing user-visible changes.How to test
shared/dagnode/tests/dagnode/test_cycle.pycoversdetect_cycleitself.Was generative AI tooling used to co-author this PR?