Skip to content

Commit cdd3c79

Browse files
jason810496claude
jason810496
and
claude
committed
Add consolidated Language SDK compatibility matrix
Users comparing Language SDKs had to read each SDK's README, and the per-SDK manifests lagged behind what the SDKs actually ship. One generated matrix, split by authoring mode, shows what each SDK supports for a Dag definition and for a Python Dag with stub TaskHandler. Co-Authored-By: Claude <noreply@anthropic.com>
1 parent a52e9f9 commit cdd3c79

18 files changed

Lines changed: 1372 additions & 387 deletions

‎.pre-commit-config.yaml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -495,6 +495,19 @@ repos:
495495
additional_dependencies: ['PyYAML>=6.0', 'rich>=13.6.0']
496496
pass_filenames: false
497497
require_serial: true
498+
- id: update-lang-sdk-central-matrix
499+
name: Update the consolidated Language SDK compatibility matrix
500+
entry: ./scripts/ci/prek/update_lang_sdk_central_matrix.py
501+
language: python
502+
files: >
503+
(?x)
504+
^(go|java|ts)-sdk/capabilities\.yaml$|
505+
^airflow-core/docs/authoring-and-scheduling/language-sdks/index\.rst$|
506+
^scripts/ci/prek/lang_sdk_compat_matrix\.py$|
507+
^scripts/ci/prek/update_lang_sdk_central_matrix\.py$
508+
additional_dependencies: ['PyYAML>=6.0', 'rich>=13.6.0']
509+
pass_filenames: false
510+
require_serial: true
498511
- id: check-go-example-mod-tidy
499512
name: Check lang-SDK Go example module is tidy against the Go SDK
500513
entry: ./scripts/ci/prek/check_go_example_mod_tidy.py

‎airflow-core/docs/authoring-and-scheduling/language-sdks/go.rst‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -701,6 +701,6 @@ All ``kwargs`` in the ``coordinators`` config entry are passed to the
701701
Limitations
702702
-----------
703703

704-
* **A Python stub Dag is still required.** The Execution API does not yet carry Dag structure for non-Python
705-
languages, so task names and dependencies are declared in Python with
706-
:func:`@task.stub <airflow.sdk.task.stub>`. This is a documented known limitation.
704+
* **A Dag declared in Go cannot express everything yet.** Dag params, assets, dynamic task mapping and
705+
human-in-the-loop tasks have no Go form. Declare such a Dag in Python, and implement its tasks in Go with
706+
:func:`@task.stub <airflow.sdk.task.stub>`.

‎airflow-core/docs/authoring-and-scheduling/language-sdks/index.rst‎

Lines changed: 223 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -204,6 +204,229 @@ Both settings can be supplied as environment variables using the standard Airflo
204204
AIRFLOW__SDK__COORDINATORS='{"my-coordinator": {...}}'
205205
AIRFLOW__SDK__QUEUE_TO_COORDINATOR='{"jdk17": "my-coordinator"}'
206206
207+
Compatibility matrix
208+
--------------------
209+
210+
Language SDKs mature independently, so they do not all support the same features yet. The tables below show
211+
what each Language SDK supports: the features common to both authoring modes, the features of a
212+
:ref:`Dag definition <language-sdks/dag-definition>`, and the features of a
213+
:ref:`Python Dag with stub TaskHandler <language-sdks/stub-taskhandler>`. A supported feature shows the SDK
214+
release that shipped it. See the legend below the tables for what each mark means.
215+
216+
.. BEGIN AUTO-GENERATED LANG-SDK COMPAT MATRIX
217+
218+
.. list-table:: Language SDK versions
219+
:header-rows: 1
220+
:widths: 30 15 15 15
221+
222+
* - Version
223+
- Go
224+
- Java
225+
- TypeScript
226+
* - Min. Airflow version
227+
- 3.4.0
228+
- 3.4.0
229+
- 3.4.0
230+
* - Current Supervisor Schema Version
231+
- 2026-10-30
232+
- 2026-10-30
233+
- 2026-10-30
234+
235+
.. list-table:: Common to both authoring modes
236+
:header-rows: 1
237+
:widths: 30 15 15 15
238+
239+
* - Dimension
240+
- Go
241+
- Java
242+
- TypeScript
243+
* - **TaskInstance states**
244+
-
245+
-
246+
-
247+
* - ``success`` (MUST)
248+
- ✓ 1.0.0
249+
- ✓ 1.0.0
250+
- ✓ 1.0.0
251+
* - ``failed`` (MUST)
252+
- ✓ 1.0.0
253+
- ✓ 1.0.0
254+
- ✓ 1.0.0
255+
* - ``up_for_retry`` (MUST)
256+
- ✓ 1.0.0
257+
- ✓ 1.0.0
258+
- ✓ 1.0.0
259+
* - ``skipped`` (SHOULD)
260+
- ✓ 1.0.0
261+
- ✓ 1.0.0
262+
- ✓ 1.0.0
263+
* - ``deferred`` (MAY)
264+
- ✓ 1.0.0
265+
- ✓ 1.0.0
266+
- ✓ 1.0.0
267+
* - ``up_for_reschedule`` (MAY)
268+
- ✗
269+
- ✗
270+
- ✗
271+
* - ``awaiting_input`` (MAY)
272+
- ✗
273+
- ✗
274+
- ✓ 1.0.0
275+
* - ``removed`` (MAY)
276+
- ✓ 1.0.0
277+
- ✓ 1.0.0
278+
- ✓ 1.0.0
279+
* - **Task runtime**
280+
-
281+
-
282+
-
283+
* - ``task-logging`` (MUST)
284+
- ✓ 1.0.0
285+
- ✓ 1.0.0
286+
- ✓ 1.0.0
287+
* - ``xcom-read-write`` (MUST)
288+
- ✓ 1.0.0
289+
- ✓ 1.0.0
290+
- ✓ 1.0.0
291+
* - ``connection-read`` (MUST)
292+
- ✓ 1.0.0
293+
- ✓ 1.0.0
294+
- ✓ 1.0.0
295+
* - ``variable-read-write`` (MUST)
296+
- ✓ 1.0.0
297+
- ✓ 1.0.0
298+
- ✓ 1.0.0
299+
* - ``self-contained-bundle`` (MUST)
300+
- ✓ 1.0.0
301+
- ✓ 1.0.0
302+
- ✓ 1.0.0
303+
* - ``retry-policy`` (MAY)
304+
- ✗
305+
- ✗
306+
- ✗
307+
* - ``task-state-store`` (MAY)
308+
- ✓ 1.0.0
309+
- ✓ 1.0.0
310+
- ✓ 1.0.0
311+
* - ``asset-state-store`` (MAY)
312+
- ✗
313+
- ✗
314+
- ✗
315+
* - ``asset-event-emit`` (MAY)
316+
- ✗
317+
- ✗
318+
- ✗
319+
* - ``asset-event-read`` (MAY)
320+
- ✗
321+
- ✗
322+
- ✗
323+
* - ``object-store`` (MAY)
324+
- ✗
325+
- ✗
326+
- ✗
327+
328+
.. list-table:: Dag definition
329+
:header-rows: 1
330+
:widths: 30 15 15 15
331+
332+
* - Dimension
333+
- Go
334+
- Java
335+
- TypeScript
336+
* - ``native-dag-authoring`` (SHOULD)
337+
- ✓ 1.0.0
338+
- ✓ 1.0.0
339+
- ✓ 1.0.0
340+
* - ``dag-args`` (MUST †)
341+
- ✓ 1.0.0
342+
- ✓ 1.0.0
343+
- ✓ 1.0.0
344+
* - ``task-args`` (MUST †)
345+
- ✓ 1.0.0
346+
- ✓ 1.0.0
347+
- ✓ 1.0.0
348+
* - ``dag-params`` (MUST †)
349+
- ✗
350+
- ✗
351+
- ✗
352+
* - ``explicit-dependencies`` (MUST †)
353+
- ✓ 1.0.0
354+
- ✓ 1.0.0
355+
- ✓ 1.0.0
356+
* - ``taskflow-dependencies`` (MUST †)
357+
- ✓ 1.0.0
358+
- ✓ 1.0.0
359+
- ✓ 1.0.0
360+
* - ``task-group`` (MAY †)
361+
- ✓ 1.0.0
362+
- ✓ 1.0.0
363+
- ✓ 1.0.0
364+
* - ``if-else-branching`` (SHOULD †)
365+
- ✓ 1.0.0
366+
- ✓ 1.0.0
367+
- ✓ 1.0.0
368+
* - ``switch-case-branching`` (SHOULD †)
369+
- ✓ 1.0.0
370+
- ✓ 1.0.0
371+
- ✓ 1.0.0
372+
* - ``trigger-dag-run`` (MAY †)
373+
- ✓ 1.0.0
374+
- ✓ 1.0.0
375+
- ✓ 1.0.0
376+
* - ``dynamic-dag-generation`` (MAY †)
377+
- ✓ 1.0.0
378+
- ✓ 1.0.0
379+
- ✓ 1.0.0
380+
* - ``human-in-the-loop`` (MAY †)
381+
- ✗
382+
- ✗
383+
- ✓ 1.0.0
384+
* - ``dag-test`` (SHOULD †)
385+
- ✗
386+
- ✗
387+
- ✗
388+
* - ``asset-scheduling`` (MAY †)
389+
- ✗
390+
- ✗
391+
- ✗
392+
* - ``dynamic-task-mapping`` (MAY †)
393+
- ✗
394+
- ✗
395+
- ✗
396+
* - ``asset-inlets-outlets`` (MAY †)
397+
- ✗
398+
- ✗
399+
- ✗
400+
401+
.. list-table:: Python Dag with stub TaskHandler
402+
:header-rows: 1
403+
:widths: 30 15 15 15
404+
405+
* - Dimension
406+
- Go
407+
- Java
408+
- TypeScript
409+
* - ``mixed-lang-stub-target`` (MUST)
410+
- ✓ 1.0.0
411+
- ✓ 1.0.0
412+
- ✓ 1.0.0
413+
* - ``taskflow-binding`` (MUST)
414+
- ✓ 1.0.0
415+
- ✓ 1.0.0
416+
- ✓ 1.0.0
417+
* - ``dynamic-task-mapping`` (MAY)
418+
- ✗
419+
- ✗
420+
- ✗
421+
* - ``asset-inlets-outlets`` (MAY)
422+
- ✗
423+
- ✗
424+
- ✗
425+
426+
*Marks:* ``✓`` supported (with the SDK release that shipped it, when known) · ``✗`` not supported · ``n/a`` not applicable · ``–`` not published. A tier marked ``†`` applies only when ``native-dag-authoring`` is supported.
427+
428+
.. END AUTO-GENERATED LANG-SDK COMPAT MATRIX
429+
207430
.. _language-sdks/bundle-spec:
208431

209432
Implementing a new compiled language SDK

0 commit comments

Comments
 (0)