Repository navigation
Status of testing Providers that were prepared on October 06, 2026 #74459
Description
Activity
Verified my changes in
apache-airflow-providers-amazon==9.38.0rc1for #73756 and #73881 using the PyPI wheel, with Airflow 3.3.2 / Python 3.12.14 on Linux ARM64. Updated with live AWS verification inap-northeast-2.- Keep S3 Dag bundle downloads within the configured directory #73756: All 8 checks pass against a real S3 bucket: bundle initialization and repeated refresh with directory prefixes with/without trailing slashes, sibling-prefix objects, and objects whose key equals the configured prefix. Verified downloaded contents, stale-file/directory cleanup, and preservation of excluded remote objects.
- Handle plain Step Functions execution errors #73881: All 9 checks pass using actual Standard Step Functions executions and
DescribeExecutionresponses. Plain, Unicode, and malformed-JSON error strings are returned correctly; JSON-valued errors and successful output retain their types. In the live empty-stringErrorPathcase, AWS omitted theerrorfield and the operator correctly returnedNone.
The 17 live-AWS checks all passed. Running the same checks with provider 9.37.0, using the same bucket and executions, reproduced 7 failures (4 S3, 3 Step Functions); reinstalling 9.38.0rc1 restored all 17 passes. Temporary buckets, state machines, and execution roles were deleted and their absence confirmed.
The earlier 20 local checks using Moto/synthetic HTTP responses also passed (8 failures on 9.37.0), including explicit
error: "", malformed success output, and service-error propagation. Those additional cases remain simulated.pip checkpassed, both merged commits are in the RC tag, and the relevant installed sources match the wheel and tag.Scope: the live checks make real AWS calls from the installed bundle/operator/hook. Operator execution is invoked directly, with UI extra-link persistence mocked; this does not cover scheduler/task-runner/XCom end-to-end behavior, an IAM-policy matrix, or the whole Amazon provider.
Reacted by Jarek Potiuk#73652 tested in
common.ai0.11.0rc1, works as expected.
Thanks for preparing the release, Jarek.Reacted by Jarek PotiukReacted by Jarek Potiuk#74017 tested that it's included in common-ai 0.11.0rc1 and works as expected.
Thank you for preparing the releaseReacted by Jarek Potiuk#71676 and #74191 tested on rc1, works.
- Add OpenSandbox backend for sandbox tools #71676 (
common.ai0.11.0rc1): I ran the PyPI wheel on Airflow 3.3.2 / Python 3.12 against a local OpenSandbox server. The PR's system-test Dag and its 64 unit tests pass, and command timeouts and sandbox cleanup behave as expected. One caveat, for a docs follow-up: when the server's egress sidecar runs in its defaultdnsmode, a deny-allSandboxSpec()only blocks DNS, so direct-IP connections still get out. With[egress] mode = "dns+nft"they are blocked too, and everything above still passes. The backend can't tell the two modes apart on policy read-back. - Notify downstream Dags when COPY INTO writes a Unity (Databricks) table #74191 (
databricks7.22.0rc1): on a real SQL warehouse, COPY INTO emitted the canonical Unity table asset event and triggered the downstream Dag. Wrong table, wrong workspace and failed SQL emitted no event.
Drafted-by: Claude Code (Opus 5.5) (no human review before posting)
Reacted by Jarek Potiuk- Add OpenSandbox backend for sandbox tools #71676 (
#71711 tested in apache-spark 6.3.3rc1, works as expected.
Verified against the RC in a clean venv: all masking forms produce the expected output (
key=value,key value, quoted values spanning tokens, uppercase keys, tab- and multi-space-separated values, and a sensitive flag with no value), and a multiline value no longer swallows the log lines that follow it.The scanner that replaced the regex also closes the complexity gap raised in review on my PR —
"secret" * 10000, which still backtracked quadratically in my version (~2.4s), now completes in 0.16ms. 50k single token: 0.66ms.Thanks for preparing the release.
Drafted-by: Claude Code; reviewed by @divyanshus2404 before posting
Reacted by Jarek Potiuk#73509 tested with
apache-airflow-providers-amazon==9.38.0rc1(Airflow 3.3.2, common-messaging 2.1.1), works as expected :)Reacted by Jarek PotiukHave tested my changes
- Add dedicated InfluxDB 3 sensor #73522 [
influxdb: 2.13.0rc1] - Fix Elasticsearch and OpenSearch response wrapper bugs #73725 [
elasticsearch: 6.9.2rc1] and [opensearch: 1.14.0rc1]
using my agent, looks good:
Thanks!
Reacted by Jarek Potiuk- Add dedicated InfluxDB 3 sensor #73522 [
Tested both of the changes I am listed for. Both work as expected.
Environment: clean virtualenv on Python 3.11, installed from PyPI,
apache-airflow3.3.2. No dbt Cloud account or GCP project involved, since everything below is observable before the first API call. In each case I also checked the previous stable release to confirm the result depends on the change rather than passing either way.dbt.cloud 4.11.0rc1, Apply hook_params to deferred dbt Cloud job runs (#74199)
Walking the installed wheel, the
DbtCloudRunJobTriggerconstruction inoperators/dbt.pynow passeshook_params; on4.10.0it does not. In both releases the trigger already accepted it, kept it acrossserialize()and built its hook with**self.hook_params, so the operator's call site was the only gap and nothing else had to change.4.10.0 4.11.0rc1 hook_paramspassed at the defer siteabsent present Checked at runtime with
hook_params={"retry_limit": 7, "retry_delay": 3.5}: the operator retains them, its own hook honours them, and a trigger built with them serializes them intact.google 22.7.0rc1, Apply impersonation_chain to deferred BigQuery existence checks (#71648)
Both deferral sites in
sensors/bigquery.pynow passimpersonation_chain, one forBigQueryTableExistenceTriggerand one forBigQueryTablePartitionExistenceTrigger; on22.6.0neither does. As above, the triggers accepted and serialized the argument in both releases, so only the sensors needed the change.22.6.0 22.7.0rc1 BigQueryTableExistenceTriggerabsent present BigQueryTablePartitionExistenceTriggerabsent present Confirmed the trigger accepts
impersonation_chainand serializes it as given.Import smoke check
Importing every module of both providers: all 12 dbt Cloud modules import, and 297 of 333 google modules. The 36 that do not are all missing optional peers in this environment, other providers plus paramiko, kubernetes, apache-beam, cassandra, oracledb and similar, each failing with an explicit message naming what is absent. Nothing failed for a reason internal to either release.
Reacted by Jarek PotiukTested apache.kafka 2.1.0rc1 and it works as expected:
- Add KafkaSharedStreamProducer and KafkaSharedStreamTrigger #68625: I ran two
AssetWatchers withKafkaSharedStreamTriggeron the same topic. The two triggers share one Kafka consumer. Both assets get every message. Offsets are committed per partition only after the asset events are stored. After a restart, committed messages are not delivered again. A connection withoutenable.auto.commit=falseis refused. On Airflow 3.1.8 and 2.11.2, importing the shared-stream triggers raisesAirflowOptionalProviderFeatureException.
I found some issues for following providers:
- google 22.7.0rc1
- Add GKEPodExecOperator for existing Pods #72577:
operators/kubernetes_engine.pynow importsKubernetesPodExecOperatorat module level.KubernetesPodExecOperatoronly exists in cncf-kubernetes 10.22.0 and later (https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes/10.22.0/changelog.html). With an older cncf-kubernetes, importingoperators/kubernetes_engine.pyraisesAirflowOptionalProviderFeatureException. So every GKE operator fails to import, not onlyGKEPodExecOperator. Should we update the GKE module like the EKS module? In Add EKS operator for commands in existing Pods #72542, ifeks.pycannot importKubernetesPodExecOperator, it defines a fallback class with the same name. The__init__of that fallback class raises the same exception that the GKE module raises on import. The other EKS operators still import, and only creatingEksPodExecOperatorfails. I can create a follow-up PR for this.
- Add GKEPodExecOperator for existing Pods #72577:
- dbt.cloud 4.11.0rc1
- Apply hook_params to deferred dbt Cloud job runs #74199: A
DbtCloudRunJobOperatorwithdeferrable=Truethat works on 4.10.0 fails on 4.11.0rc1 whenhook_paramsholdsretry_argswith tenacity objects. Created a follow-up PR for it Fix deferred dbt Cloud tasks failing with tenacity retry_args #74489.
- Apply hook_params to deferred dbt Cloud job runs #74199: A
Reacted by Jarek Potiuk and Alejandro Morgante- Add KafkaSharedStreamProducer and KafkaSharedStreamTrigger #68625: I ran two
Correcting my earlier comment: my verification of #74199 in
dbt.cloud 4.11.0rc1was wrong.I checked that
hook_paramssurvivesserialize(), but I used{"retry_limit": 7, "retry_delay": 3.5}, two JSON serializable scalars.hook_paramsis a free form dict and the one key whose documented values are Python objects isretry_args, so my input could not exercise the case that matters. With a tenacity object inretry_argsthe task fails when it defers, and it fails after the dbt Cloud job has already started, which leaves the run orphaned.Thanks to @FrankYang0529 for catching it and for the fix in #74489.
Treating that as a release blocker for
dbt.cloud 4.11.0: shipping rc1 as is would regress any deferrableDbtCloudRunJobOperatorwhosehook_paramscarriesretry_args, which worked on 4.10.0. The google 22.7.0rc1 item I reported is unaffected and my result there stands.Reacted by PoAn YangReacted by Jarek Potiuk#74406 works as expected on the databricks 7.22.0 release candidate (with common-compat 1.21.0, live workspace): a deferrable
DatabricksRunNowOperatorwith a DB-backed connection now defers and succeeds on Airflow 3.0.6, where 7.21.0 failed withYou cannot use AsyncToSync in the same thread as an async event loop. Also checked on 3.2.2.
Drafted-by: Claude Code (Opus 5.5); reviewed by @moomindani before posting
Reacted by Jarek PotiukThank you all for the testing and for such detailed reports. The live-AWS, real-warehouse and before/after-the-RC comparisons are especially helpful. I've ticked the verified items in the issue description.
Two items need follow-up:
- dbt.cloud 4.11.0rc1 / Apply hook_params to deferred dbt Cloud job runs #74199: thanks @FrankYang0529 for catching the
retry_argsserialization regression and for the fix in Fix deferred dbt Cloud tasks failing with tenacity retry_args #74489. Thanks also @SEPURI-SAI-KRISHNA for correcting your earlier verification so quickly. We will exclude dbt.cloud from this wave and release it with the fix in the next RC. - google 22.7.0rc1 / Add GKEPodExecOperator for existing Pods #72577: good catch on the module-level
KubernetesPodExecOperatorimport. A follow-up PR that mirrors the EKS fallback-class approach from Add EKS operator for commands in existing Pods #72542 would be very welcome. Please go ahead.
@zozo123: thanks for the OpenSandbox egress
dnsvsdns+nftfinding. A docs follow-up for that would be great.
Drafted-by: Claude Code (Opus 5.5); reviewed by @potiuk before posting
Reacted by Alejandro Morgante- dbt.cloud 4.11.0rc1 / Apply hook_params to deferred dbt Cloud job runs #74199: thanks @FrankYang0529 for catching the
Validated the following changes included in the current provider release candidates:
apache-airflow-providers-amazon==9.38.0rc1— Use POSIX-compliant tail to extract token line in EKS command #73690: the POSIX/bin/shregression passed (1 passed); token and expiration timestamp extraction were verified with extra stdout from a stub token helper.
Environment: Breeze, Airflow 3.3.0, Python 3.11.16; providers installed from PyPI RC wheels.
I’ll prepare a follow-up PR for GKE to mirror the EKS fallback-class approach, so importing the module still works when
KubernetesPodExecOperatoris unavailable. #74505Reacted by Jarek PotiukReacted by Jarek PotiukTested #74305 and #72953 in
apache-airflow-providers-google==22.7.0rc1. Both work as expected.Setup: a clean virtualenv with Python 3.12.13,
apache-airflow==3.3.2and the RC from PyPI. I ran the same checks against 22.6.0 for comparison. I don't have a GCP project, so the BigQuery job states and the GCS blob calls are faked. The triggers, the operator, the tenacity retry policy and the transform subprocess are the installed code. shahar1's before/after run against real BigQuery is in #74356.Check 22.6.0 22.7.0rc1 #74305: value check, job RUNNINGon the first pollfails with Job runningkeeps polling, succeeds #74305: interval check, first job RUNNINGon the first pollfails with Job completed(the second job's message)keeps polling, succeeds #74305: interval check, first job fails while the second runs reports Job runningreports the first job's error #72953: download fails twice, download_num_attempts=3fails after 1 call succeeds on call 3, after 2 s and 4 s waits #72953: upload fails once, upload_num_attempts=2fails after 1 call succeeds on call 2 #72953: download fails 3 times, download_num_attempts=3raises after 1 call raises after 3 calls Script
"""Check #74305 and #72953 against an installed apache-airflow-providers-google wheel. No GCP access: BigQuery job states and GCS blob operations are faked, everything else (triggers, operator, tenacity retry policy, transform subprocess) is the installed code. """ from __future__ import annotations import asyncio import importlib.metadata import logging import os import stat import sys import tempfile from pathlib import Path from unittest import mock import pendulum from google.api_core.exceptions import ServiceUnavailable logging.basicConfig(level=logging.WARNING, format=" log: %(message)s") VERSION = importlib.metadata.version("apache-airflow-providers-google") print(f"apache-airflow {importlib.metadata.version('apache-airflow')}, " f"apache-airflow-providers-google {VERSION}, Python {sys.version.split()[0]}") from airflow.providers.google.cloud.triggers.bigquery import ( # noqa: E402 BigQueryIntervalCheckTrigger, BigQueryValueCheckTrigger, ) RUNNING = {"status": "running", "message": "Job running"} DONE = {"status": "success", "message": "Job completed"} FAILED = {"status": "error", "message": "Syntax error: Unexpected keyword FROM at [1:8]"} def run_trigger(trigger, job_states): with mock.patch("airflow.providers.google.cloud.triggers.bigquery.BigQueryAsyncHook") as hook_cls: hook = hook_cls.return_value hook.get_job_status = mock.AsyncMock(side_effect=job_states) hook.get_sync_hook = mock.AsyncMock( return_value=mock.MagicMock(is_default_universe=mock.MagicMock(return_value=True)) ) hook.get_job_output = mock.AsyncMock(return_value={}) hook.get_records = mock.MagicMock(side_effect=lambda _: [[2]]) async def collect(): return [event.payload async for event in trigger.run()] events = asyncio.run(collect()) return events[0], hook.get_job_status.await_count def show(name, expected, event, polls): got = f"{event['status']}: {event['message']}" verdict = "OK " if got == expected else "BAD" print(f" [{verdict}] {name}\n expected {expected!r}\n got {got!r} ({polls} status calls)") print("\n#74305 BigQuery check triggers") value = BigQueryValueCheckTrigger( conn_id="google_cloud_default", sql="SELECT COUNT(*) FROM t", pass_value=2, job_id="job_1", project_id="p", poll_interval=0.01, ) event, polls = run_trigger(value, [RUNNING, DONE]) show("value check, job RUNNING on the first poll", "success: Job completed", event, polls) def interval(): return BigQueryIntervalCheckTrigger( conn_id="google_cloud_default", first_job_id="job_1", second_job_id="job_2", project_id="p", table="t", metrics_thresholds={"COUNT(*)": 1.5}, poll_interval=0.01, ) # get_job_status is awaited for the first job, then the second job, on every poll. event, polls = run_trigger(interval(), [RUNNING, DONE, DONE, DONE]) show("interval check, first job RUNNING on the first poll", "success: Job completed", event, polls) event, polls = run_trigger(interval(), [FAILED, RUNNING]) show("interval check, first job FAILED while the second runs", f"error: {FAILED['message']}", event, polls) print("\n#72953 GCSTimeSpanFileTransformOperator retry attempts") from airflow.providers.google.cloud.operators.gcs import GCSTimeSpanFileTransformOperator # noqa: E402 class FakeGCS: """A GCS client whose first download_failures downloads and upload_failures uploads fail.""" def __init__(self, download_failures, upload_failures): self.left = {"download": download_failures, "upload": upload_failures} self.calls = {"download": 0, "upload": 0} self.uploaded = [] def _maybe_fail(self, kind): self.calls[kind] += 1 if self.left[kind] > 0: self.left[kind] -= 1 raise ServiceUnavailable(f"{kind} attempt {self.calls[kind]}: 503 backend unavailable") def bucket(self, bucket_name): return self def blob(self, blob_name, chunk_size=None): gcs = self class Blob: def download_to_filename(self, filename): gcs._maybe_fail("download") Path(filename).write_text("hello\n") def upload_from_filename(self, filename): gcs._maybe_fail("upload") gcs.uploaded.append(blob_name) return Blob() def run_operator(download_failures, upload_failures, **attempts): gcs = FakeGCS(download_failures, upload_failures) hook = mock.MagicMock(project_id="p") hook.list_by_timespan.return_value = ["in/2026-10-09/a.txt"] hook.get_conn.return_value = gcs with tempfile.TemporaryDirectory() as tmp: script = Path(tmp) / "transform.sh" script.write_text('#!/bin/sh\ncp -R "$1"/. "$2"/\n') # copy input to output script.chmod(script.stat().st_mode | stat.S_IEXEC) op = GCSTimeSpanFileTransformOperator( task_id="transform", source_bucket="src", source_prefix="in/%Y-%m-%d/", source_gcp_conn_id="google_cloud_default", destination_bucket="dst", destination_prefix="out/", destination_gcp_conn_id="google_cloud_default", transform_script=str(script), **attempts, ) context = { "data_interval_start": pendulum.datetime(2026, 10, 9), "data_interval_end": pendulum.datetime(2026, 10, 10), "ti": mock.MagicMock(), } with ( mock.patch("airflow.providers.google.cloud.operators.gcs.GCSHook", return_value=hook), mock.patch("tenacity.nap.time.sleep"), # keep the retry waits, without waiting ): try: result = op.execute(context) outcome = f"success, uploaded {result}" except Exception as e: outcome = f"{type(e).__name__}: {e}" return outcome, gcs.calls for name, failures, attempts, expect in [ ("download fails twice, download_num_attempts=3", (2, 0), {"download_num_attempts": 3}, "success"), ("upload fails once, upload_num_attempts=2", (0, 1), {"upload_num_attempts": 2}, "success"), ("download fails 3 times, download_num_attempts=3", (3, 0), {"download_num_attempts": 3}, "ServiceUnavailable"), ]: outcome, calls = run_operator(*failures, **attempts) verdict = "OK " if outcome.startswith(expect) else "BAD" print(f" [{verdict}] {name}\n expected {expect}\n got {outcome} " f"(download calls {calls['download']}, upload calls {calls['upload']})")
Reacted by Jarek PotiukTested celery 3.25.0rc1 with
apache/airflow:3.3.2and[operators] default_queue = custom_queue: Use configured default queue for Celery executor callbacks (#73552) works as expected.Passing an
ExecuteCallbackworkload toCeleryExecutor._process_workloadsand checking the queue it is sent to:Callback celery 3.24.1 celery 3.25.0rc1 no queuein callback datadefault❌custom_queue✅queueset in callback datacallback_queuecallback_queue✅The configured default queue is now used, and a queue set in the callback data still takes precedence.
Reacted by Jarek PotiukValidated the following changes and the follow-up requested for this provider release candidate:
apache-airflow-providers-google==22.7.0rc1— Add GKEPodExecOperator for existing Pods #72577: the earlierexample_kubernetes_engine.pyrun passed against real GKE Autopilot (1 passedin 1099.27s), with Kubernetes provider10.24.0rc1; command execution in an existing Pod, XCom output, and Pod/cluster cleanup were verified.- Fix GKE operator imports with older Kubernetes providers #74505 is now merged. A fresh comparison with Kubernetes provider
10.21.0passed (2 passedin 4.34s): the published Google RC reproduces the module import failure, while the exact merged operator source keeps existing cluster/Pod operators usable and raises the actionable dependency error only when constructingGKEPodExecOperator, preserving the originalImportErroras its cause.
Environment: Breeze, Airflow 3.3.0, Python 3.11.16; Google RC installed from the PyPI wheel. The compatibility fix was checked separately using the merged source; it is not included in
22.7.0rc1and needs validation in a new RC.Reacted by Jarek Potiuktested #73109 in apache-airflow-providers-amazon 9.38.0rc1, works as expected. environment: clean virtualenv on python 3.11 with apache-airflow 3.3.2, installed from pypi. driving the eks delete trigger against a stubbed client yields the status deleted event, and execute_complete now logs the success line for that and for a success event, while an error event logs nothing. the same run on 9.37.0 misses the success event, so that part is this change doing its job.
Reacted by Jarek PotiukI found two more issues for following providers:
- common.ai 0.11.0rc1
- Rework common.ai Strands support into a plugin and add Google ADK #73898: It added
self._create_lock = threading.Lock()toSandboxToolset.__init__. A lock cannot be deep-copied, and Airflow deep-copies tasks anddefault_argsin several places. For example,TaskGroupdeep-copies itsdefault_argswhen it is created. On Airflow 3.3.2, a Dag file like this parses with 0.10.0 but fails with 0.11.0rc1. I create Let SandboxToolset be deep-copied again #74525 for this.
- Rework common.ai Strands support into a plugin and add Google ADK #73898: It added
- amazon 9.38.0rc1
- Bind the AWS auth manager SAML response to the browser that started the login #73698: The new
_awsam_login_statecookie is set withSameSite=Lax. After sign-in, Identity Center sends the SAML response back to/auth/login_callbackwith an auto-submitted form POST from its own site (the access portal on*.awsapps.com). Browsers do not sendSameSite=Laxcookies on cross-site POST requests, so the callback returns 401 "No login in progress for this browser. Start the login from Airflow and try again." I create Fix AWS auth manager login when the IdP is on another site #74529 for this.
- Bind the AWS auth manager SAML response to the browser that started the login #73698: The new
Reacted by Jarek Potiuk- common.ai 0.11.0rc1
Thanks everyone for testing. I ticked #73690, #73109, #74305, #72953 and #73552.
Four providers are excluded from this wave because of regressions:
- google 22.7.0rc1: Add GKEPodExecOperator for existing Pods #72577 breaks GKE operator imports with older
cncf-kubernetes. Fixed by Fix GKE operator imports with older Kubernetes providers #74505 (merged). - dbt.cloud 4.11.0rc1: deferred tasks fail with tenacity
retry_args. Fix in Fix deferred dbt Cloud tasks failing with tenacity retry_args #74489. - common.ai 0.11.0rc1: Rework common.ai Strands support into a plugin and add Google ADK #73898 made
SandboxToolsetimpossible to deep-copy, so Dags that use it indefault_argsor aTaskGroupno longer parse. Fix in Let SandboxToolset be deep-copied again #74525. - amazon 9.38.0rc1: my Bind the AWS auth manager SAML response to the browser that started the login #73698 sets the login-state cookie with
SameSite=Lax. The cross-site SAML POST from IAM Identity Center arrives without it, so every login started from Airflow fails with 401. Fix in Fix AWS auth manager login when the IdP is on another site #74529.
All four will be re-cut in an ad-hoc wave once the fixes are merged. Results already ticked here for unaffected PRs carry over. Thanks @FrankYang0529 for catching the last two and fixing them, and @AlejandroMorgante for #74505.
The other providers in this wave stay in the vote.
Drafted-by: Claude Code (Opus 5.5); reviewed by @potiuk before posting
Reacted by PoAn Yang- google 22.7.0rc1: Add GKEPodExecOperator for existing Pods #72577 breaks GKE operator imports with older
apache-airflow-providers-amazon==9.38.0rc1PR #73870: Cannot be tested deterministically. Unit tests are sufficient.
apache-airflow-providers-snowflake==6.19.0rc1PR #70703: Changes are working as expected with no observed regressions.
PR #73815: Changes are working as expected with no observed regressions.common.ai 0.11.0rc1 has one more regression.
- Return an approved agent answer as
output_typeafter human review #73904: Withenable_hitl_review=True, anAgentOperatortask now converts the approved answer with a pydanticTypeAdapterbuilt fromoutput_type. Following cases have different behavior from previous version. I create Fix agent HITL review for list, marker and function output types #74537 to fix it.output_type=[Verdict, Other](a list of output types): on 0.10.0 the task returns a dict. On 0.11.0rc1 the task fails after the approval withPydanticSchemaGenerationError.output_type=ToolOutput(Verdict)(an output marker): on 0.10.0 the task returns a dict. On 0.11.0rc1 the task fails after the approval withAttributeError.- An output function: on 0.10.0 the function runs once. On 0.11.0rc1 it runs again after the approval, so any side effect in the function happens twice.
- Return an approved agent answer as
Verified #74143 in
apache-airflow-providers-cncf-kubernetes==10.24.0rc1from PyPI, with Airflow 3.3.2 and Python 3.12.14.Checked
job_poll_interval=0.5,60, and the default10through operator deferral, trigger serialization/reconstruction, and the hook's polling loop, for both successful and failed jobs: 6 passed.With provider
10.23.0, the four non-default cases reproduced the bug: the hook requested 10-second sleeps instead. Only the provider version changed; reinstalling the RC restored all six passes.pip checkpassed in each installation.The merge commit is included in the RC tag, and the installed operator, trigger, and hook sources match the PyPI wheel and tag.
Scope: Kubernetes status responses and
asyncio.sleepwere mocked. This checks the installed polling path, without a live cluster or scheduler/triggerer deployment.
Drafted-by: Codex (GPT-6); reviewed by @keemgdeok before posting
Verified #71092 in
apache-airflow-providers-airbyte==6.2.0rc1(PyPI) with Airflow 3.3.2, Python 3.12, against a real self-hosted Airbyte OSS 2.4.0 (abctl, OAuth client credentials), Faker → /dev/null connection, viaAirbyteTriggerSyncOperator/airflow dags test.To simulate a loaded server, a local proxy delayed
POST /jobs(and in one case the token request) by 8s:- no timeout configured →
ReadTimeoutat the 5s default, so default behavior is unchanged timeoutconnection extra (30) → syncs triggered and completed successfully, including when the OAuth token request was also slowAirbyteHook(timeout=...)works and overrides the extra; lower values (2s) are honored- invalid extras (
"abc",0,-5) → clearValueError
With provider 6.1.0, the same
timeoutextra is ignored and the call fails at 5s while Airbyte still creates the job, which is the issue this fixes. Also checked directly against Airbyte without the proxy: a full sync succeeded.- no timeout configured →
I have a kind request for all the contributors to the latest provider distributions release.
Could you please help us to test the RC versions of the providers?
The guidelines on how to test providers can be found in
Verify providers by contributors
Let us know in the comments, whether the issue is addressed.
These are providers that require testing as there were some substantial changes introduced:
Provider airbyte: 6.2.0rc1
Provider akeyless: 0.3.2rc1
Provider amazon: 9.38.0rc1
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Provider anthropic: 1.2.0rc1
AnthropicHook(#73967): @kaxilLinked issues:
Provider apache.beam: 6.3.1rc1
Provider apache.kafka: 2.1.0rc1
Linked issues:
Provider apache.spark: 6.3.3rc1
SparkSubmitHook: Mask_mask_cmdsecrets in linear time (#71711): @divyanshus2404Linked issues:
Provider celery: 3.25.0rc1
Linked issues:
Provider clickhousedb: 1.0.1rc3
Provider cncf.kubernetes: 10.24.0rc1
Linked issues:
Provider common.ai: 0.11.0rc1
max_retriestoAgentSkillsToolset(#74381): @kaxilLinked issues:
include_tracebackto model-backed retry policies (#74308): @kaxilLinked issues:
Linked issues:
capabilitiesto Common AIAgentOperator(#73984): @kaxilLinked issues:
Linked issues:
HookToolsetpin arguments the model must not choose (#73900): @kaxilObjectStorageToolsetfor reading files on object storage (#73899): @kaxilLinked issues:
output_typeafter human review (#73904): @kaxilLinked issues:
common.aitoolsets (#73587): @kaxilModalHookand connection type (#73418): @kaxilLinked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
code_modefromAgentOperatorin favor of theCodeModecapability (#74312): @kaxilexecute_toolthe public methodAirflowToolsetsubclasses implement (#73938): @kaxilLinked issues:
common.sql2.2.0 forcommon.ai's SQL extras (#73867): @kaxilLinked issues:
Linked issues:
common.aitoolset guides (#74379): @kaxilcommon.aisandbox docs with when to use it and where each piece runs (#74297): @kaxilusage_limits=Nonedocstring onAgentOperatorandLLMOperator(#74307): @kaxilLinked issues:
common.aitoolsets (#73902): @kaxilLinked issues:
common.aisidebar (#73837): @kaxilretry_reasonnot just for retries but even when a task fails (#73027): @amoghrajeshProvider common.compat: 1.21.0rc1
Linked issues:
retry_reasonnot just for retries but even when a task fails (#73027): @amoghrajeshProvider common.sql: 2.3.0rc1
Provider databricks: 7.22.0rc1
Linked issues:
Provider dbt.cloud: 4.11.0rc1
Linked issues:
Linked issues:
Provider duckdb: 0.2.1rc1
Provider edge3: 5.0.0rc3
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Provider elasticsearch: 6.9.2rc1
Provider fab: 3.10.1rc1
Linked issues:
Linked issues:
Linked issues:
Provider ftp: 3.16.1rc1
Provider git: 1.0.1rc1
Linked issues:
Linked issues:
Provider google: 22.7.0rc1
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Provider hashicorp: 4.8.3rc1
Provider http: 6.3.0rc1
Linked issues:
Provider ibm.db2: 0.1.0rc1 🎉 New provider
Linked issues:
Linked issues:
Provider ibm.mq: 0.1.0rc1 🎉 New provider
Linked issues:
Linked issues:
Linked issues:
Linked issues:
Provider influxdb: 2.13.0rc1
Linked issues:
Provider keycloak: 0.11.1rc1
Provider microsoft.azure: 15.2.1rc1
Linked issues:
Linked issues:
Linked issues:
Provider modal: 0.1.0rc1 🎉 New provider
ModalHookand connection type (#73418): @kaxilLinked issues:
Provider openai: 2.1.0rc1
Provider openlineage: 2.20.3rc1
Linked issues:
Provider opensearch: 1.14.0rc1
Linked issues:
Linked issues:
Provider sftp: 7.0.0rc1
Linked issues:
Provider slack: 9.11.1rc1
Provider smtp: 3.1.1rc1
Linked issues:
Provider snowflake: 6.19.0rc1
Provider sqlite: 4.3.4rc1
Linked issues:
Provider ssh: 7.0.0rc1
Linked issues:
Provider standard: 1.20.1rc1
Linked issues:
All users involved in the PRs:
@SameerMesiah97 @bugraoz93 @abhinav-phi @atharvaajmera @ColtenOuO @X33834 @abhishekmauryaKsolves @ephraimbuddy @AshleyAHuang @dheerajturaga @haseebmalik18 @vgkowski @namanjain24-sudo @yamakazushi @shivaam @ShubhamKapoor992 @ashb @dabla @kaxil @pankajastro @shahar1 @VladaZakharova @pankajkoti @subhramit @FrankYang0529 @moomindani @Eason09053360 @SEPURI-SAI-KRISHNA @fpiped @Lee-W @sgoel2be24-cyber @r3wretrhy @amoghrajesh @robertpofuk @seanghaeli @MichalJaroslawKrzywanski-TomTom @ucaeon @JelyFishhhhhh @YAshhh29 @ferruzzi @yuseok89 @KarthikMohankumar @Andrushika @zozo123 @bingqin2 @topherinternational @MeghanaGangarapu @drewrukin @SulimanAbdulrazzaq @eladkal @Samin061 @gtxu @Amitkumar293 @kadubhumika @filipeaaoliveira @firasbouzazi @aaron-y-chen @atamagrawal @AlejandroMorgante @jeff3071 @potiuk @radhwene @olegkachur-e @divyanshus2404 @keemgdeok @ccoliu @takayoshi-makabe @o-nikolas @jingi723