@@ -519,6 +519,125 @@ API without holding its signing key. The standard ``airflow dag-processor`` comm
519519not yet use this client. Provisioning a token file alone does not switch that command to
520520HTTP or remove its need for database access.
521521
522+ In-memory sessions from workload identity
523+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
524+
525+ Processor integrations can use ``WorkloadIdentitySessionTokenProvider `` to obtain and renew
526+ ``dag_processor_session `` credentials through ``POST /execution/dag-processor/session-token ``.
527+ The provider retains these credentials only in memory. It requires a renewable, signed workload
528+ identity from the deployment platform; Airflow does not create that identity or require the
529+ processor to hold an Airflow signing key.
530+
531+ Configure the API server's ``[execution_api] dag_processor_workload_identity `` option with JSON::
532+
533+ {
534+ "issuer": "https://identity.example/",
535+ "jwks_url": "https://identity.example/openid/v1/jwks",
536+ "audience": "airflow-dag-processor-bootstrap",
537+ "algorithm": "RS256",
538+ "instance_id_claim": ["kubernetes.io", "pod", "uid"],
539+ "grants": {
540+ "system:serviceaccount:airflow:processor": ["dags-folder"]
541+ }
542+ }
543+
544+ Use your identity provider's exact issuer, HTTPS JWKS URL, and signing algorithm (``RS256 ``,
545+ ``ES256 ``, or ``EdDSA ``). Use a dedicated audience, distinct from ``[execution_api] jwt_audience ``.
546+ The API server verifies the signature, issuer, audience, issued-at time, expiry, and any not-before
547+ claim, using ``[api_auth] jwt_leeway `` for clock skew. Only exact subjects listed in ``grants `` can
548+ obtain a session, and every granted bundle must be configured on the API server. This option is
549+ empty by default, disabling the exchange.
550+ Apply the same trust configuration to all API-server replicas and restart them after changes.
551+
552+ The JWKS endpoint must accept unauthenticated HTTPS GET requests from the Airflow API server.
553+ The fetcher sends no authentication credentials and uses HTTPX's default certificate trust,
554+ including ``SSL_CERT_FILE `` or ``SSL_CERT_DIR `` when configured. These environment settings affect
555+ other HTTPX clients in the API-server process; there is no per-issuer CA or authentication option.
556+ Authenticated JWKS endpoints, including Kubernetes API-server discovery protected by service-account
557+ authentication, are not supported by this implementation. For those clusters, serve the public JWKS
558+ through a suitable HTTPS endpoint before enabling the exchange. See Kubernetes'
559+ `service account issuer discovery documentation <https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#service-account-issuer-discovery >`_.
560+ The processor-to-Airflow TLS settings described below do not configure this outbound JWKS connection.
561+
562+ ``instance_id_claim `` is a path through signed claims to a nonempty string identifying one
563+ processor replica. A shared service-account subject alone is insufficient: replicas must have
564+ different instance identities. Run one processor per identity. The session ID stays stable across
565+ identity-token renewal and API-server restarts. Changes to the issuer, audience, subject, instance
566+ claim, instance value, or bundle grants produce a different session. Request bodies and caller
567+ hostnames cannot select the session or its grants.
568+
569+ Changing grants while API-server replicas serve different configurations can repeatedly retire
570+ processor registrations, because those replicas issue different session IDs for the same identity.
571+ Stop the affected processors, finish deploying identical trust configuration to every API-server
572+ replica, and then restart the processors.
573+
574+ For Kubernetes, the example uses the Pod UID in a
575+ `bound service-account token <https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/#bound-service-account-token-volume-mechanism >`_.
576+ Project a token with the dedicated audience and mount the projection directory read-only in the
577+ processor container. Configure the projection's ``expirationSeconds `` and allow Kubernetes to
578+ rotate it. See `serviceAccountToken projected volumes
579+ <https://kubernetes.io/docs/concepts/storage/projected-volumes/#serviceaccounttoken> `_.
580+
581+ A container restart within the same Pod keeps the Pod UID and therefore the session ID.
582+ A replacement process cannot register while the previous Job is still alive. The client raises
583+ ``DagProcessorJobAlreadyRunning ``; the integration must retry registration with backoff until the
584+ previous Job completes or its heartbeat becomes stale under ``[dag_processor] health_check_threshold ``.
585+ The client does not wait automatically. This prevents two processes sharing an identity from owning
586+ active Jobs at the same time.
587+
588+ The processor integration supplies a callable that reads the current platform identity on each
589+ exchange. For example:
590+
591+ .. code-block :: python
592+
593+ from pathlib import Path
594+
595+ from airflow.dag_processing.api_client import DagProcessorAPIClient
596+ from airflow.dag_processing.credentials import WorkloadIdentitySessionTokenProvider
597+
598+ identity_path = Path(" /var/run/airflow-identity/token" )
599+ provider = WorkloadIdentitySessionTokenProvider(
600+ base_url = " https://airflow.example/execution/" ,
601+ identity_token = identity_path.read_text,
602+ )
603+ processor = DagProcessorAPIClient(
604+ base_url = " https://airflow.example/execution/" ,
605+ token_provider = provider,
606+ hostname = " processor-1" ,
607+ )
608+
609+ Use this client for the integration's Job lifecycle and close it during shutdown; closing it also
610+ closes the provider and clears its session cache. The file in this example contains the platform's
611+ projected identity, not an Airflow session credential. The provider never writes the exchanged
612+ session JWT to disk and does not fall back to a session-token file. ``token_provider `` and
613+ ``token_file `` are mutually exclusive. HTTPS is required for the exchange, and redirects are rejected.
614+ The exchange uses the same API TLS configuration and request timeout as the Task SDK client,
615+ including private certificate authorities and client certificates. Registration retries transient
616+ exchange failures using the configured Execution API retry policy; optional early renewal makes
617+ one bounded attempt so it does not delay requests using a valid Job credential.
618+
619+ Session expiry is capped by both the external identity's remaining lifetime and
620+ ``[execution_api] jwt_expiration_time ``. Clock-skew leeway never extends this cap, and an identity
621+ with less than one second remaining cannot obtain a session. Renewal rereads the platform identity
622+ and retains the same session and Job. During an identity-provider or API outage, the client can continue using
623+ an already-issued Job credential until it expires; it cannot renew without a valid identity.
624+ Removing a grant stops future exchanges after API-server configuration is reloaded. Credentials
625+ already issued remain subject to their expiry and Job ownership checks.
626+
627+ Verification uses the issuer's JWKS and does not perform online token introspection. In particular,
628+ it does not check whether a Kubernetes Pod still exists. A previously issued identity may remain
629+ usable until expiry. The JWKS cache refreshes hourly and also refreshes when an identity names an
630+ unknown signing key. Fetches, including failed attempts, are limited to one per ten seconds per
631+ API-server process. Publish new keys before signing with them and retain old keys while their tokens
632+ remain valid.
633+
634+ If a required signing key cannot be fetched, the exchange returns HTTP 503 with ``Retry-After ``,
635+ allowing registration to retry. A cached matching key remains usable during an outage. Invalid
636+ identities and keys absent from a successfully fetched keyset return HTTP 401.
637+
638+ File-based provisioning
639+ ~~~~~~~~~~~~~~~~~~~~~~~
640+
522641On a trusted host with the API signing key, provision a session token for the bundles
523642the client may access::
524643
@@ -532,9 +651,12 @@ token replacement stays visible. Each processor process needs its own session. R
532651provisioner with the same token file and bundles keeps that session, so running processors keep
533652their Jobs.
534653
654+ Credential permissions and renewal
655+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
656+
535657The three credentials have different issuers and permissions:
536658
537- - ``dag_processor_session `` is issued by trusted provisioning. It can only register a
659+ - ``dag_processor_session `` is issued by the identity exchange or trusted file provisioning. It can only register a
538660 Job or renew that Job's credential through ``POST /jobs ``.
539661- ``dag_processor `` is returned by Job registration. It can heartbeat and complete that
540662 Job, read Connections and Variables for a granted bundle, and exchange a parsing
@@ -625,6 +747,9 @@ All JWT-related configuration parameters:
625747 * - ``[execution_api] jwt_audience ``
626748 - ``urn:airflow.apache.org:task ``
627749 - Audience claim for Execution API tokens.
750+ * - ``[execution_api] dag_processor_workload_identity ``
751+ - Empty (disabled)
752+ - JSON trust settings and subject-to-bundle grants for workload identity session provisioning.
628753
629754.. important ::
630755
0 commit comments