Skip to content

Switch official Docker images to Debian Trixie - #71650

Closed
ytoprakc wants to merge 2 commits into
apache:mainfrom
ytoprakc:docker-images-debian-trixie
Closed

ytoprakc wants to merge 2 commits into
apache:mainfrom
ytoprakc:docker-images-debian-trixie

Conversation

@ytoprakc

@ytoprakc ytoprakc commented Aug 15, 2026 •

Copy link
Copy Markdown
Contributor

Switch official Airflow Docker images from Debian Bookworm to Debian Trixie.

closes: #71494

Why

  • Airflow’s Docker image policy says the Debian base should be upgraded at or before 6 months before the previous release reaches end of regular support.
  • Debian Bookworm regular support ended on July 11, 2026.
  • Debian Trixie has been stable since August 2025 and is now the appropriate Debian baseline for newly built Airflow images.
  • Moving to Trixie keeps Airflow aligned with its documented support policy and Debian’s current package/security stream.

What Changed

  • Updated the default Docker base image:

    • debian:bookworm-slim -> debian:trixie-slim
    • applies to both production and CI image builds
    • deprecated Bookworm-based custom image builds remain available during the Airflow 3.4.x transition period
  • Updated build tooling and CI defaults:

    • Breeze now defaults to Trixie
    • GitHub Actions image-build inputs now default to Trixie, while still allowing Bookworm explicitly
    • release-management image build paths now use Trixie
    • generated Breeze command-output docs were refreshed
  • Updated Debian package mappings for Trixie:

    • libldap-2.5-0 -> libldap2
    • libssl3 -> libssl3t64
    • lzma-dev is kept only for Bookworm builds
    • moved lcov installation until after Airflow’s custom Python build
  • Made system-Python detection version-independent:

    • previously the diagnostic path assumed Bookworm’s system Python package naming
    • now it detects any installed libpython* package
    • this keeps the guard useful across Debian releases and Python versions
  • Updated MariaDB client installation:

    • Trixie builds use MariaDB 11.8
    • Bookworm builds continue to use MariaDB 10.11
    • added mariadb-client-compat where needed for MariaDB 11.x
  • Updated MSSQL client installation:

    • refreshed Microsoft apt signing key
    • keeps msodbcsql18 installation working with the Trixie base
  • Fixed CI SSH startup logic:

    • previously waited for exactly 3 SSH host keys
    • now waits for any available SSH host key
    • avoids Trixie/OpenSSH-related CI timeouts where SSH was usable but exposed a different number of host key types
  • Updated Java example/e2e images:

    • use Temurin 17 JRE instead of Debian openjdk-17-jre-headless
    • needed because Trixie no longer ships that package
    • keeps Spark examples on Java 17 rather than moving them to Java 21, which Spark 3.5.x does not support
  • Updated Amazon provider optional python3-saml dependency floor:

    • xmlsec>=1.3.14 -> xmlsec>=1.3.16
    • Airflow’s lowest-dependencies CI selected xmlsec 1.3.14, and that version fails to build from source with Debian Trixie’s GCC 14
  • Updated IBM DB2 provider dependency floor:

    • ibm-db>=3.2.0 -> ibm-db>=3.2.8
    • Airflow’s all-version lowest-dependencies CI selected ibm-db 3.2.0, which fails to build from source on newer Python/Trixie CI images
  • Updated documentation:

    • Docker stack docs
    • installation/support docs
    • security model platform reference
    • Docker Compose docs
    • Docker customization examples
    • PyPI README
    • spelling wordlist
    • release newsfragment

Was generative AI tooling used to co-author this PR?
  • Yes — OpenAI Codex

Generated-by: OpenAI Codex following the guidelines

@ytoprakc
ytoprakc force-pushed the docker-images-debian-trixie branch 11 times, most recently from 40d611b to 1cbf29e Compare August 19, 2026 20:51
@ytoprakc
ytoprakc force-pushed the docker-images-debian-trixie branch 2 times, most recently from 03f53e2 to 1ea65b6 Compare August 22, 2026 15:53
@ytoprakc
ytoprakc marked this pull request as ready for review August 22, 2026 17:00
@ytoprakc
ytoprakc force-pushed the docker-images-debian-trixie branch 2 times, most recently from 8ac5c84 to 19a6c5c Compare August 22, 2026 17:20
Comment thread airflow-e2e-tests/docker/Dockerfile.java
@ytoprakc
ytoprakc force-pushed the docker-images-debian-trixie branch from 19a6c5c to f49f4ce Compare September 2, 2026 20:07
@potiuk potiuk added all versions If set, the CI build will be forced to use all versions of Python/K8S/DBs full tests needed We need to run full set of tests for this PR to merge and removed backport-to-v3-3-test labels Sep 8, 2026
@potiuk
potiuk force-pushed the docker-images-debian-trixie branch from f49f4ce to 412bc8e Compare September 8, 2026 22:07

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the labels to make sure all tests and all versions of python are run. However there is one more important change needed - if you look at past changes to docker base image we supported both - previous and new verison of theb base debian - for until next minor version (see the changelog) - this should be done similarly

@ytoprakc
ytoprakc force-pushed the docker-images-debian-trixie branch 2 times, most recently from 3d16f9b to 09f1b11 Compare September 9, 2026 10:26
Keeping Bookworm selectable gives custom image users a transition period while Trixie becomes the default. The expanded all-version lowest-dependencies CI also needs dependency floors that resolve to installable wheels on supported Python versions.
@ytoprakc
ytoprakc force-pushed the docker-images-debian-trixie branch from 09f1b11 to b61871f Compare September 9, 2026 13:01
@ytoprakc

ytoprakc commented Sep 9, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @potiuk. I updated the PR to follow the same transition pattern used in the earlier Debian base-image migrations.

Trixie is still the default for newly built CI and production images, but Bookworm remains explicitly selectable for custom image builds during the Airflow 3.4.x line. I also updated the Docker image changelog to call this out and to make clear that Bookworm-based custom image build support is expected to be removed in Airflow 3.5.0.

@potiuk

potiuk commented Sep 16, 2026

Copy link
Copy Markdown
Member

Thank you for this — and I want to be clear up front that closing it is not a judgement on the work. I went through it properly before deciding, and it does what I asked for in my last review: the dual-base transition follows the shape of the previous Debian migrations, ALLOWED_DEBIAN_VERSIONS keeps Bookworm selectable, the changelog documents both the transition and the 3.5.0 removal, and there are tests covering both bases. I also verified the Microsoft signing-key rotation against the vendor rather than taking it on trust — the key you added matches https://packages.microsoft.com/keys/microsoft-2025.asc exactly (AA86F75E427A19DD33346403EE4D7792F748182B), and Microsoft's own key README gives the reason, which is precisely our reason: the old BE1229CF key carries SHA1 signatures that newer Linux releases no longer accept. That was the highest-consequence line in the diff and it was right.

Why I'm closing it anyway

The base-image story has moved underneath this PR. We are switching the CI and PROD images to Docker Hardened Images first, and only then revisiting the Debian release. That work is in #73038, with the background in #59625.

That changes the ground this PR is standing on. Once the images are based on a hardened Python image, we no longer compile CPython on top of debian:bookworm-slim at all — install_python(), the tarball download and the sigstore/GPG verification all go away, along with the fifteen apt packages that existed only to build CPython. Several of those are exactly the packages this PR is carefully making conditional per Debian release. So the bookworm/trixie branching added here to Dockerfile, Dockerfile.ci and install_os_dependencies.sh would have to be rewritten on top of a different base rather than merged and then adapted, and the two changes would conflict heavily in the meantime.

Doing hardened first and Debian second is the order that costs the least total work — but it does mean this PR would be largely redone rather than rebased, which is not a fair thing to ask of you.

What is worth keeping, and is independent of all of the above

Several pieces here are not coupled to the base-image question at all and are useful on their own. If you would like to carry any of them forward as separate, much smaller PRs, they would be very welcome:

  • The Microsoft signing-key rotation to the 2025 key. Needed regardless of which base we end up on, for the SHA1 reason above. One request if you do: add a comment next to the key ID in scripts/ci/prek/update_docker_gpg_keys.py citing https://packages.microsoft.com/keys/microsoft-2025.asc and the full fingerprint. That hook fetches keys by ID from public keyservers, which accept uploads from anyone with any UID — so the hardcoded ID is the actual trust root and cannot validate itself. Recording where it came from makes it re-verifiable in seconds.
  • The two provider dependency floors — xmlsec>=1.3.16 in providers/amazon and ibm-db>=3.2.8 in providers/ibm/db2. Both are justified on their own terms and have nothing to do with the Debian base. Worth noting if you split them out: provider changelogs are regenerated from git log at release time rather than from newsfragments, so name the bumps in the commit message or they will be missed during release prep.
  • The MariaDB 10.11 → 11.8 selection, which will be needed by whichever base ships a newer Debian.
  • The openjdk-17 fix in the extending example. One question there rather than a request: Trixie ships openjdk-21-jre-headless, so is eclipse-temurin needed because the pinned Spark provider requires 17 specifically? If 21 works, staying on the distro package keeps our canonical example Debian-only; if 17 is genuinely required, a line in the example saying so would help, since readers copy these verbatim.

Sorry to close a PR you have already revised once in response to my review. If you would like to pick the Debian work back up after the hardened images land, please do — and feel free to follow #73038 to see when that is.


Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting

@potiuk potiuk closed this Sep 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

all versions If set, the CI build will be forced to use all versions of Python/K8S/DBs area:dev-tools area:production-image Production image improvements and fixes full tests needed We need to run full set of tests for this PR to merge kind:documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Upgrading docker images debian from version 12(bookworm) to 13(trixie).

3 participants