Skip to content

Conversation

@crazy-max
Copy link
Member

@crazy-max crazy-max commented Jun 10, 2025

fixes #209
fixes #200
fixes #83

Huge refactor by using Bake matrix strategy to reduce quite a lot code duplication. Everything has been moved to the Bake definition at root level. Same for Makefiles that have been removed as well as the helpers. Just the main Makefile is kept but not necessary. You just need Bake.

Following these changes, the ci workflows have changed. We now have two levels for distributing the workload. One job for each distro (as before) and also one for each platform. I also took the opportunity to runs builds on best-matching runner. This is now way faster and builds every distros and supported platforms for each package (even on PR) to make sure there is no regression. Before we would need to have minimal list of distros and platforms to build to reduce build time (almost 1h).

I didn't touch yet the release workflow but will do as follow-up. This PR is already huge.

follow-ups:

  • align and refactor release workflow
  • setup apt mirrors related to use bake matrix strategy #210 (comment)
  • check if we can mutualize ./pkg/*/scripts scripts
  • enable registry cache export
  • ci workflow permissions
  • use ADD to clone pkg to have material part of the provenance

@crazy-max crazy-max force-pushed the bake-matrix branch 15 times, most recently from 56f28f3 to 0dc74e0 Compare June 10, 2025 21:44
@crazy-max crazy-max requested review from neersighted and vvoland June 10, 2025 22:20
@crazy-max crazy-max marked this pull request as ready for review June 10, 2025 22:21
Copy link
Collaborator

@vvoland vvoland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall

distro: distro,
platform: platform
platform: platform,
verify: platform.startsWith('linux/386') || platform.startsWith('linux/amd64') || platform.startsWith('linux/arm64')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not verify all linux/?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would segfault through emulation atm, I can add a TODO comment.

@crazy-max
Copy link
Member Author

Debian repos not happy https://github.com/docker/packaging/actions/runs/15607567172/job/43960593852?pr=210#step:7:2061

 > [linux/arm/v7 builder-deb 6/8] RUN mk-build-deps -t "xx-apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends -y" -i /root/package/debian/control:
4.117 Get:4 http://deb.debian.org/debian bullseye/main armhf build-essential armhf 12.9 [7704 B]
4.125 Get:5 http://deb.debian.org/debian-security bullseye-security/main armhf libglib2.0-0 armhf 2.66.8-1+deb11u6 [1213 kB]
4.126 Get:6 http://deb.debian.org/debian bullseye/main armhf pkg-config armhf 0.29.2-1 [62.4 kB]
4.130 Get:7 http://deb.debian.org/debian bullseye/main armhf libseccomp-dev armhf 2.5.1-1+deb11u1 [87.0 kB]
4.134 E: Failed to fetch http://deb.debian.org/debian/pool/main/g/gcc-10/libstdc%2b%2b-10-dev_10.2.1-6_armhf.deb  Error reading from server - read (104: Connection reset by peer) [IP: 151.101.22.132 80]
4.134 E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
4.134 Fetched 8616 kB in 0s (71.6 MB/s)
4.207 (Reading database ... 17073 files and directories currently installed.)
4.207 Removing containerd.io-build-deps (1.6.16-1) ...
4.319 mk-build-deps: Unable to install all build-dep packages

We might need some retry logic in such case

Copy link

@CosmicJesterX CosmicJesterX left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tried this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use bake matrix Avoid overriding package-specific bake defaults Can't install docker-credential-pass on CentOS 9

3 participants