Add CMake 4 support and a latest-CMake-4 CI job - #59
Open
Orphis wants to merge 6 commits into
Open
Conversation
- Pin CMP0168 to OLD around every FetchContent_Populate call via the new hfc_fetchcontent_populate() wrapper (hfc_policy_helpers.cmake). Direct population (CMP0168=NEW, the default for consumers with policy version >= 3.30, typical on CMake 4) ignores SUBBUILD_DIR, keeps its stamps in the consumer's build dir and rm -rf's + re-clones the shared source cache whenever those stamps are missing (fresh build dir, cmake --fresh), defeating the central cache and HFC's invalidation logic. - Add HERMETIC_POLICY_VERSION_MINIMUM (per-content, on FetchContent_MakeHermetic) and HERMETIC_FETCHCONTENT_POLICY_VERSION_MINIMUM (global) to set CMAKE_POLICY_VERSION_MINIMUM for legacy dependencies that declare cmake_minimum_required < 3.5, which CMake 4 rejects. Injected as -D on the isolated configure command (the toolchain is read too late for the dependency's top-level cmake_minimum_required), as a set() in the proxy toolchain (covers nested code and keys the cmake-re ABI-hash cache), in the no-target add_subdirectory wrapper, and around the interlocked provider's consumer-scope add_subdirectory. - Replace deprecated make_directory() with file(MAKE_DIRECTORY). - CI: new build-cmake4 job runs the entire suite against the latest stable CMake 4.x, provisioned through the tipi distro system: TIPI_DISTRO_JSON points at a manifest generated by test/generate_cmake4_distro.sh (re-zips the Kitware release; the distro ship step only extracts zip archives). To be simplified once CMake 4 zips are hosted on the distro CDN. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Change-Id: Ib81b43519abe605e03f9267648dd359a858b22ca
check_source_cache_reuse locks the property that a dependency is downloaded exactly once into the central source cache and that new build trees reuse it: after the first build tree populates the cache, the dependency origin is renamed away, so any re-download attempt fails loudly. The consumer opts into cmake_policy(SET CMP0168 NEW) (when available) to model the modern cmake_minimum_required(VERSION 3.30+) consumers that broke before the CMP0168 pin in hfc_policy_helpers.cmake. Scenarios (x native cmake / cmake-re): - warm clean git clone: a fresh build tree configures and builds without the origin, the cached clone's HEAD is untouched, and no clone runs - URL/archive content: same, resting purely on the populate stamps next to the cache (there is no HFC-side git check on this path) - local uncommitted edits in the cached clone survive a new build tree and are what gets compiled (the debugging workflow of a56de21; native cmake only, as cmake-re's install cache is keyed on ORIGIN+REVISION) Validated green with CMake 4.4.0 and 3.31.9 (6 cases each, tipi distro) and bare CMake 3.28.3 (3 native cases) - and red against the pre-pin tree under CMake 4.4, failing in exactly the modes reported in the issue: the URL case re-downloads and the local-modifications case is destroyed by the direct-population re-clone. The clean-git case passes even pre-pin because HFC's git-warm early-return short-circuits before FetchContent_Populate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Change-Id: Ica3a245361c7f808a11ccc2dad5fc42d1336b09f
Run natively on CMP0168 NEW (CMake >= 3.30): no populate sub-build, one nested configure less per dependency. The invariant that makes direct population safe for the shared source cache is that FetchContent_Populate() now only ever runs against a COLD (missing/empty) source dir - warmness is decided by HFC from the cache contents themselves, in the new hfc_populate_cache_state module: - git content: a valid clone with HEAD at the declared GIT_TAG (or at a local git tag of that name) is WARM; local uncommitted modifications are allowed and reused (the a56de21 debugging workflow). A valid clone at another revision is UPDATEd in place (best-effort fetch + force-clean checkout, preferring origin/<tag> so branch GIT_TAGs follow the branch) - never a re-clone. Offline reconfigures keep working from the local clone. - URL content: a completed populate is recorded by a marker file next to the source dir (the legacy sub-build populate-complete stamp is also honored so pre-existing caches stay warm). - anything non-empty that is neither -> invalidated (marker, source dir, and the populate state of BOTH implementations) and re-populated. Applied at all three populate sites: the populate closure, the in-source secondary populate (URL trees always re-extract from the locally cached archive: in cmake-re mode the mirror machinery rearranges the tree behind HFC's back, so no marker can be trusted there; git trees are self-validating), and the interlocked provider (which had no warm check). A COLD populate also clears direct population's per-build-tree step stamps first, so a stale download.stamp cannot skip the download after HFC_V1_REMOVE_SOURCE_DIR_AFTER_INSTALL deleted the cache sources. HERMETIC_FETCHCONTENT_FORCE_SUBBUILD=ON restores the previous CMP0168 OLD sub-build behavior as an escape hatch; on CMake < 3.30 the sub-build remains the only implementation. check_source_cache_reuse gains a branch-tag scenario: a moved branch is followed via in-place fetch+checkout (never a re-clone) and an unreachable origin falls back to the local clone state. Validated on CMake 4.4.0: full suite 31/33 (the two failures are the pre-existing configure_stability autotools cmake-re cases, unchanged from the pristine-main baseline), including the source-migration matrix and all cmake-re variants; check_source_cache_reuse also green on CMake 3.31.9 (8/8) and 3.28.3 (4/4 native). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Change-Id: Id27ae9de9d5a08b6968abc78b72f1ee4e882e434
…dDetails HFC no longer reads FetchContent's private saved-details storage. A guarded FetchContent_Declare() override (installed by HermeticFetchContent, forwarding to the original implementation) records the raw declaration arguments into HFC's own _HermeticFetchContent_<name>_declaredDetails global property, mirroring the documented FETCHCONTENT_BASE_DIR/<name>-(src|build) SOURCE_DIR and BINARY_DIR defaults. FetchContent_MakeHermetic(), the interlocked MakeAvailable path and hfc_determine_cache_id() (which also loses its reliance on a leaked caller variable) now read that copy. __FetchContent_getSavedDetails() is only used as a fallback for contents declared before including HermeticFetchContent, with a clear error when even that is unavailable. Behavioral notes: - relative GIT_REPOSITORY URLs (CMP0150 resolution happens inside FetchContent_Declare) are only supported through the fallback path; hermetic declarations should use absolute origins - the recorded details no longer contain FetchContent's internal tokens (argument separator, injected DOWNLOAD_EXTRACT_TIMESTAMP), so <name>_DETAILS_HASH changes once: existing setups reconfigure their dependencies one time Validated on CMake 4.4.0: full suite 32/33 (only the pre-existing configure_stability autotools cmake-re cases fail, unchanged), migration matrix included; the fallback path was never taken in the validation runs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Change-Id: I9ade04f897eebf1a7b992552daa567d09d971487
The tipi-build/distro branch feature/cmake-4.4.0 now serves distro-ready CMake 4.4.0 zips from the distro CDN, so the build-cmake4 job no longer needs to download the Kitware tarball, re-zip it and synthesize a manifest: it fetches the branch's distro.json (pinned to a commit for immutability), stages the linux CMake zip into the workspace and rewrites that one URL to file:// so the network-isolated offline_source_override_test container - which only sees the workspace mount - provisions the same CMake without network access. test/generate_cmake4_distro.sh is retired. Testing CMake 4 locally is now just: export TIPI_DISTRO_JSON=https://raw.githubusercontent.com/tipi-build/distro/c5d489ccaf51f28fcb35def9e58ec78ab3491169/distro.json export TIPI_DISTRO_JSON_SHA1=eee2d2d8b9c0f40f91cf9f416a33c52ee9257fa3 tipi run <cmake|ctest> ... Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Change-Id: I09a7737271c7265c3a88e047c26f2d18da625b58
The matrix has started failing intermittently on CI only (a couple of native-cmake scenarios per run report STALE), and the log so far carries just the verdict, which is not enough to tell the candidate causes apart. Capture, at the moment of failure: - the migration cycle's configure output (HFC's populate decision: cold populate / warm reuse / update / invalidate, and whether the install and configure markers short-circuited the dependency rebuild) and build output - the source cache on disk, with the variant id each clone actually holds, so "populated the wrong sources" is distinguishable from "right sources, stale binary", plus whether a sub-build dir was configured (i.e. which populate implementation ran) - the direct-population step stamps, which live per consumer build tree and are keyed on the content name only, so they are shared across source-cache keys - a stale download.stamp there would silently skip a re-download - mtimes of the installed library, header and the consumer executable, to separate "dependency never rebuilt/reinstalled" from "consumer never relinked" Fires from both assertion helpers (the CLEAN-expecting migrations and the exact-outcome cases) only when the outcome deviates, so passing runs stay quiet; outputs are tail-bounded to stay readable in CI logs. Verified under CMake 4.4.0: a deliberately flipped expectation produces the full dump, and the unmodified scenarios pass with no output. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Change-Id: If96dc2d63d2355849556d59d2940401dfa221610
Lambourl
requested changes
Aug 21, 2026
Lambourl
left a comment
Lambourl
left a comment
Contributor
There was a problem hiding this comment.
Thanks for this great post. But I think the examples should also be built with CMake 4 (as they already are with CMake 3); that would give us an idea of how they behave with CMake 3 or CMake 4.
Otherwise, the PR looks really solid.
| # only thing this job adds over `build`. | ||
| build-cmake4: | ||
| name: Run HermeticFetchContent tests (latest CMake 4) | ||
| if: ${{ !inputs.docker_image }} |
Contributor
There was a problem hiding this comment.
This should not be overlooked; this change should be tested with every new commit to cmake-re. This would give us information on how cmake-re interacts with CMake 4.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pin CMP0168 to OLD around every FetchContent_Populate call via the new hfc_fetchcontent_populate() wrapper (hfc_policy_helpers.cmake). Direct population (CMP0168=NEW, the default for consumers with policy version
Add HERMETIC_POLICY_VERSION_MINIMUM (per-content, on FetchContent_MakeHermetic) and HERMETIC_FETCHCONTENT_POLICY_VERSION_MINIMUM (global) to set CMAKE_POLICY_VERSION_MINIMUM for legacy dependencies that declare cmake_minimum_required < 3.5, which CMake 4 rejects. Injected as -D on the isolated configure command (the toolchain is read too late for the dependency's top-level cmake_minimum_required), as a set() in the proxy toolchain (covers nested code and keys the cmake-re ABI-hash cache), in the no-target add_subdirectory wrapper, and around the interlocked provider's consumer-scope add_subdirectory.
Replace deprecated make_directory() with file(MAKE_DIRECTORY).
CI: new build-cmake4 job runs the entire suite against the latest stable CMake 4.x, provisioned through the tipi distro system: TIPI_DISTRO_JSON points at a manifest generated by test/generate_cmake4_distro.sh (re-zips the Kitware release; the distro ship step only extracts zip archives). To be simplified once CMake 4 zips are hosted on the distro CDN.
Change-Id: Ib81b43519abe605e03f9267648dd359a858b22ca