Skip to content

Commit f01083c

Browse files
martin-s-aclaude
andauthored
Give the sdist check a build directory its cache can key on (#152)
`Verify the source distribution` hits **19 of 334** compilations. The same figure every run, never converging, with the cache restoring correctly and every call cacheable. ## Why [#136](#136) diagnosed it and left it: scikit-build-core builds in a unique temporary directory that has no fixed relation to the one pip unpacked the archive into. Two independent random path components, so the sources sit somewhere different relative to the build directory every run and `CCACHE_BASEDIR` has nothing stable to rewrite against. The 19 hits are the dependencies that happen to live inside the build directory — 12 of Boost and 7 of spdlog. What #136 did not reach for is scikit-build-core's own `build-dir`, whose default is documented as "a unique temporary directory" and which the docs say gives "major rebuild speedups" when made persistent. ## Why relative, specifically The value has to be relative, and that is the whole of the fix. Measured against scikit-build-core 1.0.3, the version this project pins: ``` (default) CMAKE_BINARY_DIR = /tmp/tmpzolfmuz5/build SKBUILD_BUILD_DIR=build CMAKE_BINARY_DIR = <source>/build ``` It is resolved against the source directory, which puts the working directory inside the source tree at a fixed depth. Every absolute path then rewrites to the same relative string from one run to the next — the sources to `../…`, the fetched dependencies to `../../fetched/…` — because the random element is a single directory name at a known position. An **absolute** build directory would fix nothing: the sources would still be at a random path. ## Cost and effect The job takes 17m06s, the second-longest in the deploy run, about two minutes short of the critical path the Windows AMD64 wheels hold at 18m58s. So this buys billed minutes today rather than wall clock — the reason to do it now is the headroom before it becomes the job everything waits on. It arrives as an environment variable on the step, which is the mechanism [#134](#134) already chose for this job's cache settings and its CMake defines, so `pip install dist/*.tar.gz` stays the command a user would run. Nothing about what is compiled, with which flags, or into what layout changes — only where the intermediates are written — so it cannot mask the packaging defect the job exists to catch. ## Measuring it Not on this run. New paths mean new keys, so the run that adopts this misses everything and populates; the **second** deploy after merging is the one whose hit rate means anything. Same pattern as the coverage flag in rexlib#460. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
1 parent 80d33c6 commit f01083c

2 files changed

Lines changed: 9 additions & 0 deletions

File tree

‎.github/workflows/deploy.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -343,6 +343,7 @@ jobs:
343343
- name: Install
344344
env:
345345
CMAKE_BUILD_PARALLEL_LEVEL: 4
346+
SKBUILD_BUILD_DIR: build
346347
SKBUILD_CMAKE_DEFINE: FETCHCONTENT_BASE_DIR=${{ steps.fetched.outputs.dir }};CMAKE_C_COMPILER_LAUNCHER=ccache;CMAKE_CXX_COMPILER_LAUNCHER=ccache
347348
run: pip install --verbose dist/*.tar.gz
348349

‎AGENTS.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -389,5 +389,13 @@ The source distribution names `external/rexlib/**` explicitly, because
389389
`git ls-files` does not report a submodule's contents and the archive would
390390
otherwise carry an empty directory and fail to build.
391391
392+
The job that installs that distribution sets `SKBUILD_BUILD_DIR`, which is
393+
what lets its compiler cache work. scikit-build-core otherwise builds in a
394+
temporary directory of its own, unrelated to the one pip unpacked the archive
395+
into, so the sources sit at a different place relative to the build directory
396+
on every run and `CCACHE_BASEDIR` has nothing stable to rewrite against. The
397+
value has to stay relative — scikit-build-core resolves it against the source
398+
directory, and an absolute one would leave the sources where they were.
399+
392400
Publishing to PyPI is wired up but disabled. Every push to `main` updates the
393401
`development` pre-release with its binaries.

0 commit comments

Comments
 (0)