Repository navigation
Fetch the dependencies once a job rather than once a build - #138
Merged
Merged
Conversation
Seven archives come down from github, gitlab and sourceforge on every configure, and FetchContent puts them inside the build directory, which scikit-build-core makes fresh each time. A wheel job builds nine interpreters, so it fetches the same sources nine times over: about 380 downloads across a deploy, each one a chance for someone else's outage to end the run. Two did this week, gitlab with a timeout and sourceforge with a 504. A base directory outside the build tree is populated once and reused. Verified locally: two build trees share one without complaint, and with the sources already there a configure completes with no network at all. This leaves one fetch per job. Carrying the directory between runs would leave none, and wants a cache of its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
runner.temp is C:\a\_temp on Windows, and passing that through a bash step turns the backslashes into escapes: the value arrived as the relative a_temp/fetched and the sources landed inside the checkout, where pytest walked into pybind11's own suite and failed collecting it. An absolute path with forward slashes survives every shell. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Seven archives come down from github, gitlab and sourceforge on every
configure.
FetchContentputs them inside the build directory, andscikit-build-core makes that fresh every time, so a wheel job building
nine interpreters fetches the same sources nine times over — around 380
downloads across a deploy, each one a chance for someone else's outage to
end the run. Two did this week: gitlab timed out after 136 seconds
fetching Eigen, and sourceforge returned a 504 for half.
FETCHCONTENT_BASE_DIRputs them outside the build tree, where they arepopulated once and reused.
Checked locally
Two build trees pointed at one base directory configure without
complaint, the second reusing what the first fetched. And with the
sources already present, a configure run inside a namespace with no
network at all completes — where the same configure without them fails
with fourteen errors. So the network really does leave the path, rather
than merely being hit less often.
The seven cover every dependency, so there is no list of names to keep in
step with the CMake.
What this does not do
It leaves one fetch per job. Carrying the directory between runs would
leave none, and that wants a cache of its own, keyed and saved like the
compiler caches are — worth doing separately, once this shows what it is
worth on its own.
🤖 Generated with Claude Code