Repository navigation
CI: retry sbt runs that failed on a dependency download, cache the pinned snapshots - #112
Closed
MateuszKubuszok wants to merge 1 commit into
Closed
MateuszKubuszok wants to merge 1 commit into
MateuszKubuszok wants to merge 1 commit into
Conversation
…nned snapshots
Coursier does not retry a download that is cut off partway ("download error: Caught
java.io.IOException (Premature EOF)"). Between 2026-10-14 and 10-16, 5 ssg jobs failed that way
on the 8 MB balticporter-engine_3 snapshot jar while the project definition loaded; each of the
~20 jobs of a run downloads it afresh. In the same window one generate job got 404s from
repo1.maven.org for 18 long-published poms and one JVM job a 502 fetching the sbt launcher's
jars. All 7 were download failures, not build or test failures; a rerun passed.
- every sbt run (sbt-guarded, the generator command, format, compile, the tree-sitter update,
the re-scale install, the release) goes through .github/scripts/sbt-download-retry.sh: when the
command failed, its output names a download failure and no test ran and nothing was published,
it deletes coursier's recorded download errors and retries up to 3 times after 20s, 60s, 120s.
sbt-guarded no longer reports such a run as a crashed test process.
- setup-scala caches only the hand-pinned snapshot repository directory, per OS, keyed on
project/plugins.sbt and project/Versions.scala (about 15 MB per entry, inside the quota that
keeps the general dependency cache off).
- the Codecov upload runs only when the tests passed, so a failed JVM job shows one error.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Member
Author
|
Merged via signed rebase onto |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cause
Coursier does not retry a download that is cut off partway (
download error: Caught java.io.IOException (Premature EOF)). Between 2026-10-14 and 10-16, 5 ssg jobs failed this way on the 8 MBbalticporter-engine_3snapshot jar while the project definition loaded. Each of the ~20 jobs in a run downloads that jar fresh. In the same window, one generate job got 404s from repo1.maven.org for 18 poms that were published long ago, and one JVM job got a 502 while fetching the sbt launcher's jars. All 7 were download failures, not build failures, and a rerun passed.Changes
.github/scripts/sbt-download-retry.shnow runs every sbt command:sbt-guarded, the generator command ingenerated-port, format, compile, the tree-sitterupdate, the re-scale install and the release. A run is retried only when all of these hold: the command failed, its output matchesPremature EOF|Error fetching artifacts|Error downloading|ResolveException|could not retrieve sbt, and no test ran and nothing was published. Before each retry it deletes coursier's.*.errorfiles, then waits 20s, 60s and 120s, printing "dependency download failed, retrying (n/3)". When every retry fails it reports a download failure;sbt-guardedno longer reports it as a crashed test process. A real test failure is never retried.setup-scalacaches only…/central.sonatype.com/repository/maven-snapshotsunder the Linux, macOS and Windows coursier caches. The key is per OS and hashesproject/plugins.sbtandproject/Versions.scala. That is about 15 MB per entry, well inside the quota that keeps the general dependency cache off.success()instead ofalways(), so a failed JVM job shows one error instead of two.run:blocks were checked with shellcheck.How to verify
actionlint .github/workflows/*.ymlandshellcheck .github/scripts/sbt-download-retry.sh.setup-scalastep shows acoursier-snapshots-<OS>-…cache miss on the first run and a hit on later runs.verifyLocalpassed on this commit.