Repository navigation
Conversation
|
Note that JuliaLang/llvm-project@60c342f hasn't been ported to the 22.x branch. CC @vchuravy |
|
Added in JuliaLang/llvm-project@c63a088 |
|
Rebased on 22.1.8 and tagged: https://github.com/JuliaLang/llvm-project/releases/tag/julia-22.1.8-0 |
I remember Valentin looked into that, and it wasn't that easy because the surrounding code changed a bit in llvm/llvm-project@99e8aa5 (this was a discussion in |
According to Claude's investigation, the change has been broken for much longer (presumably unnoticed due to lack of TSAN + Asserts coverage). I can try to have it come up with something better, but I'm not familiar with the change so will have to defer to somebody else to review. EDIT: JuliaLang/llvm-project@d86df9c |
|
I have vague memories of this, @xal-0 was looking into sanitizers a while ago. |
|
Dropped the TSAN patch entirely after discussion with @vchuravy. |
|
The slow part in Yggdrasil is checking the shared libraries. Sometimes it takes 5 minutes to check a single shared library. Either the code needs to be optimized, or the file system of the host is overloaded. In that case it would be better to reduce concurrency (e.g. from 16 to 8) instead of increasing the run time. The hour you added gives you 12 more shared libs per build... |
Even the libraries in here? AFAIU the LLVM ones are just exceptionally large, presumably triggering some bad behavior in the auditor. Anyway, I agree this isn't great. I'll take a look. |
Pull request was converted to draft
|
I think a good way forward for this PR is:
|
|
I'm looking into fixing the auditor. This branch will then be a good way to stress-test the fix. |
|
Well, that's quite the improvement: from 10h to 3h for a LLVM bump :-) |
|
Even though audit was massively sped up, and a previous build finished quickly, the current one times out during the actual build. I think this may be due to the |
Stage 1 of adding LLVM 22 (cf. how v21 was split across #12004 and #12223): only the base builds that compile LLVM from source. The extraction builds (LLVM, libLLVM, Clang, LLD, MLIR) and the *_unified bumps depend on LLVM_full_jll v22.1.1+0 via get_addable_spec, which can only resolve once LLVM_full_jll is registered. They follow in a stage-2 PR (prepared on the tb/llvm22-stage2 branch). Built with GCC 13 instead of the usual GCC 10: LLVM 22 is only consumed by the matching Julia release (libLLVM/Clang major version tracks the Julia version), which bundles a GCC>=13 CompilerSupportLibraries (Julia 1.12+ ships CSL v1.5 / GCC 15), so the raised GLIBCXX floor (3.4.32) is satisfied. GCC 13 also matches the mingw-w64 v13 GCCBootstrap shards that downstream consumers (e.g. PoCL) statically link against. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The GCC 13 choice in the previous commit was meant to match the mingw-w64 v13 GCCBootstrap that PoCL (#14048) statically links against, but it is unnecessary: - PoCL uses GCC 13 only on Windows (for `.drectve -exclude-symbols`, to dodge PE export-ordinal limits); elsewhere it uses GCC 10. It already statically links the GCC-10-built LLVM 20.1.2. - The std::__get_once_mutex / __once_functor undefined-symbol break seen when linking LLVM static libs into a GCC-13 build is a toolchain bug (the GCC-13 libstdc++ was rebuilt with TLS enabled but the call_once ABI patch wasn't backported; a rootfs also rebuilt ucrt -> _setjmp), being fixed upstream -- not a reason to rebuild LLVM with GCC 13. GCC 10 matches LLVM 15-21, keeps the GLIBCXX floor low, and stays on the well-tested toolchain, while still satisfying PoCL once the toolchain fix lands (same path as the existing PoCL@20 -> LLVM@20). LLVM 22's host-compiler minimum is GCC 7.4, so GCC 10 is amply supported. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
| v"20.1.8" => "5b9f96366ce26dfc8ca91697ef0a57894791d95e", # julia-20.1.8-0 | ||
| v"21.1.8" => "7cd4442d4a6c949de43c1e2c0e20334ee59aa154", # julia-21.1.8-0 | ||
| v"22.1.1" => "c7b8aa1a7f4f46f79224d3269ce6f25cd75a4549", # julia-release/22.x | ||
| v"22.1.1" => "bb28dd22e7ad95ca869437f5e773603b6561fc9b", # julia-22.1.8-0 |
There was a problem hiding this comment.
Claude points out that it is a bit awkward here that we registered v22.1.8 as v22.1.1. Not sure there's anything we need to do, except point it out in downstream PRs
There was a problem hiding this comment.
Ah oops, I probably bumped it at some point without updating everywhere...
Stage 2 of adding LLVM 22 (cf. how v21 was split across #12004 and #12223, and the stage-1 PR #14053). LLVM_full_jll / LLVM_full_assert_jll v22.1.1+0 are now registered, so this adds the extraction builds that repackage them into the individual JLLs (LLVM, libLLVM, Clang, LLD, MLIR) and bumps the *_unified meta-packages to include v22.1.1+0. These are the JLLs consumed by Julia (libLLVM_jll, LLD_jll, Clang_jll, LLVM_jll), so registering them unblocks the LLVM 22 bump on the Julia side. Recipes mirror their v21 counterparts, only pinned to LLVM_full_jll v22.1.1+0. Note that despite the 22.1.1 version number, these are actually built from LLVM v22.1.8: common.jl maps v"22.1.1" to the fork tag julia-22.1.8-0 (commit bb28dd22), so the JLL version string is 22.1.1 but the packaged LLVM source is 22.1.8. The version number is inherited from LLVM_full_jll v22.1.1+0 as set by the stage-1 recipe. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The stage-1 recipe (#14053) registered LLVM_full_jll and LLVM_full_assert_jll as v22.1.1+0, but the source they package is actually LLVM 22.1.8 (fork tag julia-22.1.8-0, commit bb28dd22). Correct the version number to match the LLVM release actually shipped, so the downstream extraction JLLs (libLLVM, Clang, LLD, LLVM, MLIR) can register at the right version. Same source commit as v22.1.1+0; only the version number changes. This pull request was written with the assistance of generative AI. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Seems like we already had an LLVM 22 branch on https://github.com/JuliaLang/llvm-project/, so let's build a JLL.
I'm also bumping the build to GCC 13; since this JLL is almost exclusively going to be used by Julia sessions on 1.14 or higher, the raised libstdcxx requirement (3.4.28 -> 3.4.32) shouldn't pose a practical concern, while fixing the issue I'm having with linking against LLVM statically in #14048