Optional native git backend for read operations on large repositories

[FireHook]

Context

Follow-up to #736 and #182, both of which reported that version resolution dominates the configuration phase on large repositories.

#736 proposed three fixes. Only the first landed:

• ✅ an override flag for isClean → overriddenIsClean (ScmPosition: added isClean #543)
• ❌ "alternatively make the isClean-property to be lazily evaluated, so it's only computed if it is needed"
• ❌ "retrieving the tags takes some time because it iterates the whole log ... maybe it is enough to partial-compute the data, only for the relevant tags"

I hit the same wall and implemented the two remaining suggestions, plus an optional native-git read path. #1074 covers the lazy isClean part. This issue is about the rest, since it adds public DSL surface and I would rather agree on the approach before opening a ~1000 line PR.

Measurements

Repository: 1 GB pack, 169,177 commits, 745 tags, ~200 Gradle modules.

Measuring inside the real build was useless — the configuration phase swings between 50s and 104s regardless of backend, because the Android plugins dominate. So I benchmarked in an isolated single-module project pointing at the same .git, warm daemon, alternating runs. That is reproducible to within 30 ms.

	JGit	native git
main today	30.6s	—
with #1074	24.5s	—
with #1074 + native backend	—	19.0s

Cost of the underlying commands on that repository:

git rev-list HEAD              0.657s
git for-each-ref refs/tags     0.038s
git status --porcelain         1.599s

Note that tags are already packed here, so the git gc / packed-refs workaround from #736 does not apply — the walk itself is the cost.

Proposal: hybrid backend, opt-in

scmVersion {
    repository {
        backend.set("nativeGit")   // default: "jgit"
    }
}

plus -Prelease.scmBackend=nativeGit. Default stays jgit, so no behaviour change unless opted in.

ScmRepositoryFactory picks the implementation; a new NativeGitRepository implements the read half of ScmRepository and delegates every write to the existing JGit GitRepository.

Why hybrid rather than a full native rewrite

A full rewrite would also have to reimplement push/fetch/commit and the remote authentication that TransportConfigCallback / ScmIdentity does today (ssh keys, in-memory PGP, token auth). Replacing that with ambient credential and ssh config would be a real, security-sensitive behaviour change. Since all of the measured cost is in reads, delegating writes keeps release tagging and pushing bit-for-bit unchanged.

Implementation notes

• Configuration cache. Reads go through ProviderFactory.exec, not ProcessBuilder — raw process execution during configuration is a hard configuration-cache failure ("external process started ...").
• Bounded tag walk. This is Retrieving the version for large repositories is slow #736's "partial-compute" suggestion. providers.exec cannot stream, so the walk reads a bounded chunk (rev-list --max-count=2000) and only falls back to the full history if that chunk yields no match and more history exists. Exact, not approximate. On this repository the nearest tag is 103 commits from HEAD, and the bounded call costs 0.042s versus 0.660s.
• Semantics preserved. Nearest reachable tagged commit, annotated tags peeled via %(*objectname), for-each-ref ref-name ordering matching tagList(), HEAD when detached, overriddenBranchName / GITHUB_HEAD_REF / overriddenIsClean / releaseBranchNames all honoured, :(exclude) pathspecs for excludeSubFolders.
• Hermetic. GIT_CONFIG_GLOBAL / GIT_CONFIG_SYSTEM are neutralised when ignoreGlobalGitConfig is set, mirroring SystemReaderWithoutSystemConfig. GIT_OPTIONAL_LOCKS=0 avoids taking index.lock (measured: no cost).
• Requires git on PATH when enabled.

Alternatives considered

• Caching GitRepository results — discussed at length in Negative impact on configuration phase performance #182. It would help both backends and cut more than this does: resolving one version currently issues 70 git operations, including 14 working-tree scans and 8 tag walks. But it changes staleness semantics after release creates a tag, and Negative impact on configuration phase performance #182 notes the plugin is applied per-module with different tag prefixes, which complicates the cache key. Happy to explore this instead if you prefer.
• overriddenIsClean already avoids the status cost, but it is a correctness trade-off users must opt into per-project; Skip the working tree scan when uncommitted changes are ignored #1074 makes it unnecessary in the default case.

Honest caveats

• This only pays off on large repositories. Resolving a version spawns roughly a dozen git processes, so on small repositories the native backend is a wash or marginally slower. That is why it is opt-in.
• The win only materialises on configuration-cache misses. On a cache hit axion does not run at all.
• The remaining hotspot after both changes is currentPosition() recomputing isClean (10 of the 14 scans). Fixing that means making ScmPosition.isClean lazy, which touches a user-visible @Input — deliberately out of scope here.

Status

Implemented and green: NativeGitRepositoryTest mirrors the read-path scenarios of GitRepositoryTest and additionally asserts output parity with the JGit backend on the same fixture repository, plus integration tests for the DSL flag, the Gradle property and an unknown backend value.

Would you accept a PR along these lines? I am happy to change the DSL naming (backend vs something else), make it a Gradle-property-only escape hatch rather than DSL, or drop it in favour of the caching approach from #182.

[bgalek]

Hi! Part of axion-release-plugin v2 is to move out from jgit, or even skip git entiraly in some cases (like CI runners)
here is the discussion: #751

[FireHook]

Thanks, that's useful context and the v2 direction makes sense. Some numbers from our repo (1 GB pack, 169k commits, 745 tags) that may be worth having in #751, since two of them cut against parts of the plan.

What actually costs time:

git rev-list HEAD              0.657s
git for-each-ref refs/tags     0.038s
git status --porcelain         1.599s

On ".git file parsing instead of full-blown-git integration": tag refs are indeed cheap to parse (packed-refs, 0.038s). But finding the nearest reachable tag means walking the commit graph, and that's the expensive half. Native git only manages 0.66s because it reads commit-graph; with core.commitGraph=false the same walk takes 1.94s. A hand-rolled parser would have to implement commit-graph reading too, otherwise it risks being slower than JGit rather than faster.

On "skip git entirely on CI runners": that helps CI, but our pain is local developer builds — a 200-module Android repo pays this on every ./gradlew invocation, and CI env vars don't cover that path.

On timing: #751 has been quiet since mid-2024 and this hurts us today. Two questions:

1. Would you take a v1 stopgap at all, or would you rather not grow v1 given v2's direction? Happy either way — if it's the latter I'll close this and move the measurements into axion-release-plugin@v2 #751.
2. If a stopgap is acceptable, I can drop the repository.backend DSL property entirely and gate it on -Prelease.scmBackend=nativeGit only, so the public API doesn't grow at all.

Either way #1074 already landed the cheapest part of this (30.6s → 24.5s here).