Repository navigation
[release-8.0] TLog: reuse disk-space samples in the low-disk admission check - #14345
Open
saintstack wants to merge 1 commit into
Open
saintstack wants to merge 1 commit into
saintstack wants to merge 1 commit into
Conversation
…ackport) Backport of apple#14216. canTLogAcceptNewData runs for every version a remote TLog pulls and probed both disk stores each time: getStorageBytes() is a realpath() of the parent directory plus a statvfs() whose XFS implementation sums per-CPU counters. On 448-CPU hosts that was about 28% of a remote TLog's CPU at peak load (statfs 13%, readlink 15%), even with TLOG_MIN_AVAILABLE_SPACE_RATIO at its default of 0, where the result is not used. The check now returns before probing when the ratio disables it, and otherwise reuses a sample for up to TLOG_DISK_SPACE_CHECK_INTERVAL seconds (default 1.0, the low-disk retry loop's polling period); 0 probes every time. (cherry picked from commit c442deb)
saintstack
requested review from
akankshamahajan15 and
tclinkenbeard-oai
and removed request for
tclinkenbeard-oai
October 22, 2026 17:58
Contributor
Result of foundationdb-pr-macos-m1 on macOS 14.x
|
Contributor
Result of foundationdb-pr-clang on Linux RHEL 9
|
Contributor
Result of foundationdb-pr-clang-arm on Linux RHEL 9
|
Contributor
Result of foundationdb-pr on Linux RHEL 9
|
Contributor
Result of foundationdb-pr-macos on macOS 14.x
|
Contributor
Result of foundationdb-pr-cluster-tests on Linux RHEL 9
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport of #14216 to release-8.0. Clean
cherry-pick -xof c442deb; no conflicts. Same 3 files, +20/−2.canTLogAcceptNewDataruns for every version a remote TLog pulls, and it probed both disk stores each time: arealpath()of the parent directory plus astatvfs(). On XFS thatstatvfs()sums per-CPU counters, so it costs more on hosts with many CPUs. The probes ran even withTLOG_MIN_AVAILABLE_SPACE_RATIOat its default of 0, where the result is not used. The check now returns before probing when the ratio disables it. Otherwise it reuses a sample for up toTLOG_DISK_SPACE_CHECK_INTERVALseconds (default 1.0; 0 probes every time). See #14216 for the full description.Effect on 8.0.0 in a two-region deployment
Ran the same workload twice on a production-scale two-region cluster: once on stock 8.0.0, then on 8.0.0 plus this change. The workload was a sustained closed-loop write load, then a whole-machine exclude under load, then the re-include. The binary we deployed was the 8.0.0 tag plus this same cherry-pick, line-identical to this branch.
On stock 8.0.0 the remote-region TLogs spent so long probing the disk that spilling fell behind. Unspilled data piled up in memory, and three remote TLogs ran out of memory during the exclude. The primary region was unaffected. With this change:
canTLogAcceptNewData/statvfs)Primary-region TLogs were unchanged (CPU ~74% in both runs).
Caveats: