Document per-instance CPU and RAM utilization - #346
Merged
cory-salad merged 6 commits intoSep 21, 2026
Merged
Conversation
SaladCloud reports CPU and RAM utilization for every running container group instance in the Portal and via the API, but nothing in the docs explained what the numbers mean. The two questions that need answering are the ones the metric's design creates: how often it updates, and why it goes over 100%. Both percentages are measured against the vCPU and RAM you requested, not against the node. Nodes vary widely in size, so a percentage of node hardware would mean something different on every machine. Utilization above 100% is expected on GPU workloads. vCPU and RAM are included in the GPU price, so neither is capped: CPU has no cgroup quota at all, and the memory ceiling is raised above the request when the node has spare memory at container start. A GPU workload that asked for 2 vCPUs and can keep 8 cores busy really does report 400%. CPU-only workloads are billed for those resources and the limits are enforced, which is why exceeding RAM there produces exit code 137 instead. The update behavior follows from sampling every 60 seconds: CPU is an interval average rather than an instantaneous reading, the first value needs two samples so a new instance shows 0% for about a minute, and a restart resets the counter and costs one interval. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The (1-16) and (1-60 GB) parentheticals pinned the page to today's maximums, which can go higher. The sentence works without them -- what matters is that the denominator is what you configured, not what the ceiling happens to be. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The paragraph restated what the GPU subsection right below it explains concretely, and framed the over-100% case as general when it only happens on GPU workloads -- the CPU-only subsection says the limits are enforced there. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
State the GPU behavior plainly -- limits are not strictly enforced, so over-100% happens -- and pair it with the consequence that matters: headroom varies by node, so a workload that routinely runs over will fail unexpectedly on some of them. Raise the request instead. CPU-only collapses to the enforced limits and exit code 137. Drop the older-instances subsection entirely; no such instances remain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The over-100%-on-GPU bullet repeated the GPU section verbatim. What is left is the two readings that are not obvious from the sections above -- low CPU on a GPU workload is usually not a problem, low RAM on a CPU-only one is money and scheduling headroom -- so it reads better as a short paragraph than a four-item list. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Pre-existing in the hardware-metrics autoscaling guide, surfaced because CI only spell-checks changed files and this branch touches that page. Every occurrence is a lowercase identifier -- the prometheus_client package, variable names, and the http://prometheus:9090 host -- so capitalizing to satisfy the checker would break the examples. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a docs page for the CPU and RAM utilization we now report per container group instance in the Portal and via the API. There was no coverage of it — the only existing "utilization" docs describe instrumenting your own container with
psutil/nvidia-smi, which is a different thing.New page:
container-engine/explanation/container-groups/resource-utilization.mdx, under Container Groups in the nav.It answers the two questions the metric's design creates:
How often it updates. Nodes sample once every 60s, so CPU is an interval average rather than an instantaneous reading, a new instance shows
CPU 0%for about the first minute (the value is a delta and needs two samples), and a restart resets the counter and costs one interval.Why it goes over 100%. Both percentages are measured against what you requested, not against the node. On GPU container groups the requested vCPU and RAM aren't strictly enforced, so above-100% readings are real and expected. The page says not to rely on it: how much headroom a container gets varies by node, so an application that routinely runs over will work on some nodes and fail unexpectedly on others — raise the request instead. CPU-only workloads have the limits enforced, which is why overage there shows up as exit code 137.
Also adds cross-links from the container groups overview and the hardware-metrics autoscaling guide, which now distinguishes the platform-reported values from self-instrumented ones.
Behavior was taken from the platform implementation rather than inferred.
Note for reviewers
The end-to-end staleness is described as "up to roughly two minutes." The 60s node sampling interval is solid; I hedged on the backend refresh cadence. If someone knows the real number, that sentence is worth tightening.
Separately,
container-groups.mdxstill hardcodesNumber of vCPUs (1-16)andMemory (RAM) (1-60GB). Not touched here, but those will go stale as the maximums rise.🤖 Generated with Claude Code