fix: keep AAS extremes honest across rollup grains (#210) - #220
Draft
NikolayS wants to merge 2 commits into
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #210 — the largest blocker from the pre-tag audit (#160).
ash.top()andash.compare()report hour-grain numbers in columns named
peak_aas/p99_aas, socompare()fabricatesregressions on byte-identical load.
Draft on purpose. This is the riskiest change in the release: it alters a user-visible contract
on eight readers and changes
ash.compare()'s return signature. Needs REV and your approval.The rule implemented
NULL
peak_aas/p99_aaswhenever the read grain is coarser than the requested bucket.Not an arbitrary pick —
ash.timeline()already follows exactly this rule for filteredrollup_1hreads, with the shipped comment "a percentile over hour averages would masquerade as a minute
percentile".
aasandtopwere violating an invariant this same file writes down two functionsaway. This makes the file self-consistent.
top()no longer emits hour averages underpeak_aas/p99_aas.avg_aas,backend_seconds,pctstay valid.aas()follows the same rule when a filter forces hour grain.compare()gainssource_1/source_2, and suppresses all four peak/p99 values when the two windows resolve to different grains. Average deltas remain valid.rollup_1h.minute_countsis stored per(ts, datid), so the minute detail already existed and simply wasn't read. Now expanded throughrollup_1h_minutes, keeping true minute-grain peak/p99.AAS_API.md§2.4's justification ("the per-dimension arrays are stored per hour") was false for this dimension and is corrected.minute_counts IS NULL, ~98% of a mature 1.x→2.0 archive) now reportsource = 'rollup_1h_flat'instead of masquerading as measurement.On B1d's marker choice:
rollup_1h_flatwas chosen overdata_points = 0because those hoursstill hold usable average and backend-second evidence — zero data points would incorrectly signal
no coverage. That reasoning is sound and I'd keep it, but it's a contract decision worth your
confirmation.
Verification
Tests were committed before the implementation (
98e4c2c test:thene135d22 fix:) — the TDDordering is visible in the history, not just claimed.
Deterministic fixture: 59 minutes at AAS 1 plus one minute at AAS 10 → 4,140 backend-seconds,
avg 1.15, peak 10.00, p99 4.69. Every expected number is hand-computed.
RED against unpatched SQL, showing the actual wrong values:
GREEN: all eight blocks pass, including cross-reader agreement —
aas,timeline,top,summaryandreportnow assert to the same 1.15 / 10.00 / 4.69 / 4140.00 for one fixed window.That agreement is the whole point of the fix, so it is asserted directly.
Also verified:
compare()'s new signature identical on both pathsI separately confirmed the riskiest mechanic by hand: a real
v2.0-beta1install upgraded withthis branch's installer ends up with
source_1 text/source_2 textpresent andcompare()callable.
CREATE OR REPLACEcannot change a return type, so this depends on the installer'sdynamic drop block — it works, but that is the thing to re-check if anything here is rebased.
Both already collide with each other; this is a third party. Do not resolve any of it with
-X ours/-X theirs.vs #131 — overlaps
aas3885–3896,timeline4042–4063,top4470–4489, catalog5752–5764(all four function comments), and workflow3837–3992. Its chart hunk5519–5560isseparable. A human must rebuild its partial-hour policy around the new requested/read/effective
grain logic, preserve the database minute-detail exception, and keep now-relative and default
calls working. Its unconditional alignment guard must not be carried over — that is what breaks
zero-arg
ash.summary().vs #150 — overlaps
_grain_counts~3578,aas3800–3867,timeline3950–4032,_grain_by4179–4359,top4376–4507, catalog5752–5764, blueprint36–45,138,174–204,381–400, workflow3712–3871. Order matters: in_grain_by, keep this branch's earlydatabase-minute expansion then apply #150's query-attribution branch; in
top, apply #150'sexact-query decision first, then this branch's source / legacy-flat / extreme-validity logic.
compare()'s new signature belongs here, but its delegated reads must inherit #150's query policyafter the hand merge.
🤖 Generated with Claude Code
https://claude.ai/code/session_014HzBGzjFyN8dXZHbdWmYBj