Skip to content

docs: replace the sweep chart with the run's dashboard - #579

Merged
0xKT merged 1 commit into
mainfrom
docs/showcase_sweep_dashboard
Sep 21, 2026
Merged

0xKT merged 1 commit into
mainfrom
docs/showcase_sweep_dashboard

Conversation

@ZuyiZhou

Copy link
Copy Markdown
Contributor

Summary

The parameter sweep case shipped a two-panel matplotlib plot. The same run
also produced a dashboard, and it carries what the plot could only assert:
the sixteen measured cells as a grid with recall and latency in each, the
best cell ringed at top_k 10 and chunk_size 1024, and a footnote naming the
single warm-up sample behind the 256/top_k=3 outlier. This swaps the plot
for the dashboard and rewrites the alt text to describe that grid rather
than the two lines.

Language. The dashboard was authored in Chinese. Both front pages share
one image set, and the comparison board beside this cell is already English,
as is the alt text on both pages, so what lands is an English render built
the same way that board was. The Chinese original is left untouched on disk
and nothing in the repository refers to it.

Widths. The new render is 2000x1568 against the board's 2000x1547, so
the two sit almost exactly on one aspect. The board goes back to full width
and the dashboard takes 99 percent; the board only needed 86 percent against
the old plot's 2000x1332.

Type

  • Fix
  • Feature
  • Docs
  • CI / tooling
  • Refactor
  • Other

Verification

git diff --check origin/main...HEAD
  clean

COMMIT_RANGE=origin/main...HEAD make check-large-files
  exit 0

PYTHONPATH=. uv run --frozen --python 3.12 --extra dev python scripts/check_source_language.py origin/main...HEAD
  exit 0

make check-commits
  exit 0

PR_TITLE="docs: replace the sweep chart with the run's dashboard" make check-pr-title
  exit 0

git merge-tree --write-tree HEAD origin/main
  clean

The attachment was fetched anonymously before it was referenced: HTTP 200,
2000x1568, md5 fcd434b3000cbcda9339e98c22bf974b, matching the local render
byte for byte.

Rendered the section through GitHub's markdown endpoint under the published
markdown stylesheet and measured every image box. All eight pairs share a
top edge and match within 2px on height; the Frameworks pair is now 379x293
against 375x294, where before this change it was 326x252 against 375x294.

Both front pages were compared after the edit: each carries the identical
set of attachment ids, and neither still references the old plot.

No test suite is relevant to a change that edits two markdown files. The
python lint job was not run locally for the same reason; CI runs it on the
head.

  • Relevant tests pass locally
  • Relevant lint / type checks pass locally
  • User-facing docs or screenshots are updated when needed

Risk

Documentation only, in both READMEs. One attachment reference is replaced
and one width attribute is restored; no file is committed to the repository.
The old plot remains reachable at its own attachment URL, so a revert of the
commit restores it with no other action.

  • Security impact considered
  • Backward compatibility considered
  • Rollback path is clear for risky changes

Related Issues

Follows #555.

The parameter sweep case shipped a two-panel matplotlib plot. The run also
produced a dashboard, and it carries what the plot could only assert: the
sixteen measured cells as a grid with recall and latency in each, the best
cell ringed at top_k 10 and chunk_size 1024, and a footnote naming the one
warm-up sample behind the 256/top_k=3 outlier. Its alt text describes that
grid instead of the two lines.

The dashboard was authored in Chinese. Both front pages share one image and
the comparison board beside it is English, so this is the English render,
built the same way that board was, with the Chinese original left untouched.

At 2000x1568 it sits almost exactly on the board's 2000x1547, so the pair
holds level with the board back at full width and the dashboard at 99
percent, rather than the 86 percent the board needed against the old plot.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

I reviewed the complete github/main...HEAD diff, the surrounding bilingual showcase markup, the history of this section, and the current PR record. I also covered the repository rules in AGENTS.md/CLAUDE.md and CONTEXT-MAP.md, compatibility and consumers, test integrity, and architecture constraints. This is a documentation-only replacement, so there are no runtime callers, backward-compatibility effects, weakened tests, or architecture-boundary changes.

The new GitHub attachment resolves as a 2000x1568 PNG (538,929 bytes). I inspected it directly: its 16-cell grid, broadly flat latency, and highlighted top_k = 10, chunk_size = 1024 best cell agree with the alt text. Its dimensions closely match the neighboring 2000x1547 board, so the 100%/99% sizing keeps the row balanced.

Verification passed:

  • git diff --check github/main...HEAD
  • scripts/check_large_files.py github/main..HEAD
  • scripts/check_source_language.py github/main..HEAD
  • scripts/check_commit_messages.py github/main..HEAD
  • scripts/check_pr_title.py

The repository's GitHub checks were passing except for one unit-test shard still pending when reviewed; the docs build and repository-file checks had passed. The full local test suite was not rerun because the change is limited to external image links and README markup.

@0xKT

0xKT commented Sep 21, 2026

Copy link
Copy Markdown
Member

Not a blocker. Reviewed by hand. The swap is clean and both front pages
change identically -- I diffed the two hunks against each other and they are
byte-for-byte the same, 2 removed and 2 added in each file, with nothing else
touched. All three asset URLs resolve (HTTP 200, image/png), the old plot has no
remaining reference, and the new one is referenced only from the two READMEs.

The width arithmetic in the commit message checks out. Measured from the PNG
headers: the dashboard is 2000x1568, the comparison board 2000x1547, the old
plot 2000x1332. So the old width="86%" on the board is 1332/1547 = 86.1%, and
at 100%/99% the new pair stands level. Exactly as described.

I also opened the image rather than trusting the alt text, and every claim in it
holds: a sixteen-cell grid with recall and latency in each cell, the best cell
ringed at top_k 10 / chunk_size 1024, the 256/top_k=3 outlier footnoted as one
68.4 ms warm-up sample, and the whole render in English.

One thing the image says that its own data does not support:

"BEST CELL ... recall 1.000, latency 44.3 ms (lowest in the grid)". The grid
beside it reads 43.2 ms at chunk_size 128 / top_k=5, which is lower. 44.3 ms is
the lowest among the recall-1.000 cells (46.9 / 45.8 / 46.1 / 44.3), which is
almost certainly what was meant -- the cell CHOICE is right, it is the
parenthetical that overreaches. "lowest at recall 1.000" would be true.

The same subset shows up a second time: the colour scale under the grid runs
"44 ms" to "57 ms", and 43.2 ms is below its low end, so the scale was built
from the same set the "lowest" claim was.

Small, and the substantive conclusions above it are all correct -- recall is set
by top_k, latency is broadly flat, and the ringed cell is the right one to
recommend. But it is the front page, the whole point of the swap is that the
dashboard "carries what the plot could only assert", and a reader who reads the
grid will find the one sentence that does not survive it.

@0xKT
0xKT merged commit b8f6472 into main Sep 21, 2026
21 checks passed
@0xKT
0xKT deleted the docs/showcase_sweep_dashboard branch September 21, 2026 06:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants