Skip to content

Commit 01928f3

Browse files
committed
Write the benchmark history tool in C# rather than Python [patch]
The tool that reads BenchmarkDotNet's reports and draws the chart was Python, which fits none of these repositories: there is no other .py file in PreciseNumber, Semantics or SignificantNumber, and what scripting they do have is PowerShell. It was chosen because it was quick to write and needs nothing installed on a runner, which is a reason about writing it rather than about living with it — and it was about to be copied into two more repositories. It is now a .NET 10 file-based app, run as dotnet run scripts/benchmark-history.cs -- ingest ... so it needs no project, no runtime beyond the SDK that already builds the library, and no second language in the workflow. A third subcommand, baseline, replaces the inline Python that read one number back out of a report. The build covers it, which is the part that could not be had before: the first compile failed on CA1305, CA1502, CA1505, CA1506, IL2026, IL3050 and SYSLIB1045. Most are fixed rather than suppressed — cultures are explicit, the regex is source-generated, and the work moved out of top-level statements into a class so the analyzers judge each method rather than one 500-line Main. The port is verified by output rather than by reading: re-ingesting the same reports and re-rendering leaves both the history and the two SVGs byte for byte identical to what Python produced. Getting there caught one real difference — parameters were coming out in ordinal order, which reads 200, 30, 8, where BenchmarkDotNet declares them 8, 30, 200 and the Digits axis is meant to be read across in order. Also drops the .gitignore entries added with the workflow. ktsu.Sdk regenerates that file on every build, so they did not survive one — which is what quietly ate them twice while this was being written. The scratch output goes to BenchmarkDotNet.Artifacts/ instead, already ignored, and worktrees are created under RUNNER_TEMP, outside the repository where the commit step cannot see them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017jrnV7N94UGL8fDRRE8Xt8
1 parent 671f1d4 commit 01928f3

3 files changed

Lines changed: 615 additions & 481 deletions

File tree

‎.github/workflows/benchmark-history.yml‎

Lines changed: 14 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,8 @@ concurrency:
3939
env:
4040
DOTNET_VERSION: "10.0"
4141
HISTORY: docs/benchmarks/history.json
42+
# Already ignored, and ktsu.Sdk regenerates .gitignore on build so a new entry would not last.
43+
RUNS: BenchmarkDotNet.Artifacts
4244
CHART: docs/benchmarks/performance.svg
4345
# The set drawn in the README. Name globs rather than a [BenchmarkCategory], because the backfill
4446
# runs this same filter against older checkouts that predate any attribute added today.
@@ -87,15 +89,8 @@ jobs:
8789
dotnet run -c Release --project PreciseNumber.Benchmarks -- \
8890
--filter '*BaselineBenchmarks.ReferenceWork' \
8991
--job "$BENCHMARK_JOB" \
90-
--artifacts "$GITHUB_WORKSPACE/bench/baseline"
91-
ns=$(python3 - <<'PY'
92-
import glob, json
93-
path = glob.glob("bench/baseline/results/*BaselineBenchmarks-report-full.json")[0]
94-
with open(path, encoding="utf-8-sig") as handle:
95-
document = json.load(handle)
96-
print(round(document["Benchmarks"][0]["Statistics"]["Mean"], 4))
97-
PY
98-
)
92+
--artifacts "$GITHUB_WORKSPACE/$RUNS/baseline"
93+
ns=$(dotnet run scripts/benchmark-history.cs -- baseline --results "$RUNS/baseline")
9994
echo "Reference workload: $ns ns"
10095
echo "ns=$ns" >> "$GITHUB_OUTPUT"
10196
@@ -107,17 +102,17 @@ jobs:
107102
run: |
108103
set -euo pipefail
109104
version="${VERSION#v}"
110-
work="$GITHUB_WORKSPACE/backfill/$version"
105+
work="${RUNNER_TEMP}/bench-$version"
111106
git worktree add --detach "$work" "$VERSION"
112107
113108
(cd "$work" && dotnet run -c Release --project PreciseNumber.Benchmarks -- \
114109
--filter $HEADLINE_FILTER \
115110
--job "$BENCHMARK_JOB" \
116-
--artifacts "$GITHUB_WORKSPACE/bench/$version")
111+
--artifacts "$GITHUB_WORKSPACE/$RUNS/$version")
117112
118-
python3 scripts/benchmark_history.py ingest \
113+
dotnet run scripts/benchmark-history.cs -- ingest \
119114
--history "$HISTORY" \
120-
--results "bench/$version" \
115+
--results "$RUNS/$version" \
121116
--version "$version" \
122117
--commit "$(git rev-parse --short "$VERSION^{commit}")" \
123118
--date "$(git log -1 --format=%cs "$VERSION")" \
@@ -141,7 +136,7 @@ jobs:
141136
for index in "${!refs[@]}"; do
142137
ref="${refs[$index]}"
143138
label="${labels[$index]:-${ref#v}}"
144-
work="$GITHUB_WORKSPACE/backfill/$label"
139+
work="${RUNNER_TEMP}/bench-$label"
145140
146141
echo "::group::$label ($ref)"
147142
rm -rf "$work"
@@ -159,11 +154,11 @@ jobs:
159154
(cd "$work" && dotnet run -c Release --project PreciseNumber.Benchmarks -- \
160155
--filter $HEADLINE_FILTER \
161156
--job "$BENCHMARK_JOB" \
162-
--artifacts "$GITHUB_WORKSPACE/bench/$label")
157+
--artifacts "$GITHUB_WORKSPACE/$RUNS/$label")
163158
164-
python3 scripts/benchmark_history.py ingest \
159+
dotnet run scripts/benchmark-history.cs -- ingest \
165160
--history "$HISTORY" \
166-
--results "bench/$label" \
161+
--results "$RUNS/$label" \
167162
--version "$label" \
168163
--commit "$(git rev-parse --short "$ref^{commit}")" \
169164
--date "$(git log -1 --format=%cs "$ref")" \
@@ -176,7 +171,7 @@ jobs:
176171
177172
- name: Redraw the chart
178173
shell: bash
179-
run: python3 scripts/benchmark_history.py render --history "$HISTORY" --out "$CHART"
174+
run: dotnet run scripts/benchmark-history.cs -- render --history "$HISTORY" --out "$CHART"
180175

181176
- name: Commit the history and the chart
182177
shell: bash
@@ -204,6 +199,6 @@ jobs:
204199
uses: actions/upload-artifact@v7
205200
with:
206201
name: benchmark-history-${{ github.run_id }}
207-
path: bench/
202+
path: ${{ env.RUNS }}/
208203
retention-days: 30
209204
if-no-files-found: warn

0 commit comments

Comments
 (0)