Repository navigation
Add performance regression CI guardrails - #52
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds dedicated performance-regression CI guardrails for the validated accelerated LDSFL-Meander backends.
The new checks use warmed, repeated, same-runner reference/candidate comparisons rather than absolute whole-solver wall-time thresholds.
No solver equations, production defaults, or scientific/numerical behavior are changed.
What is protected
This PR adds CI protection for:
fastmath=Falseparallel=FalseThe existing Python 3.12 Numba-equivalence job is retained and extended to include recursive SEMIANA tests.
Performance guardrails
The CI benchmark uses:
Current relative performance floors are:
k0123Absolute whole-solver wall time is intentionally not used as a blocking gate because repeated 5,000-step profiling showed substantial host/runtime variability.
New CI workflow
Adds:
.github/workflows/performance.ymlbenchmarks/ci_performance_guardrails.pydocs/performance_regression_ci.mdThe performance workflow runs on:
mainIt uses Ubuntu with Python 3.12 and installs
.[dev,numba].The workflow uploads:
performance_guardrails.jsonas a diagnostic artifact.
Deterministic hardening
Additional tests now protect:
kernel.stats.cache_pathThe cache check was intentionally implemented using the dispatcher cache state rather than
targetoptions, because Numba does not expose thecache=True/Falsesetting reliably throughtargetoptions.Local validation
Focused Numba tests:
Quality checks:
py_compile: passedgit diff --check: passedFinal performance guardrail run:
k0123The SEMIANA and neck guards both reported:
cache_enabled: falseand the run would fail if caching were enabled for those required kernels.
Files changed
.github/workflows/performance.yml.github/workflows/tests.ymlbenchmarks/ci_performance_guardrails.pydocs/performance_regression_ci.mdtests/test_neck_numba.pytests/test_semiana_recursive_numba.pytests/test_vertical_numba.pyNotes