Skip to content

Commit ab968f0

Browse files
docs: require a scenario for every detection rule (#148)
Three of the 16 detection rules shipped with no scenario exercising them: TIMEOUT_NO_HEARTBEAT, FIRMWARE_UPDATE_FAILURE and REPEATED_BOOT_NOTIFICATION. REPEATED_BOOT_NOTIFICATION is the worked example: it landed in #114 and no scenario followed, because the checklist never asked for one. States the invariant at the top of "Contributing Detection Rules" and adds it as step 7 of the checklist, paired with the step 6 false-positive audit. Step 6 confirms a rule stays quiet where it should; step 7 confirms it fires where it should. The three uncovered rules are being filled by #137, #138 and #139. Closes #140
1 parent 450b5be commit ab968f0

1 file changed

Lines changed: 12 additions & 0 deletions

File tree

‎CONTRIBUTING.md‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -261,6 +261,13 @@ trace with expected failure outcomes and optional assertions.
261261

262262
Detection rules identify failure patterns in traces.
263263

264+
**Every detection rule ships with a scenario that exercises it, added in the same
265+
pull request as the rule.** A rule without a scenario has no regression guard, so
266+
nothing in the corpus catches it breaking. This is the invariant that keeps the
267+
scenario corpus honest as rules are added, and it means the scenario count in the
268+
READMEs tracks real coverage rather than volume. See
269+
[How to Add a Scenario](#how-to-add-a-scenario).
270+
264271
### How to Add a Detection Rule
265272

266273
1. Add the failure code to `FailureCode` in `packages/toolkit/src/core/types.ts`.
@@ -290,6 +297,11 @@ Detection rules identify failure patterns in traces.
290297
6. **Audit all existing scenarios** — new rules may trigger on existing fixtures.
291298
Fix false positives or add the new code to `expectedFailures`.
292299

300+
7. **Add a scenario that exercises the new rule**, in this same pull request,
301+
following [How to Add a Scenario](#how-to-add-a-scenario). This is the positive side of
302+
step 6: step 6 confirms the rule stays quiet where it should, and this confirms
303+
it fires where it should. A rule shipped without one is incomplete.
304+
293305
## Contributing Good-First-Issues
294306

295307
Good-first-issues are labeled with `good-first-issue` and are designed for

0 commit comments

Comments
 (0)