Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions cmd/depsnort/d161_zero_coverage_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
package main

// D-161 regression: pointing the tool at a repo whose only dependency manifest
// is unrecognized must not read as a clean pass. Live on swytchdb/swytch.jepsen,
// a project.clj declaring a JDBC driver with three real advisories produced
// "no supported projects found" at exit 0 — with -fail-on-incomplete set. The
// D-59 machinery (recognized manifest → incomplete coverage → exit 3 under the
// gate) already existed; the Clojure names were simply absent from the tables.
// -require-project covers the remaining shape D-59 cannot: a directory with NO
// recognized manifest at all (a deleted or renamed go.mod in a CI job that
// expects one), where the deliberate empty-repo-sweep clean exit is wrong.

import (
"os"
"path/filepath"
"testing"
)

const leinProject = `(defproject swytch.jepsen "0.1.0"
:dependencies [[org.clojure/clojure "1.12.4"]
[org.postgresql/postgresql "42.7.4"]])
`

func TestClojureManifestIsIncompleteCoverageNotCleanPass(t *testing.T) {
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "project.clj"), []byte(leinProject), 0o644); err != nil {
t.Fatal(err)
}
// Without the gate: disclosed, advisory-tier, still exit 0.
if code := run([]string{"scan", "-no-osv", "-no-registry", dir}); code != 0 {
t.Errorf("ungated scan of a gap-only repo: exit = %d, want 0 (disclosure, not a gate)", code)
}
// With the gate: the recognized-but-unread manifest is degraded coverage.
if code := run([]string{"scan", "-no-osv", "-no-registry", "-fail-on-incomplete", dir}); code != 3 {
t.Errorf("-fail-on-incomplete on a project.clj repo: exit = %d, want 3 (zero-coverage repos must not pass the coverage gate)", code)
}
}

func TestRequireProjectFailsOnNothingToScan(t *testing.T) {
empty := t.TempDir()
// The default stays deliberate: a sweep across repos is not failed by an
// empty one, gate or no gate.
if code := run([]string{"scan", "-no-osv", "-no-registry", empty}); code != 0 {
t.Errorf("empty dir: exit = %d, want 0", code)
}
if code := run([]string{"scan", "-no-osv", "-no-registry", "-fail-on-incomplete", empty}); code != 0 {
t.Errorf("empty dir with -fail-on-incomplete: exit = %d, want 0 (nothing was expected here)", code)
}
// -require-project inverts it, in both discovery modes.
if code := run([]string{"scan", "-no-osv", "-no-registry", "-require-project", empty}); code != 3 {
t.Errorf("empty dir with -require-project: exit = %d, want 3", code)
}
if code := run([]string{"scan", "-no-osv", "-no-registry", "-require-project", "-no-recursive", empty}); code != 3 {
t.Errorf("empty dir with -require-project -no-recursive: exit = %d, want 3", code)
}
}

func TestRequireProjectPassesWhenAProjectExists(t *testing.T) {
if code := run([]string{"scan", "-no-osv", "-no-registry", "-require-project",
"../../internal/ecosystem/npm/testdata/emptylock"}); code != 0 {
t.Errorf("-require-project on a real project: exit = %d, want 0", code)
}
}
8 changes: 8 additions & 0 deletions cmd/depsnort/gap.go
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,14 @@ var gapManifestByName = map[string]string{
"pubspec.yaml": "dart",
"Podfile": "cocoapods",
"Package.swift": "swift",
// D-161: the swytchdb live scan walked a Leiningen project whose project.clj
// declared a JDBC driver carrying three real advisories, and exited 0 with
// "nothing to scan" — these two names were simply missing from this table
// while the D-59 machinery for them already existed. .clj is any Clojure
// source and .edn any EDN data, so both are exact-name per the dedication
// rule above.
"project.clj": "leiningen", // Clojure/Leiningen; resolves from Clojars + Maven Central
"deps.edn": "clojure", // Clojure tools.deps
// go.work is deliberately omitted: it is a workspace aggregator whose local
// `use` modules are each scanned on their own, so disclosing the workspace
// file as an unread gap would be a spurious note on an already-covered repo —
Expand Down
4 changes: 4 additions & 0 deletions cmd/depsnort/gap_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,10 @@ func TestClassifyGapManifest(t *testing.T) {
"flake.lock": "nix",
"conan.lock": "conan",
"deno.lock": "deno",
// D-161 — Clojure manifests, missed live on a Leiningen project whose
// JDBC driver carried three real advisories.
"project.clj": "leiningen",
"deps.edn": "clojure",
}
for name, wantEco := range gaps {
if eco, ok := classifyGapManifest(name); !ok || eco != wantEco {
Expand Down
21 changes: 19 additions & 2 deletions cmd/depsnort/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@
// 0 clean, or only advisory findings
// 1 a block-class finding (FLAG) was present
// 2 a gate-eligible finding was present AND --fail-on-eligible was set
// 3 resolution coverage was degraded AND --fail-on-incomplete was set
// 3 resolution coverage was degraded AND --fail-on-incomplete was set,
// or zero projects were discovered AND --require-project was set
// 64 usage error
// 70 internal/operational error
//
Expand Down Expand Up @@ -144,6 +145,9 @@ scan flags:
proof attached — never hidden, never re-gated
-fail-on-incomplete let degraded resolution coverage fail the run (exit 3);
coverage is always REPORTED, this only makes it gate
-require-project let zero discovered projects fail the run (exit 3)
instead of the clean nothing-to-scan pass — for a CI
job pointed at a repo that must contain one
-offline use only the local OSV cache; never touch the network
-no-osv skip the OSV data-source layer entirely
-osv-cache string OSV advisory cache directory
Expand Down Expand Up @@ -1025,6 +1029,7 @@ func cmdScan(args []string) int {
failEligible := fs.Bool("fail-on-eligible", false, "gate-eligible warnings fail the run (exit 2)")
realRoots := fs.String("real-roots", "", "comma-separated substrings naming the roots you actually build/ship; findings no designated root can reach are labeled contained (with proof) — never hidden, never re-gated")
failIncomplete := fs.Bool("fail-on-incomplete", false, "degraded resolution coverage fails the run (exit 3)")
requireProject := fs.Bool("require-project", false, "zero discovered projects fails the run (exit 3) instead of the clean nothing-to-scan pass")
offline := fs.Bool("offline", false, "use only the local OSV cache; never touch the network")
noOSV := fs.Bool("no-osv", false, "skip the OSV data-source layer entirely")
noCargoFetchSrc := fs.Bool("no-cargo-fetch-source", false, "skip fetching build.rs from crates.io for cargo dependencies whose source is not on disk (vendor/ or CARGO_HOME); the unexamined crates stay disclosed as source-unavailable (-offline also stops the fetch)")
Expand Down Expand Up @@ -1121,6 +1126,13 @@ func cmdScan(args []string) int {
// a full-send sweep legitimately crosses repos with no supported
// ecosystem (Go, C, a docs tree). Exit clean with a loud stderr
// note, so a CI gate over many repos is not failed by an empty one.
// -require-project inverts that default for a CI job pointed at a
// repo that MUST contain one: there, a deleted or renamed manifest
// would otherwise be indistinguishable from a clean pass (D-161).
if *requireProject {
fmt.Fprintf(os.Stderr, "depsnort: no supported projects found under %q and -require-project is set — zero coverage is not a clean pass\n", path)
return verdict.ExitIncomplete
}
fmt.Fprintf(os.Stderr, "depsnort: no supported projects found under %q (nothing to scan)\n", path)
return exitClean
}
Expand Down Expand Up @@ -1153,7 +1165,12 @@ func cmdScan(args []string) int {
notes := discoveryCoverageGaps(path, projects, adapters, *noBuildDirs)
if len(projects) == 0 && len(notes) == 0 {
// No supported manifest here and nothing below — genuinely nothing to
// scan, not an internal error. Exit clean.
// scan, not an internal error. Exit clean, unless the operator said
// this path must contain a project (D-161, as above).
if *requireProject {
fmt.Fprintf(os.Stderr, "depsnort: no supported project at %q and -require-project is set — zero coverage is not a clean pass\n", path)
return verdict.ExitIncomplete
}
fmt.Fprintf(os.Stderr, "depsnort: no supported project at %q (nothing to scan)\n", path)
return exitClean
}
Expand Down
41 changes: 41 additions & 0 deletions docs/DECISIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -6686,3 +6686,44 @@ and treating raise arguments as inert text would be wrong for exceptions whose c
effects. And this closes the class for setup.py only; the same displayed-instruction shape in other
grammars (a Rakefile `puts`, a PowerShell `Write-Host`) is unexamined here, disclosed rather than assumed
closed.

## D-161 — zero coverage is not a clean pass: Clojure gap names, and -require-project

**Trigger:** the swytchdb live scan (2026-08-28, four repos). `swytch.jepsen` — a Leiningen project whose
`project.clj` declares `org.postgresql:postgresql@42.7.4`, a JDBC driver carrying three real advisories —
produced `no supported projects found (nothing to scan)` at exit 0, with `-fail-on-incomplete` set. The
run's report framed this as "-fail-on-incomplete passes the zero-coverage case", a silent-failure shape.

**Verification narrowed the diagnosis.** The report's framing was half right. The D-59 machinery for
exactly this — a RECOGNIZED manifest no adapter can resolve is disclosed as incomplete coverage and fails
the coverage gate — already existed and works: a bare `pom.xml` under `-fail-on-incomplete` exits 3 today.
The jepsen miss was two absent table entries, not an absent mechanism: `project.clj` and `deps.edn` were
simply not in `gapManifestByName`, so a Clojure repo fell through to the nothing-to-scan clean exit. What
remained genuinely open was the OTHER zero-coverage shape, which D-59 cannot reach by design: a directory
with NO recognized manifest at all. That clean exit is deliberate (a full-send sweep across many repos must
not fail on an empty or Go-only one), but for a CI job pointed at a repo that MUST contain a project, a
deleted or renamed manifest is indistinguishable from a clean pass.

**The fix, in two parts matching the two shapes.** `project.clj` ("leiningen") and `deps.edn` ("clojure")
join the exact-name gap table — both extensions are general (.clj is any Clojure source, .edn any EDN
data), so name-only per the OPU-18 dedication rule. A jepsen-shaped repo now flows through the normal
pipeline: disclosed as incomplete coverage, a real JSON verdict emitted (previously: no output at all),
exit 3 under `-fail-on-incomplete`. And `-require-project`, opt-in, makes zero discovered projects fail
the run at exit 3 — in both discovery modes — instead of the clean nothing-to-scan pass. The default is
unchanged: sweeps still cross empty repos clean, and `-fail-on-incomplete` alone still passes a genuinely
empty directory, because nothing was expected there.

**Validation:** `d161_zero_coverage_test.go` — the jepsen shape (ungated: disclosed at exit 0; gated:
exit 3), the empty directory (clean by default and under `-fail-on-incomplete`; exit 3 under
`-require-project`, recursive and `-no-recursive` both), and `-require-project` passing on a real project.
Mutation-checked: reverting the two fix files fails all three new tests while the pre-existing suite is
untouched. Live-fired through the built binary on the original repro directories with the same results.
Full suite green (34 packages), `-race` clean on the touched packages, gofmt/vet silent.

Residual limitations: Homebrew formulae (the other unscanned swytchdb repo) are deliberately not added —
a formula is a distribution-integrity surface, not a dependency manifest, and `.rb` is any Ruby source, so
recognizing it would need real formula parsing, not a name-table entry; left as the narrower, speculative
ask the run's report itself judged it. Actually PARSING Maven/Clojars manifests (resolving the declared
dependencies against OSV's strong Maven data) remains ecosystem work of a different size, tracked as a
backlog item, not smuggled in here. And `-require-project` asserts only that at least one project or
recognized gap was discovered — it does not (and should not) judge how many, or which.
Loading