Repository navigation
chore(scripts): add tinyanalyzer-driven dependency audit across core and vendor submodules #6353
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
6c86beb
c841338
64f7993
25f5394
b559b68
1fd2b8d
d9ca782
712a9db
0d3cbbf
6cd7549
ba5436c
39bc7b5
f1da47f
ad43951
1f57451
7a9c056
51f2de3
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -58,6 +58,7 @@ | |
| "rust:check": "pnpm --filter openhuman-app rust:check", | ||
| "rust:clippy": "cargo clippy -p openhuman -- -D warnings && pnpm --filter openhuman-app rust:clippy", | ||
| "rust:layout": "node scripts/ci/check-openhuman-rust-layout.mjs", | ||
| "dep:audit": "bash scripts/dep-audit/run.sh", | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Reject symlinked lockfiles before restoring them The command snapshots lockfiles with [RULE] symlink-lockfile · |
||
| "agent:runtime-boundary": "node scripts/ci/check-agent-runtime-boundary.mjs", | ||
| "typecheck": "pnpm --filter openhuman-app compile", | ||
| "tauri:ios:init": "bash scripts/ios-init.sh", | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,149 @@ | ||
| # `scripts/dep-audit/` — Cargo dependency audit | ||
|
|
||
| Finds dependencies we can drop, unify, or slim across OpenHuman **and every | ||
| Cargo submodule under `vendor/`**, using | ||
| [`tinyanalyzer`](https://github.com/tinyhumansai/tinyanalyzer). | ||
|
|
||
| ```bash | ||
| pnpm dep:audit # full sweep -> target/dep-audit/REPORT.md | ||
| pnpm dep:audit --snapshot # ...and archive it as docs/dep-audit/<date>.md | ||
| pnpm dep:audit --targets '^(root|tinyagents)$' --top 25 | ||
| ``` | ||
|
|
||
| A full sweep of 24 targets takes about 20 seconds; nothing is compiled, the | ||
| tool only runs `cargo metadata` and parses source. | ||
|
|
||
| The run has no side effects on the tree: `cargo metadata` rewrites a | ||
| `Cargo.lock` that is stale relative to its manifest (`crates/openhuman-app`'s | ||
| lockfile in particular), so `run.sh` snapshots every target's lockfile before | ||
| analyzing it and restores it afterwards, printing which ones it had to put | ||
| back. Refresh those deliberately if you want them refreshed. | ||
|
senamakel marked this conversation as resolved.
|
||
|
|
||
| ## Files | ||
|
|
||
| | File | Role | | ||
| | --- | --- | | ||
| | `run.sh` | Discovers targets, runs `tinyanalyzer` once per target, then calls `report.mjs`. `--help` lists the flags. | | ||
| | `report.mjs` | Folds the per-target JSON into `REPORT.md` and `summary.json`. Re-runnable on its own: `node scripts/dep-audit/report.mjs --reports target/dep-audit`. | | ||
| | `tinyanalyzer.toml` | Shared analyzer config passed to every target (`--config`). Holds the `ignore_unused` list; see below before editing it. | | ||
| | `../../docs/dep-audit/<date>.md` | Committed snapshots from `--snapshot` runs, for diffing against the next run. | | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - `tinyanalyzer` on `PATH` (`run.sh` prints the install one-liner if missing). | ||
| - Submodules checked out: `git submodule update --init --recursive vendor/`. | ||
| - Node 20+ (for `report.mjs`), `git`, `grep`. | ||
|
|
||
| ## What gets analyzed | ||
|
|
||
| `run.sh` builds the target list itself, so a new submodule is picked up | ||
| automatically: | ||
|
|
||
| 1. `root` — the OpenHuman workspace (`Cargo.toml` at the repo root). | ||
| 2. `openhuman-app` — the Tauri host. It is `exclude`d from the root | ||
| workspace and has its own `Cargo.lock`, so it is a separate graph. | ||
| 3. Every entry of `git submodule status --recursive` that has a `Cargo.toml`, | ||
| named after its directory (`tinyagents`, `tinybus`, `tinycortex`, …). | ||
|
|
||
| Nested checkouts (`vendor/tinymcp/vendor/tinybus`) are **skipped when a | ||
| checkout of the same repo at the same commit was already analyzed**, and | ||
| otherwise get a path-derived suffix (`tinybus@vendor_tinybox_vendor_tinybus`) | ||
| so both pins show up. A suffixed row in the report therefore *is* a finding: | ||
| that submodule pins a different commit of a shared dependency than its | ||
| siblings. `--keep-nested` analyzes every checkout regardless. | ||
|
|
||
| ## Reading `REPORT.md` | ||
|
|
||
| ### Targets | ||
|
|
||
| One row per target with its commit and headline counts. "Crates in graph" is | ||
|
senamakel marked this conversation as resolved.
|
||
| the *full* `cargo metadata` resolve — every platform and every optional | ||
| feature — which is why Windows-only crates appear on a Linux run and why a | ||
| duplicate listed here may not show in `cargo tree` on your host. | ||
|
|
||
| ### 1. Declared dependencies no source file names | ||
|
|
||
| tinyanalyzer's check is textual: a dependency is "unused" if no `.rs` file in | ||
| the package mentions the crate. That misses crate names inside attributes | ||
| (`#[tokio::test]`, `#[derive(thiserror::Error)]`), so `report.mjs` re-checks | ||
| every flag with a grep over the package's own sources, **including | ||
| `[[test]]` / `[[example]]` targets declared by `path =` in its `Cargo.toml`** | ||
| (the root crate keeps its integration tests in `tests/` that way). Verdicts: | ||
|
|
||
| | Verdict | Meaning | Action | | ||
| | --- | --- | --- | | ||
| | **remove** | No `crate::…`, `use crate`, `#[crate…` or `crate!` anywhere. | Delete the line, `cargo check` (both feature-on and feature-off builds if it was `optional`), delete the `dep:` feature if one existed. | | ||
| | **remove** (name only) | The bare word occurs in a comment or string but never as a path. | Same as above; the mention is not a use. | | ||
| | keep (attribute/macro path) | Used through an attribute or macro body. | Nothing. Listed so the tool's false positives stay visible. | | ||
|
|
||
| **Graph win** is the number of crates that leave the target's build if that | ||
| one line is deleted. It is `0 (kept by …)` when another package in the same | ||
| workspace still depends on the crate: the manifest gets cleaner, the build | ||
| does not get smaller. Sort your effort by graph win. | ||
|
|
||
| A crate that is used *only* through attributes everywhere (currently | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Scope the thiserror suppression to verified false positives The shared configuration globally suppresses [RULE] overbroad-unused-suppression · |
||
| `thiserror`) can be added to `ignore_unused` in `tinyanalyzer.toml` so it stops | ||
| being reported. Do that only for crates that can never be a real finding; | ||
| every entry hides the crate from the check in all 24 targets. | ||
|
|
||
| ### 2. Crates resolved at more than one version | ||
|
|
||
| Each version is compiled and linked separately. **Only the `root` (and | ||
| `openhuman-app`) sections cost the shipped build**; submodule sections show | ||
| where a requirement should move so the root can unify. | ||
|
|
||
| "Pulled in via" names the *direct* dependencies whose subtree carries that | ||
| version, walked from tinyanalyzer's edge list. `direct dep of <pkg>` means one | ||
| of our own packages declares it. To unify: | ||
|
|
||
| - If one version's "via" list is a single old crate we control (a submodule | ||
| or a direct dep with a stale requirement), bump it. | ||
| - If the old version is only reachable through an *unused* direct dependency | ||
| (section 1), removing that dependency removes the duplicate too. | ||
| - If both versions are reached through third-party crates we do not control, | ||
| check whether one of them has a newer release; otherwise accept it. | ||
|
|
||
| `cargo tree -i <crate>@<version>` in that target gives the full chain. | ||
|
|
||
| ### 3. Heaviest direct dependencies | ||
|
|
||
| Per target, the direct dependencies with the largest **exclusive** transitive | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Exclude development dependencies from shipped-build cost claims The analyzer configuration explicitly includes development dependencies, but this section presents every direct dependency's exclusive footprint as build cost without distinguishing [RULE] development-dependency-cost · |
||
| footprint — crates that would leave the build entirely if this one were | ||
| dropped. "Reaches" is the raw transitive count, most of which something else | ||
| pulls in anyway. "Source" is checked-out source size, not binary size. | ||
|
|
||
| A high exclusive count usually means default features pulling in a subtree we | ||
| do not use. Try `default-features = false` plus the two or three features | ||
| needed, then re-run the audit and compare the row. `scripts/dep-sim.py` and | ||
| `scripts/assert-shed.sh` remain the tools for *proving* a reduction before | ||
| claiming it in a PR. | ||
|
|
||
| ### 4. Version drift across repositories | ||
|
|
||
| Crates that two or more targets depend on directly but at | ||
| **semver-incompatible** versions (different major, or different minor for | ||
| `0.x`). Because the root workspace `[patch]`-es submodules in, every such row | ||
| becomes a duplicate in section 2 of `root`. Aligning the submodule's | ||
| requirement with the root's is usually a one-line change in that submodule | ||
| and removes a duplicate for free. Patch-level drift is counted but not | ||
| listed; cargo unifies it. | ||
|
|
||
| ## Workflow for a clean-up pass | ||
|
|
||
| 1. `pnpm dep:audit --snapshot` on a fresh branch. | ||
| 2. Work section 1 by graph win, then section 4, then section 2's `root` | ||
| rows. Submodule changes go to that submodule's own upstream as their own | ||
| PR; bump the gitlink here afterwards (see the submodule PR conventions in | ||
| `AGENTS.md`). | ||
| 3. Re-run without `--snapshot` and diff `target/dep-audit/REPORT.md` against | ||
| the committed snapshot. Commit a new snapshot with the clean-up PR. | ||
|
|
||
| ## Extending | ||
|
|
||
| - `summary.json` next to `REPORT.md` has the same data as structured JSON | ||
| (`targets[].unused`, `.duplicates`, `.heavy`, and top-level `drift`) for a | ||
| future CI gate, e.g. failing on any new `remove` row with a graph win. | ||
| - Per-target `<name>.json` files are the raw tinyanalyzer reports and also | ||
| carry file, complexity and dead-code findings that this report ignores. | ||
| `tinyanalyzer vendor/<name>` opens the interactive dashboard over the same | ||
| data. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Resolve renamed dependencies before auditing imports
This command invokes the audit that scans source imports using the manifest dependency name rather than Cargo's resolved package/crate name. A dependency such as
serde1 = { package = "serde" }can therefore be reported as unused even though the source correctly importsserde, encouraging deletion of a required dependency and breaking the target build. Resolve each dependency'spackagerename before textual scanning.[RULE] renamed-dependency-detection ·