Skip to content

Commit fe1fa00

Browse files
fix(node): rebase quarantine PR onto main and close visibility/CID gaps
Merge current main (IPFS walk budget/admission) with quarantine fixes: canonical-only CID fold, authorize_repo_read on visibility mutations, mirror-only CID HTTP regression for list_all_repos SQL filter, and owner PUT/DELETE quarantine denies on the visibility route. Co-authored-by: Cursor <cursoragent@cursor.com>
2 parents 0cc91df + 225644a commit fe1fa00

63 files changed

Lines changed: 27634 additions & 1037 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.env.example‎

Lines changed: 127 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -24,8 +24,11 @@ DATABASE_URL=postgresql://gitlawb:changeme@localhost:5432/gitlawb
2424
# ── Database pool & startup resilience ────────────────────────────────────
2525
# Maximum connections in the PostgreSQL pool. A cap, not a floor —
2626
# connections open lazily. Size against the DB server's max_connections,
27-
# remembering admin tooling opens its own pool.
28-
GITLAWB_DB_MAX_CONNECTIONS=20
27+
# remembering admin tooling opens its own pool. Each concurrent write pins one
28+
# connection for its whole duration (the connection-affine advisory lock), so the
29+
# node REJECTS at boot any value below GITLAWB_MAX_CONCURRENT_GIT_PUSHES + 8
30+
# headroom — keep this comfortably above that (default 48 for pushes 32).
31+
GITLAWB_DB_MAX_CONNECTIONS=48
2932
# Seconds a request waits for a pool connection before failing with 503.
3033
GITLAWB_DB_ACQUIRE_TIMEOUT_SECS=5
3134
# Upper bound on each startup connect+migrate attempt, in seconds. Keep it
@@ -109,15 +112,135 @@ GITLAWB_MAX_PACK_BYTES=2147483648
109112

110113
# Max seconds a served git upload-pack / receive-pack (clone / push) may run
111114
# before it is aborted with a 504. Bounds a hung git that would otherwise pin a
112-
# worker and, on push, the repo write lock. Does NOT cover the info/refs
113-
# advertisement or the withheld-blob path, which remain unbounded. Default 600.
115+
# worker and, on push, the repo write lock. Also bounds both info/refs
116+
# advertisements, the withheld-blob pack build, and the push-side candidate
117+
# discovery (rev-list / cat-file), all reaped via process-group teardown (#174).
118+
# On the path-scoped upload-pack path the withheld-blob classification walk and
119+
# the pack serve share ONE deadline, so this value bounds their COMBINED duration
120+
# rather than giving each stage a full budget: a walk that consumes it leaves the
121+
# serve nothing and the clone gets a 504. Serving large path-scoped repos may
122+
# need a higher value here than when each stage was budgeted separately.
123+
# Must be 1..=3153600000 (100 years): the node derives deadlines from this value,
124+
# and a larger one cannot be represented. Default 600.
114125
GITLAWB_GIT_SERVICE_TIMEOUT_SECS=600
115126

127+
# Max seconds the storage-ACQUISITION phase of a served git op may run before the
128+
# request is shed with a 503, separate from the git-run timeout above. A
129+
# concurrency permit is taken before this phase and GITLAWB_GIT_SERVICE_TIMEOUT_SECS
130+
# only starts once git spawns, so without this a stalled backend (a hung Tigris
131+
# HEAD/GET, or a hung pg advisory-lock iteration on push) pins the permit and drains
132+
# the pool until every later request 503s. On expiry the permit is released
133+
# (fail-closed). Kept separate because acquisition and git execution are distinct
134+
# cost centers. Must be positive; set very large to effectively disable. Default 30.
135+
GITLAWB_GIT_ACQUIRE_TIMEOUT_SECS=30
136+
137+
# Max concurrent git READ ops (upload-pack + the upload-pack info/refs
138+
# advertisement) served at once, a global pool separate from the push pool below.
139+
# The anon receive-pack info/refs advertisement has its OWN pool (see below), not
140+
# this one. Over-cap sheds a clean 503 + Retry-After. Anonymous reads draw from
141+
# here, so pair it with GITLAWB_MAX_CONCURRENT_READS_PER_CALLER (below) so one
142+
# caller cannot monopolize the pool. Default 128.
143+
GITLAWB_MAX_CONCURRENT_GIT_OPS=128
144+
145+
# Max concurrent git-receive-pack (push) POST operations, in a pool separate from
146+
# the read pool (GITLAWB_MAX_CONCURRENT_GIT_OPS) so anonymous reads cannot shed an
147+
# authenticated push at admission. The anon receive-pack info/refs advertisement
148+
# runs in a SEPARATE pool of the same size (disjoint from this one), so an
149+
# advertisement flood cannot shed a push either. Over-cap sheds a 503 +
150+
# Retry-After. Default 32.
151+
GITLAWB_MAX_CONCURRENT_GIT_PUSHES=32
152+
153+
# Max pushes PARKED at once waiting for the per-repo write lease of the SAME repo.
154+
# A parked push holds its already-buffered pack body in memory, so this bounds that
155+
# memory for a hot repo. Past the cap the newest push sheds a clean 503 +
156+
# Retry-After ("repo is busy with another push") instead of joining the queue; a
157+
# push to a DIFFERENT repo is never affected. The lease holder is not counted.
158+
# Raising GITLAWB_MAX_CONCURRENT_GIT_PUSHES does not raise this; set it explicitly.
159+
# Default 8.
160+
GITLAWB_REPO_LEASE_MAX_WAITERS=8
161+
162+
# Max concurrent post-push pin loops (IPFS + Pinata pin_new_objects) across all
163+
# repos. The per-repo encrypt-task set caps the pin-task COUNT; this caps how many
164+
# of those loops RUN CONCURRENTLY, and so how much MB-scale object-id list memory is
165+
# held WHILE BEING PINNED. It does not bound retained memory in general: on the local
166+
# IPFS path the caller builds its list before this pool is entered, so a loop parked
167+
# waiting for a permit still holds its full list, and how many park is capped only
168+
# per repo. Do not size memory from this knob alone. A loop DEFERS (waits) when the
169+
# pool is full, never drops a pin. Default 8.
170+
GITLAWB_MAX_CONCURRENT_PIN_TASKS=8
171+
172+
# Max concurrent read ops (upload-pack + the upload-pack info/refs advertisement)
173+
# a single caller may hold, so one caller cannot monopolize the read pool. Keyed
174+
# on the resolved SOURCE IP, never the DID: a signature does not move a caller off
175+
# this cap. The source-IP key is only as granular as GITLAWB_TRUSTED_PROXY below:
176+
# left unset, a node behind an edge/NAT keys all callers on the edge IP and this
177+
# collapses to one global cap. Set GITLAWB_TRUSTED_PROXY for per-client keying; a
178+
# high-fanout caller (CI behind one NAT) then needs the operator to raise this.
179+
# Default 16.
180+
GITLAWB_MAX_CONCURRENT_READS_PER_CALLER=16
181+
182+
# Two further per-source concurrency caps exist on the PUSH side but have NO
183+
# environment variable of their own: both are DERIVED as
184+
# GITLAWB_MAX_CONCURRENT_GIT_PUSHES / 8, with a floor of 1. They are not settable
185+
# independently, so raising the push pool raises both proportionally, and the
186+
# default push pool of 32 gives each a cap of 4.
187+
# * The anon receive-pack info/refs advertisement cap: one source IP may hold at
188+
# most this many slots in the dedicated advert pool, so saturating that pool
189+
# takes ~8 distinct source IPs.
190+
# * The authenticated receive-pack POST cap: one source IP may hold at most this
191+
# many WRITE-pool slots. This one is load-bearing for push availability, since
192+
# it is acquired before the global write permit: without it, one host minting
193+
# disposable did:key identities could open enough slow pushes to monopolize the
194+
# write pool and 503 every other source (owner enforcement defaults off, and
195+
# the push rate limiter caps arrival rate, not in-flight concurrency).
196+
# Keyed on the resolved source IP, never the DID, so a DID farm does not defeat
197+
# them; keying granularity follows GITLAWB_TRUSTED_PROXY like the read cap above.
198+
116199
# ── Push rate limiting (git-receive-pack flood brake) ─────────────────────
117200
# Max receive-pack requests (info/refs advertisement + push POST) per client
118201
# IP per hour. 0 disables. Default 600.
119202
GITLAWB_PUSH_RATE_LIMIT=600
120203

204+
# ── /ipfs/{cid} visibility-walk admission (#174) ──────────────────────────
205+
# GET /ipfs/{cid} runs a per-repo full-history git walk in a blocking thread to
206+
# decide whether the caller may read a path-scoped blob. It is publicly reachable,
207+
# so it is bounded to keep a permissionless caller from fanning out unbounded
208+
# concurrent walks and exhausting blocking-pool threads + PIDs.
209+
# Max concurrent /ipfs walks across all callers (a pool of its own, disjoint from
210+
# the served-git pools). Over-cap sheds a 503. Default 32.
211+
GITLAWB_MAX_CONCURRENT_IPFS_WALKS=32
212+
# Max concurrent /ipfs walks a single SOURCE IP may hold (keyed like the git
213+
# per-caller caps via GITLAWB_TRUSTED_PROXY; reject-before-insert bounded map).
214+
# Default 4.
215+
GITLAWB_IPFS_WALK_PER_SOURCE=4
216+
# Max EXPENSIVE path-scope visibility walks per single /ipfs request (only a
217+
# blob in a path-scoped repo costs a full-history walk). Over-cap repos are
218+
# skipped without a verdict and the scan continues; if the object is then found
219+
# nowhere the request sheds a retryable 503 instead of a false 404. Default 64.
220+
GITLAWB_IPFS_MAX_REPOS_WALKED=64
221+
# Ceiling on repos one /ipfs request may VISIT past the visibility gate. Each
222+
# visit costs a repo acquire — on a Tigris cache miss a full archive download,
223+
# so this is also the worst-case object-store fetch count per request — plus a
224+
# cat-file probe. On exhaustion the scan stops and sheds a retryable 503.
225+
# Default 1024.
226+
GITLAWB_IPFS_MAX_REPO_VISITS=1024
227+
# Absolute wall-clock budget for one admitted /ipfs request's acquire+walk
228+
# lifetime (all stages of the whole scan). Per-stage clamps bound the acquire
229+
# and walk stages to the remaining budget, and no stage starts once it is
230+
# exhausted (the scan then sheds a retryable 503). The object-type probe and
231+
# content-read cat-file subprocesses are budget-checked before starting AND each
232+
# run under their own deadline (the lesser of GITLAWB_GIT_SERVICE_TIMEOUT_SECS and
233+
# the remaining budget), reaped via process-group teardown, so a hung cat-file
234+
# cannot hold the request's walk slot past it. One hang path is still unbounded:
235+
# the probe's object-store readability check is a plain filesystem sweep with
236+
# nothing to reap, so a wedged filesystem can hold the slot past the deadline.
237+
# Must be 1..=3153600000 (100 years): the node derives an Instant deadline from
238+
# this value, and a larger one cannot be represented. Default 600.
239+
GITLAWB_IPFS_REQUEST_BUDGET_SECS=600
240+
# Max /ipfs/{cid} requests per client IP per hour (route flood brake, distinct
241+
# from the concurrency caps above). 0 disables. Default 600.
242+
GITLAWB_IPFS_RATE_LIMIT=600
243+
121244
# ── Creation rate limiting (repo/agent/issue/PR flood brake) ──────────────
122245
# Max creation requests (POST /api/v1/repos, /api/register, fork, issues,
123246
# pulls) per client IP per hour, in addition to the per-DID limit. The per-DID

‎.github/workflows/pr-checks.yml‎

Lines changed: 57 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,13 @@ on:
66
# result before it lands. Guards against semantic merge races where two
77
# individually-green PRs break main in combination (#113 + #145 → #229).
88
merge_group:
9+
# Release-branch heads are created and pushed by GITHUB_TOKEN (release-please
10+
# and the lock-sync job), and GitHub's recursion guard means those pushes
11+
# trigger no workflow runs. workflow_dispatch is the documented carve-out that
12+
# GITHUB_TOKEN CAN trigger, so the release flow dispatches this workflow at
13+
# the release branch to attach check runs to its head. Read-only workflow
14+
# (contents: read), so the dispatch surface adds no privilege.
15+
workflow_dispatch:
916
push:
1017
branches:
1118
- main
@@ -46,7 +53,20 @@ jobs:
4653
run: cargo fmt --all -- --check
4754

4855
- name: cargo clippy
49-
run: cargo clippy --workspace --all-targets -- -D warnings
56+
run: cargo clippy --locked --workspace --all-targets -- -D warnings
57+
58+
release-tag-validation:
59+
name: release tag validation
60+
runs-on: ubuntu-latest
61+
timeout-minutes: 5
62+
steps:
63+
- name: Check out repository
64+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
65+
with:
66+
persist-credentials: false
67+
68+
- name: Test release tag resolver
69+
run: scripts/test-resolve-release-tag.sh
5070

5171
test:
5272
name: test (${{ matrix.toolchain }})
@@ -94,7 +114,7 @@ jobs:
94114
- name: cargo test
95115
env:
96116
DATABASE_URL: postgres://postgres:postgres@localhost:5432/gitlawb_test
97-
run: cargo test --workspace
117+
run: cargo test --locked --workspace
98118

99119
build-release:
100120
name: build --release
@@ -117,7 +137,7 @@ jobs:
117137
key: release
118138

119139
- name: cargo build --release
120-
run: cargo build --release --workspace
140+
run: cargo build --locked --release --workspace
121141

122142
audit:
123143
name: cargo audit
@@ -199,7 +219,7 @@ jobs:
199219
key: msrv
200220

201221
- name: cargo check on MSRV
202-
run: cargo check --workspace --all-targets
222+
run: cargo check --locked --workspace --all-targets
203223

204224
docker-build:
205225
name: Docker build smoke test
@@ -246,7 +266,7 @@ jobs:
246266
key: windows
247267

248268
- name: cargo test (shipped Windows crates)
249-
run: cargo test -p gl -p git-remote-gitlawb
269+
run: cargo test --locked -p gl -p git-remote-gitlawb
250270

251271
# gitlawb-core is embedded by every consumer (gl, git-remote-gitlawb, the node
252272
# daemon), so it must stay lean. This gate fails if gitlawb-core's normal
@@ -276,3 +296,35 @@ jobs:
276296

277297
- name: Check gitlawb-core dependency allowlist
278298
run: bash scripts/check-gitlawb-core-deps.sh
299+
300+
# release-please only rewrites a crate's version when the line carries the
301+
# `# x-release-please-version` annotation AND the manifest is listed under
302+
# extra-files in release-please-config.json. Miss either half and the crate
303+
# is skipped silently at release time, freezing at its creation version:
304+
# icaptcha-client was added as a workspace member after the config was last
305+
# written and sat at 0.4.0 through four releases before anyone noticed.
306+
# Exhaustive-by-construction, like the allowlist gate above: the required set
307+
# is derived from `cargo metadata`, so a newly added crate reds CI whether or
308+
# not its author knew this gate existed.
309+
release-versioning:
310+
name: release versioning coverage
311+
runs-on: ubuntu-latest
312+
timeout-minutes: 15
313+
steps:
314+
- name: Check out repository
315+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
316+
with:
317+
persist-credentials: false
318+
319+
- name: Set up Rust toolchain
320+
uses: dtolnay/rust-toolchain@b3b07ba8b418998c39fb20f53e8b695cdcc8de1b # stable
321+
with:
322+
toolchain: stable
323+
324+
- name: Cache cargo
325+
uses: Swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0
326+
with:
327+
key: release-versioning
328+
329+
- name: Check every workspace crate is release-managed
330+
run: bash scripts/check-release-versioning.sh

‎.github/workflows/pr-triage.yml‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,9 +70,15 @@ jobs:
7070
const changedRust = names.some(n => n.startsWith("crates/") && n.endsWith(".rs"));
7171
// A test can live in a tests/ dir, a *_test.rs file, or (the common Rust
7272
// pattern) an inline #[test] / #[cfg(test)] block added inside the source file.
73+
// Async tests use a namespaced attribute (#[tokio::test], #[async_std::test],
74+
// possibly with args like #[tokio::test(flavor = ...)]) rather than the bare
75+
// #[test]. Match only on added lines, with the attribute at the start of the
76+
// line after horizontal whitespace, covering bare, cfg(test), namespaced,
77+
// optional leading ::, and raw-identifier (r#test) forms. No head-file fetch
78+
// or lexer: needs-tests is advisory, not a merge gate.
7379
const addsInlineTest = files.some(f =>
7480
f.filename.endsWith(".rs") && f.patch &&
75-
/^\+.*#\[(test\]|cfg\(test\))/m.test(f.patch));
81+
/^\+[ \t]*#\[[ \t]*(?:cfg[ \t]*\([ \t]*test[ \t]*\)|(?:::[ \t]*)?(?:[\w-]+[ \t]*::[ \t]*)*(?:r#)?test\b)/m.test(f.patch));
7682
const touchedTests =
7783
names.some(n => n.includes("/tests/") || n.endsWith("_test.rs")) || addsInlineTest;
7884
if (changedRust && !touchedTests) want.add("needs-tests");

0 commit comments

Comments
 (0)