feat(node): pin the per-push object delta instead of re-enumerating the whole repo (#90)

* feat(node): add push-delta object enumeration module

Compute the per-push object delta (git rev-list --objects <new> --not <old>)
for the pin path instead of enumerating the whole repo, with a fully-peeled
ref-type guard (cat-file -t '<sha>^{}') that fails closed to a full scan on a
non-commit tip, any git error, or the GITLAWB_PIN_FORCE_FULLSCAN kill-switch.
The whole-repo lister moves here for the fallback and the reconciliation sweep.

Pin-enumeration scope only (N16); the withheld filter is unchanged.

* feat(node): pin the per-push object delta instead of the whole repo

Wire the push-delta enumeration into the IPFS and Pinata pin paths so pin work
scales with push size, not repo size (N16).

- U2: both pin_new_objects twins take a candidate OID set instead of scanning
  the whole repo internally; repo_path is retained for reading object bytes.
- U4: extract the anonymous replication withheld gate (caller=None, no caller
  param) into a shared helper returning (announce, withheld), so the push path
  and the future reconciliation sweep cannot drift. The per-caller read-serve
  withheld call in git_upload_pack is deliberately left separate.
- U3: the post-receive handler resolves the push delta once (off-thread),
  falls back to a full scan on a non-commit tip / git error / kill-switch, and
  passes the candidate set to both pin paths. Logs delta size and fallbacks.

The reconciliation sweep (U5) is deferred: it depends on the history-deep
blob_paths fix (#84) landing on main first.

* fix(review): log degraded pin-candidate paths, extract wiring to push_delta

Code review found the push candidate computation collapsed both a failed
full-scan fallback and a panicked spawn_blocking task to a silent empty set,
diverging from the sibling withheld path that logs and fails closed.

Extract resolve_candidates_for_push into git::push_delta (it owns the
PinCandidates dispatch and full-scan fallback), logging every degraded path
(fallback taken, fallback failed, task panicked) instead of failing silently.
Pinning nothing on a hard failure is still safe (the withheld filter always
runs; the reconciliation sweep backstops durability) but is now observable.
Moving the dispatch out of the HTTP handler also makes it unit-testable.

Add tests: tag-of-tag-of-commit peels to a Delta, and resolve_candidates_for_push
delta + non-commit-tip full-scan-fallback paths.

* fix(node): fail announce closed when withheld walk fails (#90)

A failed or panicked withheld_blob_oids walk left announce=true while
withheld=None, so a push we could not vet would still gossip ref updates,
notify peers, and write a permanent Arweave anchor. Force announce false
in that arm so the gate fails closed on both axes.

Author: beardthelion

crates/gitlawb-node/src/api/repos.rs

@@ -17,6 +17,75 @@ use crate::state::AppState;
 use crate::visibility::{visibility_check, Decision};
 use crate::webhooks;
 
+/// The git all-zeros object id — the create/delete sentinel in a ref update.
+const ZERO_SHA: &str = "0000000000000000000000000000000000000000";
+
+/// The set of blob OIDs withheld from **anonymous** replication for a repo, or
+/// `None` when the repo must not replicate at all (private / mode A /
+/// undetermined — fail closed). This is the anonymous replication gate:
+/// `caller` is hard-coded to `None` and there is intentionally no caller
+/// parameter, which distinguishes it from the per-caller read-serve projection
+/// in `git_upload_pack` (which passes the real caller). Both the push pin path
+/// and the reconciliation sweep call this helper so the two cannot drift on
+/// what is withheld. `rules` is the already-fetched visibility-rule snapshot
+/// (callers fetch once and may reuse it, e.g. for encrypt-then-pin).
+///
+/// Returns `(announce, withheld)`: `announce` is whether the repo may be
+/// announced/replicated to the anonymous public at all (also gates gossip and
+/// Arweave anchoring downstream), and `withheld` is the anonymous withheld blob
+/// set when announceable (`None` when not announceable). A failed/panicked
+/// withheld walk fails closed on both axes: `announce` is forced false and
+/// `withheld` is `None`, so an unvetted push neither replicates blobs nor
+/// announces. Returning both keeps the gate's announce decision a single
+/// source rather than recomputing it at each call site.
+async fn replication_withheld_set(
+    rules: Option<Vec<crate::db::VisibilityRule>>,
+    owner_did: &str,
+    is_public: bool,
+    disk_path: std::path::PathBuf,
+) -> (bool, Option<std::collections::HashSet<String>>) {
+    let announce = match &rules {
+        Some(rules) => visibility_check(rules, is_public, owner_did, None, "/") == Decision::Allow,
+        None => false,
+    };
+    if !announce {
+        return (false, None);
+    }
+    let withheld = match rules {
+        Some(rules) if rules.is_empty() => Some(std::collections::HashSet::new()),
+        // withheld_blob_oids walks every ref with blocking `git ls-tree`; keep
+        // that off the async worker thread.
+        Some(rules) => {
+            let owner_did = owner_did.to_string();
+            tokio::task::spawn_blocking(move || {
+                crate::git::visibility_pack::withheld_blob_oids(
+                    &disk_path, &rules, is_public, &owner_did, None,
+                )
+            })
+            .await
+            .map_err(|e| {
+                tracing::warn!(err = %e, "withheld_blob_oids task panicked; skipping replication")
+            })
+            .ok()
+            .and_then(|r| {
+                r.map_err(|e| {
+                    tracing::warn!(err = %e, "withheld_blob_oids failed; skipping replication")
+                })
+                .ok()
+            })
+        }
+        None => None,
+    };
+    // Fail closed on a failed/panicked withheld walk: with `announce` already
+    // true here, a `None` withheld can only mean the walk errored (rules are
+    // necessarily `Some`, else we returned above). Suppress the announce too so
+    // a push we couldn't vet does not gossip, notify peers, or anchor to Arweave.
+    match withheld {
+        Some(withheld) => (announce, Some(withheld)),
+        None => (false, None),
+    }
+}
+
 // ── Request / Response types ───────────────────────────────────────────────
 
 #[derive(Debug, Deserialize)]
@@ -692,49 +761,40 @@ pub async fn git_receive_pack(
     // network-facing announcements. Fail closed: a private or undetermined repo
     // never leaks.
     let rules_opt = state.db.list_visibility_rules(&record.id).await.ok();
-    let announce = match &rules_opt {
-        Some(rules) => {
-            visibility_check(rules, record.is_public, &record.owner_did, None, "/")
-                == Decision::Allow
-        }
-        None => false,
-    };
-    let withheld: Option<std::collections::HashSet<String>> = if !announce {
-        None
+    let (announce, withheld) = replication_withheld_set(
+        rules_opt.clone(),
+        &record.owner_did,
+        record.is_public,
+        disk_path.clone(),
+    )
+    .await;
+
+    // Compute the per-push pin candidate set once, off the async worker (git
+    // subprocess). On the happy path this is the push delta (objects this push
+    // introduced); on any non-commit tip or git error it fails closed to a full
+    // repo scan. Only needed when something will actually replicate. All
+    // degraded paths log inside the helper rather than failing silently.
+    let candidates: Vec<String> = if withheld.is_some() {
+        let new_tips: Vec<String> = ref_updates
+            .iter()
+            .map(|u| u.new_sha.clone())
+            .filter(|s| s != ZERO_SHA)
+            .collect();
+        let old_tips: Vec<String> = ref_updates
+            .iter()
+            .map(|u| u.old_sha.clone())
+            .filter(|s| s != ZERO_SHA)
+            .collect();
+        crate::git::push_delta::resolve_candidates_for_push(disk_path.clone(), new_tips, old_tips)
+            .await
     } else {
-        match &rules_opt {
-            Some(rules) if rules.is_empty() => Some(std::collections::HashSet::new()),
-            // withheld_blob_oids walks every ref with blocking `git ls-tree`;
-            // keep that off the async worker thread.
-            Some(rules) => {
-                let path = disk_path.clone();
-                let rules = rules.clone();
-                let owner_did = record.owner_did.clone();
-                let is_public = record.is_public;
-                tokio::task::spawn_blocking(move || {
-                    crate::git::visibility_pack::withheld_blob_oids(
-                        &path, &rules, is_public, &owner_did, None,
-                    )
-                })
-                .await
-                .map_err(|e| {
-                    tracing::warn!(err = %e, "withheld_blob_oids task panicked; skipping replication for this push")
-                })
-                .ok()
-                .and_then(|r| {
-                    r.map_err(|e| {
-                        tracing::warn!(err = %e, "withheld_blob_oids failed; skipping replication for this push")
-                    })
-                    .ok()
-                })
-            }
-            None => None,
-        }
+        Vec::new()
     };
 
     // Pin new git objects to the local IPFS node (no-op if ipfs_api is empty).
     // Skipped entirely when the public cannot read the repo (withheld == None).
     if let Some(withheld_ipfs) = withheld.clone() {
+        let candidates_ipfs = candidates.clone();
         let ipfs_api = state.config.ipfs_api.clone();
         let repo_path_clone = disk_path.clone();
         let db_clone = state.db.clone();
@@ -751,6 +811,7 @@ pub async fn git_receive_pack(
             let pinned = crate::ipfs_pin::pin_new_objects(
                 &ipfs_api,
                 &repo_path_clone,
+                candidates_ipfs,
                 &db_clone,
                 &withheld_ipfs,
             )
@@ -853,6 +914,7 @@ pub async fn git_receive_pack(
         let self_public_url = state.config.public_url.clone();
         let node_keypair = Arc::clone(&state.node_keypair);
         let withheld_pinata = withheld;
+        let candidates_pinata = candidates;
         tokio::spawn(async move {
             let pinned = match &withheld_pinata {
                 Some(withheld) => {
@@ -861,6 +923,7 @@ pub async fn git_receive_pack(
                         &pinata_upload_url,
                         &pinata_jwt,
                         &repo_path_clone,
+                        candidates_pinata,
                         &db_clone,
                         withheld,
                     )

crates/gitlawb-node/src/git/mod.rs

@@ -1,4 +1,5 @@
 pub mod issues;
+pub mod push_delta;
 pub mod repo_store;
 pub mod smart_http;
 pub mod store;