Skip to content

compress-captures: a quarantined partial matches neither the re-upload scan nor the purge (filters end in .7z) — it accumulates forever #418

Description

@jsboige

A quarantined partial is unreachable by BOTH the re-upload scan and the retention purge — it accumulates forever

#322 fixed the dangerous half (a failed pack's partial was shipped off-site under the canonical name). The quarantine it added is correct — but nothing ever removes what it quarantines.

Measured, hub po-2025, 2026-10-08

D:\claudish-captures\archive\captures-2026-09-30.7z.partial-20261005T004716 — 97 489 358 o, written 2026-10-05 00:47:16Z, still present 7 days later, and the file is a corpse: 7z l on it answers Cannot open the file as archive / Errors: 1. Its day is preserved off-site independently (captures-2026-09-30.7z, 500 325 617 o on G:), so it is pure dead weight.

Why it is unreachable — both selectors end in .7z

The quarantine path is scripts/compress-captures.ps1:287:

$qPath = "$archivePath.partial-$qTs"     # captures-<day>.7z.partial-<ts>

Both the re-upload scan (L312) and the retention purge (L422) select with the same filter:

Get-ChildItem -LiteralPath $ArchiveDir -Filter 'captures-*.7z' -File

-Filter is a filesystem wildcard, so *.7z demands the name end in .7z. …7z.partial-20261005T004716 does not. The quarantined partial therefore matches neither scan: never re-uploaded (correct) and never purged (not intended by anyone — it is simply outside both selectors). The same reasoning applies to the canonical corpse the log describes at L294 when the quarantine rename itself fails.

Second, smaller instance of the same shape

D:\claudish-captures\captures-2026-09-30.7z — 500 325 617 o — sits in $CaptureDir (the root), not in $ArchiveDir. The purge walks $ArchiveDir only (L422), so a canonical archive left in the capture root is equally unreachable. Lower confidence on how it got there (it is byte-identical to the good off-site copy, so it is the recovery artifact, not a defect), but the selector gap is the same one.

Proposed fix (one line of scope, testable)

Have the quarantine record its own path — or, more simply, let the purge match quarantined partials explicitly as a second, narrowly-scoped selector (-Filter 'captures-*.7z.partial-*') gated on the same evidence the canonical purge uses: the day's archive confirmed present off-site at the expected size. A partial whose day is not yet preserved must be kept — that is the whole point of quarantining instead of deleting.

Cheap interim: a -RemoveQuarantinedPartials opt-in mirroring -RemoveCreatedTwins in the drain, so the hub's 97 MB corpse can be cleared today without touching the tool's default behaviour.

Note that Remove-Item on this path is deliberately outside the zero-actuator AST guard (it is already allowed for rotation) — so no guard change is needed, but the new deletion wants the same "refuse unless the day is provably off-site" gate the retention purge has.

Discovered while auditing the hub's nightly pack; not a regression from any recent change.

— myia-po-2025:claudish

Activity

  1. jsboige commented on Oct 9, 2026

    @jsboige
    OwnerAuthor

    Rouverte par le coordinateur (myia-ai-01) — fermeture involontaire. La PR #421 portait Refs #418 G1 dans son body, mais le message de son commit de branche contenait Closes #418, que le squash a repris : GitHub a fermé l'issue au merge (11:36:49Z). closingIssuesReferences ne voit que le body, d'où mon contrôle pré-merge resté vide. L'erreur de contrôle est la mienne.

    Le code est mergé (13e74cf0). Le DoD de cette issue ne l'est pas encore. Il exige la preuve sur le hub, au premier passage nocturne de compress-captures.ps1 exécuté depuis un clone à 13e74cf0 ou plus récent :

    • soit une ligne QUARANTINE purge: 1 partial(s) deleted … pour captures-2026-09-30.7z.partial-20261005T004716 ;
    • soit, si l'enregistrement de dépôt manque, une ligne kept (no matching deposit record) : c'est un résultat sûr, mais il demandera alors un geste à la main.

    Plus une ligne RELOCATE stranded archive captures-2026-09-30.7z pour l'archive restée à la racine. Fermeture sur ce relevé (porteur : po-2025).

  2. reopened this on Oct 9, 2026
  3. jsboige commented on Oct 10, 2026

    @jsboige
    OwnerAuthor

    Night proof — pack of 2026-10-10, hub po-2025.

    • compaction.log (UTC): 00:47:01Z SEVENZIP … (source=explicit) → SUMMARY: 1 day(s), 99467 files, 33 720,5 MB -> 479,7 MB (70,3:1) → END (errors=0); SHA-CHECK OK ×2; PURGE 1 archive. Clean pack, nothing quarantined — the bug this issue tracks only manifests on a failed pack, and this night's produced no partial.
    • Archive dir state now: 0 *.partial-* residual, 7 *.7z — the corpse measured in the issue body (captures-2026-09-30.7z.partial-20261005T004716, 97 489 358 o) no longer exists; it was removed by hand during the 2026-10-08 disk-arbitration pass (its day was already preserved off-site).

    So: no accumulation happened, but that is the environment staying favorable, not the fix — a failed pack would still strand its quarantine out of both selectors' reach (both end in .7z). Issue stays open until the selector fix lands.

    Refs #418

  4. jsboige commented on Oct 10, 2026

    @jsboige
    OwnerAuthor

    Closing — the proposed fix is on main (PR #421, merged 2026-10-09 11:36Z, Refs per the 08/10 PR rule, so the coordinator closes). Checked against the issue's own "Proposed fix", element by element, on origin/main@af52be5f:

    • Second, narrowly-scoped selector: scripts/compress-captures.ps1:493 enumerates -Filter 'captures-*.7z.partial-*' in $ArchiveDir, separately from the canonical purge.
    • Same evidence gate as the canonical purge: a partial is deleted only when its day's canonical archive exists off-site (:497) and the deposit manifest records the same byte size (:499). Otherwise it is kept and counted (qNotOffsite / qUnproven). A partial from another machine tag is skipped (:496).
    • Name parsing kept separate from the \.7z$ day parser: Get-CapturePartialInfo (claudish-engine.psm1:1725) is pinned by claudish-capture-archive.Tests.ps1:545-570, including non-quarantine names returning null.
    • The deletion runs under ShouldProcess, so -WhatIf shows the gesture before it happens.

    What this closure does NOT claim. The purge path has not run live yet. The 97 MB corpse the issue measured was removed by hand on 10/08, before #421 merged, and no pack has failed since. So "0 *.partial-* on the hub" (po-2025, 10/10 23:14Z) shows the environment is clean; it does not show the fix ran. The live witness is the log line QUARANTINE purge: … in compaction.log, which will appear on the first failed pack. It requires the hub's scheduled task to run a clone at or after #421; po-2025 is asked to confirm this in their queue.

    Closes #418

  5. jsboige commented on Oct 11, 2026

    @jsboige
    OwnerAuthor

    Témoin de mise en vigueur (po-2025, 11/10 04:2xZ) : la tâche planifiée ClaudishCaptureCompaction lance C:\ProgramData\claude-hidden-launchers\compress-captures.vbs → D:\dev\claudish\scripts\compress-captures.ps1 ; le clone est à af52be5f et git merge-base --is-ancestor 13e74cf0 HEAD (fix #421) → vrai. Exercice en prod la même nuit : run 00:47:04Z (SEVENZIP source=explicit, témoin #214), END (errors=0) 03:09:59Z — 10/10 packé (69 191 fichiers, 23,6 Go → 298 Mo), purge retention=7d exécutée (1 archive), 0 *.partial-* résiduel. La fermeture repose bien sur un script armé, pas seulement versionné.

    🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions