Skip to content

Add provenance-preserving federated search#109

Open
jmcte wants to merge 6 commits into
mainfrom
codex/issue-96-federated-search
Open

Add provenance-preserving federated search#109
jmcte wants to merge 6 commits into
mainfrom
codex/issue-96-federated-search

Conversation

@jmcte

@jmcte jmcte commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add bounded federated search over opted-in local provider archives
  • preserve provider-native ids and return timestamp, sensitivity, redaction, schema, fingerprint, and archive provenance for every hit
  • support provider/time filters, deterministic ranking, bounded pagination, current archive retention, migration, and tombstones
  • keep high-sensitivity providers and body indexing behind explicit opt-ins

Governing Issue

Closes #96

Validation

  • Relevant local checks passed

  • Required PR checks are expected to satisfy CI Gate

  • Skipped checks are explained below

  • swift test --disable-sandbox — 137 tests passed

  • bash scripts/ci/run-fast-checks.sh — passed; 88.80% source line coverage, mutation checks, debug and release builds passed

  • synthetic tests cover cross-provider ranking, pagination, provider filters, redaction, body gating, and evidence provenance

No checks skipped.

Bootstrap Governance

  • Changes are scoped to the linked issue
  • Contributor or PR guidance changes are reflected in CONTRIBUTING.md, .github/PULL_REQUEST_TEMPLATE.md, and docs/bootstrap/onboarding.md when applicable
  • PR author enabled auto-merge where GitHub allows it, or GitHub plan-limit evidence/unavailable reason is recorded and the fallback merge-readiness policy applies
  • No real secrets, runtime auth, or machine-local env files are committed

No contributor guidance changed. This PR is stacked through #108 and remains draft until its foundations merge.

Flow Contract

  • Owner lane: Hermes local search
  • Repair owner: jmcte
  • Autonomy class: review-gated implementation
  • Risk class: high; cross-provider discovery and sensitive metadata

Flow Merge Readiness

  • Every blocker has a next actor and next action
  • No active blocking requested changes remain
  • Non-author approval is present when required
  • PR author enabled auto-merge where GitHub allows it, or recorded why it is unavailable/unsafe

Next actor: foundation reviewers, then independent product and architecture review.

Merge Automation

  • PR author enabled auto-merge with gh pr merge --auto --squash, or the reason it is unavailable/unsafe is noted below

Auto-merge is deferred only while the PR is stacked on archive and Messages foundations.

Notes

  • No global people, place, trip, or account identity is created.
  • Search reads current archive documents directly, so retention, migration, and tombstones apply without a second derived index.

Expose a deterministic metadata-only provider registry through JSON and text output, covering every current Apple-data command family with maturity, source, permission, sensitivity, capability, and polling declarations. Document the v1 compatibility policy.\n\nCloses #89.
Add a private read-only SQLite snapshot engine with WAL and SHM companions, query-only enforcement, busy and process timeouts, structured failures, and deterministic cleanup. Migrate Messages and Safari history and document remaining providers.\n\nCloses #90.
Add explicit scan and wall-clock budgets, structured crawl reports, cancellable Drive enumeration, Finder-tag budget metadata, and redacted watch partial-success envelopes. Extend the live audit with explicit budgets and aggregate outcomes.\n\nCloses #92.
Define versioned archive sync, storage, status, migration, retention, and privacy contracts with idempotent upserts, explicit tombstones, bounded attempts, persisted freshness and counts, private atomic files, and fail-closed sensitivity gates.\n\nCloses #93.
Archive bounded Messages metadata from consistent snapshots with cursors, deduplication, deletion tombstones, source fingerprints, and versioned search. Keep bodies behind explicit retention and confirmation gates and exclude all action capabilities.
Search opted-in local archives with deterministic ranking, filters, time bounds, pagination, redaction, and per-hit evidence. Preserve provider-native identities and require explicit sensitive and body opt-ins.
@jmcte
jmcte marked this pull request as ready for review July 13, 2026 07:29
@jmcte
jmcte requested a review from a team as a code owner July 13, 2026 07:29
@jmcte
jmcte enabled auto-merge (squash) July 13, 2026 07:30

@Hermes-omt Hermes-omt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review deferred pending branch refresh. The PR head 1ddae63 is based on 2b0bcc6, while current main is d7f88dc; GitHub reports mergeable_state=dirty. A local merge-tree confirms conflicts in README.md, CommandLine.swift, CommandRunner.swift, ProviderManifest.swift, ProviderManifestTests.swift, and docs/provider-manifest.md. Repair handoff: rebase or merge d7f88dc into the PR branch, resolve these conflicts, push the new head, and wait for fresh CI before re-requesting substantive review. Exact-head source inspection and targeted federated-search tests passed locally.

@athena-omt athena-omt added status:needs-review PR is ready for Athena review. review:athena Athena review governance requested. labels Jul 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review:athena Athena review governance requested. status:needs-review PR is ready for Athena review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

v2: Add provenance-preserving federated local search

3 participants