feat(backfill): metadata backfill runner — GET dataset → recompute → send to backend (#378 send side)

[divyasinghds]

Draft — the send side of the pre-cutover metadata backfill (piece #2 of the #378 plan). Opening for review of the orchestration + entrypoint; the backend endpoints it calls (#1166 POST, #1198 GET) are already merged to develop.

What & why

Datasets ingested before the federated-alignment cutover carry no enriched schema and no per-column feature_stats on the global-metadata channel, so the backend combine-time alignment checks and the edge metadata API can't reconcile them. #378 (build_dataset_metadata) recomputes that {schema, meta_data} for a table from its persisted rows — but deliberately stopped before sending. This PR is the runner that ties it to the backend.

For each dataset it does exactly what was asked — get data from backend → based on category compute all attributes → send updated attributes back:

1. GET the backend record by ingestor_id (GET global_meta/metadata_backfill/by-ingestor/<id>/, #1198). The authoritative category + data_format drive the recompute and aren't recoverable from the table alone.
2. Recompute the enriched {schema, meta_data} from persisted rows via build_dataset_metadata (SQL aggregates, no row re-ingest), keyed on that category.
3. POST it back (POST global_meta/metadata_backfill/<table>/, #1166) — the endpoint upserts GlobalMetaData and re-folds any competition built from the table (#1198).

What this PR adds

Piece	What
APIClient.get_dataset_metadata(ingestor_id)	GET the dataset record; None on 404 (skip, not error); local-mode mock
APIClient.send_metadata_backfill(table, schema, meta_data)	POST the recomputed payload; returns {table_name, created, competitions_refolded}
Database.list_registered_runs()	Enumerate the client's registered ingest journal — the datasets to sweep
metadata_backfill_runner.backfill_dataset	GET → build → POST for one dataset
metadata_backfill_runner.backfill_datasets	The per-dataset-guarded sweep (one bad table can't abort the rollout)
tracebloc-backfill console script	Entrypoint reading DB/backend creds from the same env as the ingestor — no ingest.yaml

Skips & guards

• Competitions / merged datasets are skipped (detected via the backend record's is_competition / source_dataset_ids): they have no client-side raw table to read, and their reconciled fold is rebuilt backend-side by the re-fold the source POSTs trigger.
• Already-new-shape datasets are skipped (meta_data.attributes present) so a re-run is cheap; disable with skip_if_current=False.
• Idempotent / re-runnable — the backend upsert is a safe overwrite; new ingests already emit the new shape, so this is a one-time rollout step, not always-on.

Scope / non-goals

• Covers plain / source datasets — the per-table metadata feat(backfill): metadata builder for pre-cutover datasets (di#360) #378 computes.
• label_column and the uploader-declared scalar attributes (color_mode, language, …) aren't persisted backend-side, so they aren't passed; the SQL-derivable enriched schema + feature_stats (the backfill's primary value) are recomputed regardless. Same per-fact limitations as build_dataset_metadata's docstring.

What reviewers should focus on

1. Enumeration source — list_registered_runs() (registered ingest journal) as the set to sweep. Right boundary, or should the operator pass an explicit ingestor_id list?
2. Competition/merged skip — is is_competition or source_dataset_ids the right detector for "no local table to recompute"?
3. skip_if_current marker — using non-empty meta_data.attributes as "already new-shape." Conservative (a category with legitimately-empty attributes re-POSTs, idempotently). Acceptable?
4. Entrypoint — separate tracebloc-backfill console script (vs. a subcommand on tracebloc-ingest), reading creds from env.

Tests

• test_metadata_backfill_runner.py — happy path (GET→build→POST with category/data_format threaded), not-found, competition/merged skip, already-current skip (+ skip_if_current=False reprocess), missing-category error, journal enumeration, and per-dataset guard (a failing dataset doesn't abort the sweep).
• test_api_client_methods.py — the two new methods: success / 404→None / HTTP-error / local-mode / payload shape.
• test_database.py — list_registered_runs returns registered-only, mapped, NULL task preserved.

Full suite green (1866 passed).

Rollout / pairing

• Independently deployable; dead until run. Pairs with backend #1166/#1198 (already on develop).
• Run once per client during rollout: tracebloc-backfill in the ingestor image's env. Non-zero exit iff any dataset errored.

🤖 Generated with Claude Code

---

Note

Medium Risk
Rollout job mutates global metadata on the backend for every registered dataset; failures are isolated but a bad recompute or POST could affect alignment/competition folds until re-run (idempotent).

Overview
Adds a one-time rollout sweep for pre-cutover datasets: for each registered ingest, GET backend metadata by ingestor_id, recompute {schema, meta_data} via build_dataset_metadata from local MySQL, then POST to the metadata-backfill API (upsert + competition re-fold).

New pieces: APIClient.get_dataset_metadata / send_metadata_backfill (404 → skip; HTTP errors omit response bodies in messages for log safety), Database.list_registered_runs (registered = 1 journal rows), metadata_backfill_runner with skip rules (competition/merged, already has meta_data.attributes), per-dataset error isolation, and tracebloc-backfill entrypoint using the same env as the ingestor (no ingest.yaml).

^{Reviewed by Cursor Bugbot for commit e1aa20e. Bugbot is set up for automated code reviews on this repo. Configure here.}

[LukasWodka]

👋 Heads-up — Code review queue is at 32 / 30

Above the WIP limit. The team convention is to review existing PRs before opening new work.

Open PRs currently in Code review (oldest first):

• .github#64 — fix(closure-router): add GH_TOKEN to closing-PR lookup so issues mirror their closing PR's Status · author: @LukasWodka · no reviewer assigned
• averaging-service#206 — feat(docker): containerize averaging-service (CPU-only) + build CI (#1163) · author: @saadqbal · no reviewer assigned
• backend#1153 — WS-C: delegate CheckModelMixin rewrite path to SDK ModelFileRewriter + pin tracebloc==0.14.0 (#1101) · author: @divyasinghds · reviewer: @shujaatTracebloc
• backend#1188 — test(checker): vendor SDK rewrite golden corpus as WS-C contract gate (#1101) · author: @shujaatTracebloc · no reviewer assigned
• backend#1193 — Reject non-federatable sklearn estimators (KNN / GaussianProcess) at upload (#923) · author: @shujaatTracebloc · no reviewer assigned
• backend#1200 — Revert "chore(deps): bump torch from 2.11.0 to 2.13.0" · author: @saqlainsyed007 · no reviewer assigned
• cli#389 — fix(delete): verify the host-data wipe before printing ✔ (RFC-0003) · author: @saadqbal · reviewer: @LukasWodka
• cli#390 — feat(cli): auto-update — nudge + tracebloc upgrade (F1) · author: @LukasWodka · no reviewer assigned
• cli#391 — fix(push): stop progress bar colliding with staging log lines (D3) · author: @LukasWodka · no reviewer assigned
• cli#392 — feat(install): prefer ~/bin when already on PATH (B2, RFC 0001) · author: @LukasWodka · no reviewer assigned

Pull from review before opening new work. (This is a nudge from the kanban WIP check, not a block.)

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit b5d88e8. Configure here.}

[LukasWodka]

Functional review \u2014 passed\n\nBasis: the behavioural suites that ran on this PR at merge against real environments, not mocks:\n\n- \u2014 exercised against a real MySQL 8.0 service container, not a mock\n\nI could not reach the dev API (no dev credentials), so rather than rubber-stamp I used the strongest evidence available: these suites exercise the actual behaviour this change alters, on real infrastructure. Advancing \u2192 .\n\nIf the functional reviewer wants a manual pass on dev in addition, please move it back and say so.