Skip to content

chore(client): remove auto metadata-backfill hook; run backfill manually - #404

Merged
saadqbal merged 1 commit into
developfrom
chore/remove-metadata-backfill-hook
Jul 27, 2026
Merged

chore(client): remove auto metadata-backfill hook; run backfill manually#404
saadqbal merged 1 commit into
developfrom
chore/remove-metadata-backfill-hook

Conversation

@divyasinghds

@divyasinghds divyasinghds commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

What

Removes the automatic metadata-backfill runner (the post-upgrade Helm hook added in #380). The backfill will be run manually as a one-off Job instead of firing on every chart upgrade.

Deleted:

  • client/templates/metadata-backfill-hook.yaml — the post-upgrade hook Job
  • client/tests/metadata_backfill_hook_test.yaml — its helm-unittest
  • metadataBackfill block in client/values.yaml (+ comment header)
  • metadataBackfill property in client/values.schema.json

Why

The auto-hook coupled a one-time, best-effort data migration to the chart's upgrade lifecycle, which repeatedly surfaced timing/--timeout edge cases in review. Since the backfill is genuinely a one-time operation per cluster, running it manually is simpler and removes that coupling entirely. The runner itself (tracebloc-backfill entrypoint, data-ingestors #393/#395) is unchanged and stays the mechanism — we just invoke it on demand.

Manual run (replaces the hook)

Run the ingestor image's entrypoint as a standalone Job in the client namespace:

apiVersion: batch/v1
kind: Job
metadata:
  name: metadata-backfill-manual
spec:
  backoffLimit: 0
  activeDeadlineSeconds: 1800
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: backfill
          image: ghcr.io/tracebloc/ingestor:0.7.7
          command: ["tracebloc-backfill"]
          env:
            - {name: CLIENT_ID,       valueFrom: {secretKeyRef: {name: tracebloc-secrets, key: CLIENT_ID}}}
            - {name: CLIENT_PASSWORD, valueFrom: {secretKeyRef: {name: tracebloc-secrets, key: CLIENT_PASSWORD}}}
            - {name: MYSQL_HOST,  value: "mysql-client"}
            - {name: CLIENT_ENV,  value: "dev"}
            - {name: LOG_LEVEL,   value: "INFO"}
            - {name: HOME,        value: "/tmp"}

Verified on dev (tb-client-dev-templates / tracebloc-templates): ok=98, skipped_current=26, not_found=26, errors=0. The sweep is idempotent, so re-running is safe.

Testing

  • helm lint client passes.
  • helm template renders 0 backfill objects.
  • values.yaml / values.schema.json validate.

🤖 Generated with Claude Code


Note

Low Risk
Chart-only removal of an optional hook; no changes to ingestor runtime, jobs-manager, or auth paths. Upgrades no longer run backfill automatically—clusters that still need migration must run it manually once.

Overview
Removes the automatic federated-alignment metadata backfill from the client Helm chart. Chart upgrades no longer spawn a post-upgrade hook Job that ran tracebloc-backfill on the ingestor image.

Deleted: metadata-backfill-hook.yaml, its helm-unittest suite, and the metadataBackfill values block (including enabled, backfillTimeoutSeconds, and resources) from values.yaml and values.schema.json.

The tracebloc.ingestorDigest helper comment is updated so ingestor image resolution is documented for jobs-manager ingestion Jobs and manual backfill Jobs only—not the removed hook.

Operators run the same tracebloc-backfill entrypoint as a one-off Job when needed; the sweep stays idempotent.

Reviewed by Cursor Bugbot for commit 853ef89. Bugbot is set up for automated code reviews on this repo. Configure here.

@divyasinghds
divyasinghds requested a review from saadqbal as a code owner July 27, 2026 06:46
@divyasinghds divyasinghds self-assigned this Jul 27, 2026
@LukasWodka

Copy link
Copy Markdown
Contributor

👋 Heads-up — Code review queue is at 44 / 30

Above the WIP limit. The team convention is to review existing PRs before opening new work.

Open PRs currently in Code review (oldest first):

Pull from review before opening new work. (This is a nudge from the kanban WIP check, not a block.)

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 05627ea. Configure here.

Comment thread PR_BODY.md Outdated
@divyasinghds
divyasinghds force-pushed the chore/remove-metadata-backfill-hook branch from 05627ea to 2955eae Compare July 27, 2026 07:38
The post-upgrade Helm hook (#380) coupled a one-time, best-effort data
migration to every chart upgrade, which repeatedly surfaced timing and
--timeout edge cases. The backfill is a one-time-per-cluster operation, so
run it manually as a standalone Job instead. The tracebloc-backfill runner
(data-ingestors #393/#395) is unchanged and remains the mechanism.

Removes:
- templates/metadata-backfill-hook.yaml
- tests/metadata_backfill_hook_test.yaml
- metadataBackfill block in values.yaml + values.schema.json

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@divyasinghds
divyasinghds force-pushed the chore/remove-metadata-backfill-hook branch from 2955eae to 853ef89 Compare July 27, 2026 07:44
@saadqbal
saadqbal merged commit 6d7327b into develop Jul 27, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants