Skip to content

chore(release): bump version to 0.7.6#394

Merged
shujaatTracebloc merged 2 commits into
developfrom
release/v0.7.6
Jul 24, 2026
Merged

chore(release): bump version to 0.7.6#394
shujaatTracebloc merged 2 commits into
developfrom
release/v0.7.6

Conversation

@divyasinghds

@divyasinghds divyasinghds commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Release 0.7.6 — bumps __version__ (0.7.5 → 0.7.6).

Included since v0.7.5

Why now

The published v0.7.5 image predates #393, so it lacks the tracebloc-backfill entrypoint the client chart's backfill hook invokes. Cutting 0.7.6 produces an ingestor image that carries it.

Release steps (after merge)

  1. Merge this PR to develop.
  2. Push tag v0.7.6release-image.yml builds + signs the multi-arch image and publishes ghcr.io/tracebloc/ingestor:0.7.6 (and updates floating :0.7 / :0).
  3. Promote developmaster for the PyPI package (publish-master.yml), per the usual flow.

No code changes beyond the version bump.

🤖 Generated with Claude Code


Note

Low Risk
Small validator behavior fix plus version bump; no auth, data, or API surface changes beyond more reliable validation messaging.

Overview
Release 0.7.6 bumps __version__ from 0.7.5 to 0.7.6.

Image resolution auto-detection no longer uses the first file in glob order blindly. ImageResolutionValidator walks the image list and sets expected_resolution from the first file that actually reads successfully. That way a corrupt file listed first cannot turn a too-small dataset into a misleading “auto-detection failed” result that varied by OS/filesystem ordering.

A new test test_floor_wins_when_corrupt_file_is_first forces corrupt-before-readable ordering (via monkeypatched _get_image_files) and asserts the minimum-size floor error still wins while corrupt paths stay in invalid_files metadata.

Reviewed by Cursor Bugbot for commit d1c7b1b. Bugbot is set up for automated code reviews on this repo. Configure here.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…s[0]

test_too_small_result_still_reports_corrupt_files was order-dependent:
auto-detection read only image_files[0], so when a corrupt file came
first in glob order (deterministic on the Linux CI, not on macOS) it
returned None, expected_resolution stayed unset, and
_validate_image_resolutions bailed with "auto-detection failed" before
the min-size floor check could run — masking the real verdict and
making the outcome depend on filesystem ordering.

Scan for the first *readable* image instead; if none are readable,
expected_resolution stays unset and the honest "auto-detection failed"
still surfaces. Adds an order-independence guard test that forces the
corrupt file first (fails pre-fix, passes post-fix).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@shujaatTracebloc

Copy link
Copy Markdown
Contributor

Pushed a fix for the red pytest — the failure was not the version bump.

test_too_small_result_still_reports_corrupt_files was order-dependent. Auto-detection in ImageResolutionValidator.validate() read only image_files[0], so when the corrupt broken.jpg came first in glob order (deterministic on the Linux CI, not on macOS — hence green locally, red on CI) it returned None, expected_resolution stayed unset, and _validate_image_resolutions bailed with "auto-detection failed" before the min-size floor check could run. Reproduced it deterministically by forcing the corrupt file first.

Fix (d1c7b1b): auto-detect from the first readable image instead of [0]; if none are readable, the honest "auto-detection failed" still surfaces. Added an order-independence guard test that forces the corrupt file first (fails pre-fix, passes post-fix). Full suite green locally (1867 passed, 1 xfailed).

Since #394 merges into develop, this lands the fix on develop in the same merge.

@shujaatTracebloc
shujaatTracebloc merged commit 9b0f439 into develop Jul 24, 2026
6 checks passed
@shujaatTracebloc
shujaatTracebloc deleted the release/v0.7.6 branch July 24, 2026 09:50
shujaatTracebloc added a commit that referenced this pull request Jul 27, 2026
* fix(validators): stop doubling "Validator" in validation log lines (D1) (#392)

validator.name already ends in "Validator" (e.g. "Data Validator"), so appending
the word again produced "Data Validator Validator successfully passed" (and the
same doubling in the failed/error paths). Drop the redundant literal in all
three spots so the lines read "Data Validator successfully passed" /
"Data Validator failed: …" / "Data Validator error: …". Log-string only, no
logic change; updated the one docstring that quoted the old doubled message.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

* feat(backfill): metadata backfill runner — GET dataset → recompute → send to backend (#378 send side) (#393)

* feat(backfill): metadata backfill runner — GET dataset, recompute, send to backend

Wires data-ingestors #378 (build_dataset_metadata) to the backend metadata
endpoints (#1166 POST, #1198 GET) into a one-time, per-client pre-cutover
backfill sweep. For each registered dataset it:

  1. GETs the backend record by ingestor_id (authoritative category +
     data_format, not recoverable from the table alone);
  2. recomputes {schema, meta_data} from persisted rows via
     build_dataset_metadata, keyed on that category;
  3. POSTs it back — the endpoint upserts GlobalMetaData and re-folds any
     competition built from the table.

- APIClient.get_dataset_metadata / send_metadata_backfill — the two API calls,
  with local-mode mocks, 404→None (skip), and error surfacing.
- Database.list_registered_runs — enumerate the client's REGISTERED ingest
  journal (the datasets to sweep).
- metadata_backfill_runner — backfill_dataset (GET→build→POST, skips
  competitions/merged and already-new-shape datasets) + backfill_datasets
  (per-dataset guarded sweep) + a `tracebloc-backfill` console-script entrypoint
  reading DB/backend creds from env (no ingest.yaml).

Idempotent and re-runnable; new ingests already emit new-shape metadata, so this
is a rollout step, not always-on.

Tests: runner orchestration/skip/guard paths, the two API methods
(success/404/error/local/payload shape), and the DB enumeration.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore: retrigger checks after retargeting PR base to develop

The FR gate (staging/prod-promotion guard) ran against the original master
base and its stale failure lingered on the head commit after retargeting to
develop, where the gate does not apply. Empty commit to reset the head SHA.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(backfill): don't leak raw exception text / response bodies in the sweep (Bugbot #393)

Per the org rule "error messages and logs must never embed customer cell values":

- Runner: on a per-dataset failure, record and log the exception TYPE only
  (`type(exc).__name__`) instead of `str(exc)[:500]`, and use logger.error
  rather than logger.exception (no traceback). This runs in a Helm-hook Job
  whose output lands in install logs.
- APIClient.get_dataset_metadata / send_metadata_backfill: raise
  HTTPError("HTTP <status>") without response.text. The backfill POST body
  carries this table's categorical vocab (customer cell values), so a backend
  error echoing it back must not reach the exception string / logs. The response
  object is still attached for deliberate programmatic inspection.

Test asserts the recorded error is the exception type ("RuntimeError"), proving
the raw message ("boom") is not surfaced.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

* chore(release): bump version to 0.7.6 (#394)

* chore(release): bump version to 0.7.6

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(image-validator): auto-detect from first readable image, not files[0]

test_too_small_result_still_reports_corrupt_files was order-dependent:
auto-detection read only image_files[0], so when a corrupt file came
first in glob order (deterministic on the Linux CI, not on macOS) it
returned None, expected_resolution stayed unset, and
_validate_image_resolutions bailed with "auto-detection failed" before
the min-size floor check could run — masking the real verdict and
making the outcome depend on filesystem ordering.

Scan for the first *readable* image instead; if none are readable,
expected_resolution stays unset and the honest "auto-detection failed"
still surfaces. Adds an order-independence guard test that forces the
corrupt file first (fails pre-fix, passes post-fix).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: shujaat <shujaat@tracebloc.io>

* feat(backfill): discover dataset tables directly, not just the runs journal (#395)

* feat(backfill): discover dataset tables directly, not just the runs journal

The runner enumerated the runs journal (list_registered_runs), which only has
entries for datasets ingested by the post-journal ingestor. The pre-cutover
backlog the backfill exists for was ingested BEFORE the journal, so it had no
entry and was silently swept 0 (verified on dev: empty journal, ~170 real
dataset tables untouched).

Enumerate dataset tables directly instead:
- Database.list_dataset_ingestor_ids() — scan information_schema for tables
  carrying the framework `ingestor_id` column (excluding the run journal + salt
  store), then read each table's distinct non-null ingestor_ids. Covers
  journalled AND pre-journal datasets (the journal is a strict subset).
- backfill_datasets() defaults to this discovery. A table appended by several
  ingest runs has multiple ingestor_ids; metadata is table-level, so a table is
  backfilled once — the first id the backend resolves wins, siblings are skipped
  (a not_found/error id doesn't mark the table done, so another id can resolve
  it). Explicit ingestor_ids still supported.

Tests: table discovery is the default; framework tables excluded; multi-id table
backfilled once; sibling id retried after a not_found.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore(release): bump version to 0.7.7

Bundled with the table-discovery change so a single merge + v0.7.7 tag yields a
deployable image carrying it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(backfill): guard per-table discovery so one bad table can't abort the sweep (Bugbot #395)

list_dataset_ingestor_ids scanned every dataset table with no per-table guard, so
a single failing SELECT DISTINCT (scan timeout, table dropped mid-sweep, perms)
aborted enumeration and skipped ALL tables. Wrap the per-table query in
try/except: log the table name + exception TYPE only (never str(exc) — a driver
message can embed cell values and feeds install logs) and continue.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

* fix: normalize validator log labels (Bugbot, promotion PR #396) (#397)

Append "Validator" only when validator.name doesn't already end in it,
so names like "Ingestable Records" keep the suffix and names like
"Data Validator" don't double it.

- tracebloc_ingestor/ingestors/base.py

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: lukasWuttke <54042461+LukasWodka@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Divya <divyasingh@tracebloc.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants