Part of epic tracebloc/backend#1142 (Theme D — preflight parity). Follow-up from #371.
LOW/MED. For the image filename column there is a case-sensitivity asymmetry between the ingestor's preflight and its transfer read — the same #340 class already fixed for the label column, still open for filename:
- Preflight (
IngestableRecordsValidator._match_column(header, "filename"), data-ingestors validators/ingestable_records_validator.py) matches case-insensitively — a header like Filename / FileName passes.
- Transfer (
record_processor.py:279 / file_transfer.py:179) reads the file key with a case-sensitive record.get("filename") — a non-lowercase header resolves to None, so every row fails at transfer ("No filename found in record", exit 9), after the upload.
Net: a Filename-header image dataset passes the ingestor's preflight but is doomed at transfer. #340 fixed exactly this class for the label column via ingestors/base.py _resolve_label_column (pins the configured label name to the actual header case-/whitespace-insensitively). No equivalent resolver exists for filename.
Preferred fix: ingestor-side — normalize/resolve the filename column to the transfer read path (mirror the #340 label resolver), so preflight and transfer agree. That fixes it for every client, not just the CLI.
CLI note: cli#371 added CheckImageFilenameColumn, which matches case-insensitively (consistent with the ingestor's preflight validator and the text family). Once the ingestor resolves filename case-insensitively end-to-end, no CLI change is needed. If instead the decision is that the cluster contract stays case-sensitive, the CLI check should be tightened to require exact-lowercase filename (trimmed) so it previews the transfer rejection.
Part of epic tracebloc/backend#1142 (Theme D — preflight parity). Follow-up from #371.
LOW/MED. For the image
filenamecolumn there is a case-sensitivity asymmetry between the ingestor's preflight and its transfer read — the same #340 class already fixed for the label column, still open forfilename:IngestableRecordsValidator._match_column(header, "filename"),data-ingestorsvalidators/ingestable_records_validator.py) matches case-insensitively — a header likeFilename/FileNamepasses.record_processor.py:279/file_transfer.py:179) reads the file key with a case-sensitiverecord.get("filename")— a non-lowercase header resolves toNone, so every row fails at transfer ("No filename found in record", exit 9), after the upload.Net: a
Filename-header image dataset passes the ingestor's preflight but is doomed at transfer. #340 fixed exactly this class for the label column viaingestors/base.py_resolve_label_column(pins the configured label name to the actual header case-/whitespace-insensitively). No equivalent resolver exists forfilename.Preferred fix: ingestor-side — normalize/resolve the filename column to the transfer read path (mirror the #340 label resolver), so preflight and transfer agree. That fixes it for every client, not just the CLI.
CLI note: cli#371 added
CheckImageFilenameColumn, which matches case-insensitively (consistent with the ingestor's preflight validator and the text family). Once the ingestor resolvesfilenamecase-insensitively end-to-end, no CLI change is needed. If instead the decision is that the cluster contract stays case-sensitive, the CLI check should be tightened to require exact-lowercasefilename(trimmed) so it previews the transfer rejection.