Skip to content

Backfill should handle rows with missing AnalysisDate gracefully #571

@kbighorse

Description

@kbighorse

Context

Rows with missing AnalysisDate were previously inserted with a 1970-01-01 epoch sentinel, which is indistinguishable from real data downstream. In #558 these rows are now skipped with an error logged to BackfillResult.errors.

Work Required

Decide on a longer-term strategy for date-less observations:

  • Option A: Keep skipping (current behavior) and report to data stewards for manual correction
  • Option B: Allow nullable observation_datetime (schema change) for legacy data
  • Option C: Derive date from a related field (e.g., sample collection date from NMA_Chemistry_SampleInfo)

Refs

Changed in #558

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions