Context
From ADR-0017 Related Debt + docs/testing.md timestamp-precision caveat. Priority: P2.
seankochel channel has auto_transcript: none — its videos produce only mindmaps, not chunked transcripts. Gold entries for seankochel use bullet-level timestamps from the mindmap ([MM:SS] - topic), not chunk-level timestamps from a transcript.
TimestampPrecisionMetric measures whether a retrieved chunk's timestamp_seconds falls within [start - tolerance, end + tolerance] of the expected hit. If the mindmap bullet is at [5:30] but the relevant content spans [5:10-5:50], the tolerance may be too strict, causing false-negative timestamp failures that mask real recall.
Proposed change
- Audit the seankochel entries. For each seankochel gold entry in
golden_dataset.yaml, manually verify:
- Does the bullet's
[MM:SS] represent the start of the relevant content, or the midpoint, or the point where the topic is first mentioned?
- What's a fair tolerance window given mindmap granularity (15s? 30s? 60s)?
- Adjust
tolerance_sec in affected entries, OR document a channel-specific default in metrics.
- Optionally add a
source: mindmap|transcript field to expected hits so the metric can pick a different tolerance.
Acceptance criteria
Dataset edit justification per ADR-0017 frozen-contract rule: this is a correction (removing measurement noise from a known granularity mismatch), not a recalibration of difficulty. Per the ADR Consequences section, corrections don't require a new ADR.
Context
From ADR-0017 Related Debt + docs/testing.md timestamp-precision caveat. Priority: P2.
seankochelchannel hasauto_transcript: none— its videos produce only mindmaps, not chunked transcripts. Gold entries for seankochel use bullet-level timestamps from the mindmap ([MM:SS] - topic), not chunk-level timestamps from a transcript.TimestampPrecisionMetricmeasures whether a retrieved chunk'stimestamp_secondsfalls within[start - tolerance, end + tolerance]of the expected hit. If the mindmap bullet is at[5:30]but the relevant content spans[5:10-5:50], the tolerance may be too strict, causing false-negative timestamp failures that mask real recall.Proposed change
golden_dataset.yaml, manually verify:[MM:SS]represent the start of the relevant content, or the midpoint, or the point where the topic is first mentioned?tolerance_secin affected entries, OR document a channel-specific default in metrics.source: mindmap|transcriptfield to expected hits so the metric can pick a different tolerance.Acceptance criteria
provenance: concept_augmented_verified+ correctly-calibratedtolerance_sec(or channel-level default).docs/testing.mdtimestamp caveat section with the resolution.Dataset edit justification per ADR-0017 frozen-contract rule: this is a correction (removing measurement noise from a known granularity mismatch), not a recalibration of difficulty. Per the ADR Consequences section, corrections don't require a new ADR.