feat(validation): evolve the trust gate into a per-area trust map

[NathanG-TD]

Closes #46.

The change

A validation run published one verdict for the whole product: any CRITICAL/ERROR failure anywhere set agent_use_allowed = stop and withdrew autonomous use of everything. A defect in Prediction cost an agent the Domain question it was actually asked.

A run now also publishes one entry per area — module, entity, pattern, capability — carrying coverage, status, confidence, open gaps and a recommended action. A consumer reads the entries for the areas its query plan touches, proceeds, and discloses their confidence alongside the answer. Nothing in the contract withholds use: what a failure costs is bounded by the area it belongs to, and strong becomes a positive statement of confidence rather than the absence of a block.

Wire schema 2.1 (additive over 2.0)

• New ValidationArea entity (validation_area) and the validation_trust_map view. A 2.0 producer projects one derived PRODUCT entry, capped at partial, so the map has one shape everywhere.
• agent_use_allowed deprecated in place: retained so a 2.0 reader still parses a 2.1 record, published as go, never branched on.
• trust_status kept as an advisory summary of the map.
• Checks declare a scope,

[earthshiner]

PR #57 review — per-area validation trust map

PR: feat(validation): evolve the trust gate into a per-area trust map
Objective: Issue #46 — Evolve validation from a binary trust gate to a per-area trust map

Overall assessment

This is the right design direction and it addresses the central problem described in issue #46: a failure in one part of a product should not withdraw unrelated, usable areas from an agent. The new validation_area contract, trust vocabulary, orientation changes, and updates to the consumer, reviewer, and builder roles are generally coherent.

I would nevertheless request changes before merging. There are several cases in the read path where the implementation can omit an unknown area or present older evidence as current. Those cases work against the main objective of making trust explicit, scoped, and honest.

Requested changes

1. Evaluate staleness against the run that produced each area

Why this matters

validation_trust_map selects the latest record independently for each area. Its rows can therefore come from different runs. The current staleness query reads only validation_latest, which is the latest run for the producer as a whole.

For example:

• run R1 validates Domain on 1 August;
• run R2 validates Prediction on 15 August;
• the map contains the Domain row from R1 and the Prediction row from R2;
• the current staleness query examines only R2 and can make the older Domain evidence appear fresh.

The query also returns evidence_is_stale separately without changing the confidence returned by the map, although the normative contract says stale evidence reads as unknown.

Relevant code: consumer-queries.sql lines 64–79

Suggested fix

Join every published area to its parent validation_run using all three parent keys:

INNER JOIN {db}.validation_run AS r
    ON  r.product_prefix = a.product_prefix
    AND r.producer_id    = a.producer_id
    AND r.run_id         = a.run_id

Expose the recorded confidence separately and make the consumer-facing confidence reflect expiry:

, a.confidence AS recorded_confidence
, CASE
      WHEN COALESCE(
               r.evidence_expires_dts,
               a.completed_dts + INTERVAL '7' DAY
           ) < CURRENT_TIMESTAMP(6)
        THEN 'unknown'
      ELSE a.confidence
  END AS confidence
, CASE
      WHEN COALESCE(
               r.evidence_expires_dts,
               a.completed_dts + INTERVAL '7' DAY
           ) < CURRENT_TIMESTAMP(6)
        THEN 1
      ELSE 0
  END AS evidence_is_stale

Where the product declares a maximum evidence age, use that to calculate the effective expiry before falling back to seven days. If that value is not currently stored in queryable orientation metadata, add a typed field for it rather than expecting consumers to recover it from prose or JSON.

For a stale row, the effective recommended_action should be to rerun the validator. Preserve the validator's original recommendation in a separate field if it is still useful.

Also add a conformance check that validation_area.completed_dts equals the parent run's completed_dts; otherwise an incorrect duplicated timestamp can alter both latest-row selection and staleness.

Acceptance test

Create two runs with different areas and dates. Confirm that the older area becomes unknown when it expires even while a newer area from the same producer remains fresh.

---

2. Base the legacy fallback on the current run and its schema version

Why this matters

The fallback currently asks whether any validation_area row has ever existed for the product and producer. It does not inspect the schema version of the latest run or restrict the existence check to that run.

This creates two incorrect outcomes:

1. A malformed 2.1 producer that publishes no area rows is silently presented as a valid legacy PRODUCT map.
2. If a producer previously published 2.1 area rows and later publishes a 2.0 run, the historical rows suppress the required fallback for the current run.

Relevant code: 04-trust-map-views.sql lines 55–90

Suggested fix

Restrict the derived fallback to explicitly supported legacy versions and test for area rows belonging to the latest run:

FROM {db}.validation_latest AS v
WHERE v.payload_schema_version IN ('1.0', '2.0')
  AND NOT EXISTS (
        SELECT 1
        FROM {db}.validation_area AS a
        WHERE a.product_prefix = v.product_prefix
          AND a.producer_id    = v.producer_id
          AND a.run_id         = v.run_id
      )

Avoid a lexical comparison such as payload_schema_version < '2.1'; explicit supported versions behave predictably when a future major version is introduced.

Add a separate VAL-18 check for every 2.1 run, not only runs containing failures:

SELECT
      r.product_prefix
    , r.producer_id
    , r.run_id
FROM {db}.validation_run AS r
WHERE r.payload_schema_version = '2.1'
  AND NOT EXISTS (
        SELECT 1
        FROM {db}.validation_area AS a
        WHERE a.product_prefix = r.product_prefix
          AND a.producer_id    = r.producer_id
          AND a.run_id         = r.run_id
      );

That prevents a broken 2.1 publication from being disguised as backward compatibility.

Acceptance tests

• A 2.0 run with no area rows receives one derived PRODUCT entry.
• A 2.1 run with no area rows fails VAL-18 and receives no legacy fallback.
• A current 2.0 run still receives its fallback when older 2.1 area rows exist for the same producer.

---

3. Return explicit unknown entries for requested areas missing from the map

Why this matters

The consumer query starts from validation_trust_map and filters it using :scope_keys. A requested scope with no map row simply disappears from the result. The caller must notice that the output contains fewer keys than the input, but the supplied contract does not enforce that reconciliation.

This is the dangerous failure mode the trust map is meant to remove: an unvalidated area can once again become invisible instead of being reported as unknown.

Relevant code: consumer-queries.sql lines 17–37

Suggested fix

Represent the query plan's requested scopes as a two-column relation—such as a request-scoped volatile table—and drive the query from that relation using a LEFT OUTER JOIN:

SELECT
      s.scope_kind
    , s.scope_id
    , COALESCE(m.area_status, 'no-evidence') AS area_status
    , COALESCE(m.confidence, 'unknown') AS confidence
    , m.checks_ran
    , m.checks_expected
    , m.coverage_ratio
    , COALESCE(
          m.open_gaps,
          'No trust-map entry was published for an area used by this query.'
      ) AS open_gaps
    , COALESCE(
          m.recommended_action,
          'Add this area to the validator profile and publish a validation_area entry.'
      ) AS recommended_action
FROM requested_validation_scope AS s
LEFT OUTER JOIN {db}.validation_trust_map AS m
    ON  m.product_prefix = :product_prefix
    AND m.producer_id    = :trust_producer
    AND m.scope_kind     = s.scope_kind
    AND m.scope_id       = s.scope_id;

Keep scope_kind and scope_id separate rather than joining on a concatenated KIND:id value. This avoids delimiter ambiguity and lets Teradata use statistics on the actual key columns.

If the consumer interface cannot supply a relation, add a companion query that returns requested scopes MINUS published scopes, and require the caller to merge those rows as no-evidence / unknown before analytical use.

Acceptance test

Request two scopes when only one has been published. Confirm that two rows are returned and that the missing scope is explicitly no-evidence / unknown with an actionable recommendation.

---

4. Make VAL-14 resolve every scope kind it claims to validate

Why this matters

The normative rule says that scope_id resolves to a real module, entity, pattern, capability, or product. The implementation currently rejects blank values and validates ENTITY scopes, but arbitrary non-empty values for MODULE, PATTERN, CAPABILITY, and PRODUCT pass.

A typo such as MODULE:domian makes evidence invisible to a consumer looking for MODULE:domain.

Relevant code: conformance-queries.sql lines 45–70

Suggested fix

At minimum, add the checks that can be resolved from existing deployed metadata:

• PRODUCT: require scope_id = product_prefix.
• MODULE: resolve the anchor through the product's Semantic module map.
• ENTITY: retain the existing module-qualified entity check.

For PATTERN and CAPABILITY, choose one of these approaches explicitly:

1. Preferred: publish a small queryable scope/profile registry containing every valid (scope_kind, scope_id) pair for a validator profile. Validate every validation_area row against that registry. The same relation can then support the other half of VAL-18: proving that every area covered by the profile received an entry.
2. Smaller change: state that pattern and capability resolution is a producer build-time assertion, implement it in the validator/profile tooling, and narrow the runtime SQL comment and VAL-14 wording so they do not claim to validate something they cannot see.

A unified scope registry is preferable because it makes checks_expected, scope identity, and missing-area coverage independently verifiable rather than relying on the producer's output to validate itself.

Acceptance tests

Insert an invalid identifier for each of the five scope kinds. Confirm that each is rejected by either runtime conformance or the explicitly documented producer build gate.

---

5. Treat TRUST_GATE and TRUST_MAP as one role during duplicate detection

Why this matters

The manifest treats TRUST_GATE and TRUST_MAP as aliases, but the duplicate-role conformance query groups them as different strings. Both can therefore be active for one product without violating the current duplicate check. The manifest then uses MAX(...) across the two rows and can select an entrypoint based on string ordering rather than an intentional precedence rule.

Relevant code:

• Duplicate-role check
• Manifest alias projection

Suggested fix

Canonicalise the legacy spelling before grouping:

SELECT
      c.product_id
    , c.canonical_resource_role
    , COUNT(*) AS n
FROM (
    SELECT
          o.product_id
        , CASE
              WHEN o.resource_role IN ('TRUST_MAP', 'TRUST_GATE')
                THEN 'TRUST_MAP'
              ELSE o.resource_role
          END AS canonical_resource_role
    FROM {{ product }}_Semantic.data_product_orientation AS o
    WHERE o.is_active = 1
) AS c
GROUP BY c.product_id, c.canonical_resource_role
HAVING COUNT(*) > 1;

Also make manifest precedence deterministic during the compatibility window:

, COALESCE(
      MAX(CASE
              WHEN o.resource_role = 'TRUST_MAP'
                THEN o.fully_qualified_object_name
          END),
      MAX(CASE
              WHEN o.resource_role = 'TRUST_GATE'
                THEN o.fully_qualified_object_name
          END)
  )

This prefers the canonical spelling while continuing to read an existing legacy-only product. The duplicate check should still report a product that publishes both.

Acceptance tests

• A product with only TRUST_GATE remains readable.
• A product with only TRUST_MAP remains readable.
• A product with both fails the singular-role conformance check.
• If the manifest is queried before conformance is corrected, it deterministically prefers TRUST_MAP.

Test coverage requested

No tests were added in this PR for the new projection and compatibility semantics. I recommend adding focused contract tests—ideally in tooling/validation/tests/test_validation_trust_map.py—covering:

1. mixed-age areas from different runs;
2. explicit expiry and the default seven-day window;
3. a legacy run with no area records;
4. a malformed 2.1 run with no area records;
5. a requested scope with no published entry;
6. invalid identifiers for every scope kind;
7. simultaneous TRUST_GATE and TRUST_MAP orientation rows.

The tests do not necessarily need a live Teradata instance. The selection, fallback, staleness, and canonicalisation rules can be exercised against small golden fixtures in a platform-neutral projection model, with separate checks ensuring the Teradata SQL and documentation declare the same behaviour.

Verification performed during review

• python tooling/validation/design_lint.py design implementation — clean.
• python tooling/skill/verify_skill.py — package conforms.
• GitHub reported no status checks or workflow runs on the reviewed head commit.
• The unit suite attempted 100 tests: 78 passed, 3 skipped, and 19 could not run because the review sandbox denied Python temporary-directory creation. Those 19 were environmental errors rather than failed assertions.

Recommendation

The conceptual model is sound, and the documentation changes are mostly consistent with the objective. I recommend merging after the five read-path and conformance issues above are addressed and protected by focused fixtures. The important outcome is that every area a consumer touches produces an explicit, correctly aged trust statement—even when that statement is “unknown”.

[dtehan-td]

The idea of an on/off switch for a data set that is enforced within the view is good, this allows me to turn off data to all agents quickly (valuable if the data has been poisoned). Seems that this is something that I should be able to do at the data product level and at the individual table level. If those flags are available in the initial data product discovery queries, we can use the flags in agent hooks to have a second layer of security in case the view was not set up right. In short a kill switch is a great idea, both at the agent (LLM Key can be deactivated) and at the data (view shuts off access).