Skip to content

Gate system enrichment API by per-OpDiv insights flag#359

Merged
danielbowne merged 2 commits into
mainfrom
feature/enrichment-opdiv-gate
Jun 23, 2026
Merged

Gate system enrichment API by per-OpDiv insights flag#359
danielbowne merged 2 commits into
mainfrom
feature/enrichment-opdiv-gate

Conversation

@danielbowne

Copy link
Copy Markdown
Collaborator

Summary

Gates the existing system enrichment read endpoint (GET /api/v1/systemenrichment/{fisma_uuid}) on a new per-OpDiv capability flag, so ZTMF Insights enrichment is served only for OpDivs that have an insights pipeline feeding them. Today that is CMS only; enabling another OpDiv later is a single database update, no code change.

Why

Enrichment data (system metadata from the ZTMF Insights pipeline) currently exists only for CMS, but the read endpoint returned a payload for any system that had a row, with no notion of which OpDivs are entitled to the feature. This adds that boundary and makes it data-driven so the feature can extend to other OpDivs as an add-on.

Changes

  • Migration 0042: adds opdivs.insights_enabled (boolean, default false), backfills it true for the active CMS OpDiv. Down migration drops the column.
  • Model: OpDiv.InsightsEnabled *bool (pointer so an update that omits it does not reset it), conditional set on save. FindSystemEnrichment returns a row only when the system's fismauid maps to an OpDiv with insights_enabled = TRUE (an EXISTS predicate, which avoids fanning out on the non-unique fismauid).
  • Controller: resolves the system first, then authorizes against its OpDiv (CanAccessFismaSystem), replacing the prior per-system-only check. A system in a non-enabled OpDiv returns 404, identical to a system with no enrichment row, so the gate never reveals which systems exist.
  • Tests: model integration test for the gate, plus an Emberfall E2E case proving a system with an enrichment row in a disabled OpDiv returns 404.
  • OpenAPI regenerated from annotations.

Behavior and compatibility

  • Backwards compatible. Additive schema change; deploys to dev on PR, prod on merge.
  • Enrichment is only hidden for systems in OpDivs that are not insights-enabled. The pipeline only feeds CMS, and the migration enables CMS, so existing enrichment continues to be served unchanged.
  • Minor authorization tightening: OpDiv-scoped admins now read enrichment only for systems in their granted OpDivs (previously any admin tier could read any system's enrichment). Unscoped admins (OWNER, HHS) and assigned ISSOs are unaffected.

Testing

Full local suite passes: unit, integration (including the new gate test), and E2E (Emberfall). Validated end to end against real enrichment data in the impl environment.

Issues

Implements CMS-Enterprise/ztmf-misc#85 (API endpoints for SDL enrichment data). The read surface landed in #211; this PR adds the OpDiv-conditional gate that completes the consumption API. The issue's original per-function response shape is superseded by the generic jsonb payload (the pipeline owns structure). It will be closed on merge (cross-repo keywords do not auto-close from this repository).

…led flag

Adds opdivs.insights_enabled (migration 0042, default false, CMS backfilled
true) and gates the existing GET /systemenrichment/{fisma_uuid} endpoint on it:
enrichment is served only for systems whose OpDiv has insights enabled. CMS-only
today; enabling another OpDiv later is a single UPDATE, no code change.

- model: InsightsEnabled *bool on OpDiv with conditional Save and updated
  RETURNING. FindSystemEnrichment joins fismasystems -> opdivs and requires
  insights_enabled, so a non-enabled OpDiv returns ErrNoData (404),
  indistinguishable from a missing row (no existence leak).
- controller: OpDiv-aware fetch-then-gate via FindFismaSystemByUUID +
  CanAccessFismaSystem, replacing the blanket admin-read bypass so OpDiv-scoped
  admins no longer read other OpDivs' enrichment.
- tests: model gate integration test and an Emberfall case proving a REBELLION
  system with an enrichment row but insights disabled returns 404.
- openapi regenerated from annotations.
@danielbowne danielbowne added hhs For HHS ZT work zt-insights ZTMF Insights / SDL Snowflake enrichment pipeline work labels Jun 22, 2026
@danielbowne danielbowne self-assigned this Jun 22, 2026
MackOverflow
MackOverflow previously approved these changes Jun 22, 2026

@MackOverflow MackOverflow left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clean, this can be done later (not an approval blocker) but 1 nit:

FindFismaSystemByUUID has no LIMIT 1 — the comment acknowledges fismauid is not schema-unique and says "first match is sufficient," but without a LIMIT the query could return multiple rows and fail at the queryRow scan level rather than deterministically returning the first. A LIMIT 1 would make the behavior explicit and avoid a potential runtime error if duplicates ever appear.
// backend/internal/model/fismasystems.go
sqlb := stmntBuilder.
Select(fismaSystemColumns...).
From("fismasystems").
Where("LOWER(fismauid) = LOWER(?)", fismaUUID).
Limit(1) // add this

Maybe we aren't making this explicit on purpose, but it does leave that limit open.

fismauid is not schema-unique. The lookup feeds CollectOneRow, which
takes the first row and discards the rest, so a duplicate would return
an arbitrary row. LIMIT 1 makes the single-row intent explicit at the
query level instead of relying on driver row-discarding behavior.

Addresses review nit on #359.

@MackOverflow MackOverflow left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clean

@danielbowne danielbowne merged commit 7c0abb9 into main Jun 23, 2026
8 checks passed
@danielbowne danielbowne deleted the feature/enrichment-opdiv-gate branch June 23, 2026 00:07
danielbowne added a commit that referenced this pull request Jun 23, 2026
#351) (#361)

## Summary

Routine refresh of the `impl` branch from `main` so the experimental
environment does not drift from production code. Merge (not rebase), per
the impl-sync policy, because `impl` is a shared, deployed branch.

## What main brought in

The bulk of main's recent history (OpDiv-scoped RBAC, dual-IdP
scaffolding, pgxpool, OpenAPI autogeneration, the N+1 users-list fix)
was already present on impl from the previous sync, so the net new
content here is small:

- Scores diff endpoint to compare two data calls (#356)
- Okta JWT `kid` validation and hardened key-fetch client (#354)
- Route `/login/*` through CloudFront to the ALB for Entra login (#351)
- The enrichment OpDiv gate as it landed on main (#359), reconciled with
impl's own copy (#358)

## Conflict resolution

- **Makefile** - kept impl's per-environment port scaffolding
(`ZTMF_ENV` detection, parametrized `TEST_DB_PORT`). Reverting to main's
hardcoded ports would break side-by-side worktree runs.
- **fismasystems.go / scores.go / openapi.yaml** - took main's canonical
versions (main's `FindFismaSystemByUUID` adds an explicit `LIMIT 1`;
scores-diff is additive). OpenAPI regenerated from annotations
afterward, no drift.

## Environment safety

- Entra remains dormant on impl: there is no impl tfvars override, so
`entra_enabled` stays at its default of false. Main's #350 only enabled
it in dev.
- No new migrations introduced by this sync beyond what is already
merged on main.

## Testing

Full local suite on the impl port set (unit, integration,
E2E/Emberfall).

---------

Co-authored-by: cameron testerman <11036339+voidspooks@users.noreply.github.com>
Co-authored-by: MackOverflow <114018913+MackOverflow@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hhs For HHS ZT work zt-insights ZTMF Insights / SDL Snowflake enrichment pipeline work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants