Skip to content

[codex] add prod-like oracle comparator#43

Merged
InfoSecHack merged 1 commit into
mainfrom
codex/prod-like-oracle-comparator
Jun 4, 2026
Merged

[codex] add prod-like oracle comparator#43
InfoSecHack merged 1 commit into
mainfrom
codex/prod-like-oracle-comparator

Conversation

@InfoSecHack

Copy link
Copy Markdown
Owner

Summary

  • add local-only prod-like oracle comparator for IAMScope findings JSON
  • compare frozen oracle rows to emitted findings using explicit deterministic v1 mappings
  • separate oracle matches, mismatches, missing/not-currently-live-comparable rows, unsupported static-only rows, environmental extras, and unmapped sandbox extras
  • write sanitized comparison-summary.md and comparison.json outside the repository tree

Observed local comparison against /tmp collection

  • oracle rows: 24
  • emitted findings: 20
  • sandbox-source findings: 8
  • environmental extras: 12
  • unmapped sandbox extras: 2
  • oracle matches: 5
  • oracle mismatches: 1
  • not currently live comparable: 14
  • unsupported static-only: 4
  • emitted verdicts: 3 validated, 5 blocked, 12 inconclusive
  • emitted patterns: 16 passrole_lambda, 4 passrole_ecs

Boundaries

  • no live AWS
  • no Terraform
  • no AWS CLI, STS, Lambda, API, or iam:PassRole calls
  • no raw live findings committed
  • no raw account IDs or IAM ARNs written to comparison outputs
  • no composite score or pass/fail benchmark label

Validation

  • python -m pytest -q tests/test_prod_like_oracle_comparison.py: 7 passed
  • python scripts/compare_prod_like_oracle_findings.py --oracle tests/fixtures/prod_like/aws_accuracy_oracle_v1/oracle_rows.json --findings /tmp/iamscope-prodlike-collection/findings.json --out /tmp/iamscope-prodlike-oracle-comparison: passed
  • cat /tmp/iamscope-prodlike-oracle-comparison/comparison-summary.md: inspected
  • jq . /tmp/iamscope-prodlike-oracle-comparison/comparison.json: passed
  • account/ARN hygiene greps: clean
  • Terraform/live artifact scan: clean
  • ./scripts/check.sh: passed
  • ./scripts/test_fast.sh: 1930 passed
  • git diff --check: passed

@InfoSecHack InfoSecHack marked this pull request as ready for review June 4, 2026 18:19
@InfoSecHack InfoSecHack merged commit 23adf5d into main Jun 4, 2026
6 checks passed
@InfoSecHack InfoSecHack deleted the codex/prod-like-oracle-comparator branch June 4, 2026 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant