Skip to content

[codex] add prod-like oracle comparison triage notes#44

Merged
InfoSecHack merged 1 commit into
mainfrom
codex/prod-like-oracle-triage-notes
Jun 4, 2026
Merged

[codex] add prod-like oracle comparison triage notes#44
InfoSecHack merged 1 commit into
mainfrom
codex/prod-like-oracle-triage-notes

Conversation

@InfoSecHack

Copy link
Copy Markdown
Owner

Summary

  • keep oracle-i-001 as an oracle_mismatch when expected inconclusive emits blocked
  • add a deterministic triage note explaining complete-confidence boundary evidence and likely oracle/fixture expectation conflict
  • keep uncertainty-probe blocked extras as unmapped_sandbox_extra and add an explanatory triage note
  • preserve environmental_extra and unsupported_static_only behavior

Observed local comparison after rerun

  • oracle matches: 5
  • oracle mismatches: 1
  • environmental extras: 12
  • unmapped sandbox extras: 2
  • not currently live comparable: 14
  • unsupported static-only: 4
  • emitted verdicts: 3 validated, 5 blocked, 12 inconclusive
  • emitted patterns: 16 passrole_lambda, 4 passrole_ecs

Boundaries

  • no live AWS
  • no Terraform
  • no AWS CLI, STS, Lambda, API, or iam:PassRole calls
  • no raw live findings committed
  • no score or pass/fail benchmark label

Validation

  • python -m pytest -q tests/test_prod_like_oracle_comparison.py: 7 passed
  • comparator rerun against /tmp/iamscope-prodlike-collection/findings.json: passed
  • cat comparison-summary.md: inspected
  • jq mismatch/unmapped extras view: inspected
  • account/ARN hygiene greps: clean
  • Terraform/live artifact scan: clean
  • ./scripts/check.sh: passed
  • ./scripts/test_fast.sh: 1930 passed
  • git diff --check: passed

@InfoSecHack InfoSecHack marked this pull request as ready for review June 4, 2026 18:57
@InfoSecHack InfoSecHack merged commit 64ce0e9 into main Jun 4, 2026
6 checks passed
@InfoSecHack InfoSecHack deleted the codex/prod-like-oracle-triage-notes branch June 4, 2026 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant