Skip to content

fix: complete Neptune and Postgres state cutover#186

Merged
jonathanhaaswriter merged 29 commits intomainfrom
fix/writer-main-neptune-pg-cutover-20260330
Apr 10, 2026
Merged

fix: complete Neptune and Postgres state cutover#186
jonathanhaaswriter merged 29 commits intomainfrom
fix/writer-main-neptune-pg-cutover-20260330

Conversation

@jonathanhaaswriter
Copy link
Copy Markdown
Collaborator

Summary

  • remove local snapshot read fallbacks so graph reads stay on the live/configured runtime path
  • migrate app-state persistence and Snowflake backfill paths to Postgres while preserving the legacy Snowflake migration source
  • sync generated config env docs and stabilize time-windowed graph intelligence API tests

Validation

  • env GOFLAGS=-mod=vendor go -C "/Users/jonathan/Documents/cerebro-worktrees/writer-main-neptune-pg-cutover-20260330" test ./internal/app ./internal/appstate ./internal/agents ./internal/findings ./internal/api ./internal/cli ./internal/snowflake ./internal/remediation ./scripts/generate_agent_sdk_packages -count=1
  • python3 scripts/devex.py run --mode changed --base-ref writer/main
  • python3 scripts/devex.py run --mode pr --base-ref writer/main

Comment thread internal/app/app_graph_status.go
Comment thread internal/app/app_state_postgres.go Outdated
Comment thread internal/findings/postgres_store.go
Comment thread internal/app/app_state_postgres.go
Comment thread internal/app/app_init_agents.go
Comment thread internal/findings/postgres_store.go Outdated
Comment thread internal/app/app_secrets.go
Test added 2 commits April 1, 2026 10:30
- CurrentSecurityGraph: return nil when configured backend errors instead
  of falling back to empty live graph; preserve live-graph fallback for
  in-memory mode (no configured backend)
- Findings Load: remove LIMIT 10000 so older unresolved findings survive
  restart; 30-day resolved cutoff still bounds the result set
- syncMutation: log Sync errors via slog instead of silently discarding
- migrateAgentSessions: guard against missing agent_sessions table in
  legacy Snowflake (matches migrateFindings pattern)
- Agent toolset: fall back to LegacySnowflake when Snowflake is nil in
  cutover mode so query_assets/get_asset_context continue working
- Secret rotation: skip Snowflake rotation when warehouse backend is not
  snowflake and credentials are absent, instead of hard-failing
@jonathanhaaswriter jonathanhaaswriter force-pushed the fix/writer-main-neptune-pg-cutover-20260330 branch from 28ae404 to 83a99ea Compare April 1, 2026 21:37
@jonathanhaaswriter
Copy link
Copy Markdown
Collaborator Author

jonathanhaaswriter commented Apr 6, 2026

Pushed 7425303 to cover the remaining cutover review regressions. The branch already includes the legacy Postgres findings import, legacy Snowflake agent-session migration fallback, LegacySnowflake tool wiring, and error-propagating Postgres finding mutations; this follow-up adds regression coverage for configured-graph view errors, non-truncated Postgres finding loads, and dropping Snowflake credentials on Postgres secret reloads.

@jonathanhaaswriter
Copy link
Copy Markdown
Collaborator Author

Audited the outstanding review feedback on the current branch head and confirmed the requested fixes are already present: configured-graph errors now return nil instead of silently falling back, legacy Postgres findings import is preserved, legacy Snowflake agent-session migration still backfills transcript history, agent tools fall back to LegacySnowflake in cutover mode, Postgres findings no longer truncate the load window, and mutation sync errors propagate instead of reporting false success. I also reran python3 ./scripts/devex.py run --mode changed --base-ref writer/main locally on fix/writer-main-neptune-pg-cutover-20260330 and it passed end-to-end. @josh-writer could you take another pass when you have a moment?

…ne-pg-cutover-20260330

# Conflicts:
#	internal/api/server_handlers_graph_intelligence_test.go
Comment thread internal/app/app_state_postgres.go
Comment thread internal/app/app_secrets.go
@jonathanhaaswriter
Copy link
Copy Markdown
Collaborator Author

Pushed 5ccb0acef to finish the remaining follow-up work on this branch. In addition to the earlier cutover review fixes, this update swaps the Postgres database/sql driver from pgx stdlib to lib/pq, bumps the Go toolchain line to go1.26.2, refreshes vendored deps, and clears the new govulncheck blocker. I reran python3 ./scripts/devex.py run --mode changed --base-ref writer/main locally on the branch and it passed end-to-end; the fresh CI run is now in flight. @josh-writer could you take another pass when you have a moment?

     Preserve the legacy Snowflake source when cutover migration or retention still depends on it, rebuild agent tooling/session state safely on reload, and make Postgres findings migration/schema/dedup updates durable. Add regression tests and changed-file staticcheck cleanup for the cutover paths.
…owflake

fix(app): harden pg cutover migration and agent tooling refresh
@jonathanhaaswriter jonathanhaaswriter merged commit e75f8ae into main Apr 10, 2026
30 of 31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants