ENG-2635: Add namespace support for Google Cloud SQL Postgres#7507
Open
ENG-2635: Add namespace support for Google Cloud SQL Postgres#7507
Conversation
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
JadeCara
pushed a commit
that referenced
this pull request
Feb 26, 2026
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add GoogleCloudSQLPostgresNamespaceMeta with database_name (optional) and schema (required), following the same pattern as Snowflake/BigQuery. Update GoogleCloudSQLPostgresQueryConfig with generate_table_name() for schema-qualified SQL, and GoogleCloudSQLPostgresConnector to fetch namespace_meta from DB and pass it to the query config. Also includes shared fixes from PR #7500: - Guard against None db session in get_namespace_meta - Guard against None secrets in namespace validation - Skip namespace validation for mismatched connection types Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
e0bef11 to
b71e285
Compare
Contributor
Greptile SummaryThis PR adds namespace metadata support to the Google Cloud SQL Postgres connector, enabling schema-qualified and optionally database-qualified table names in generated SQL queries. The implementation closely follows the existing Snowflake connector pattern. Key Changes:
Test Coverage:
Observations:
Confidence Score: 4/5
Important Files Changed
Last reviewed commit: 03e297e |
10 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ticket ENG-2635
Description Of Changes
Add namespace metadata support to the Google Cloud SQL for Postgres connector, enabling schema-qualified and optionally database-qualified table names in generated SQL queries. This is PR 2 of the ENG-2635 work — PR 1 (#7500) added the same capability for standard Postgres.
When a dataset has
fides_meta.namespaceconfigured with aschema(and optionallydatabase_name), queries are generated with qualified table names like"billing"."customer"or"prod_db"."billing"."customer"instead of just"customer".The
set_schemaconnection secret (db_schema) path remains as the legacy approach. The two mechanisms are mutually exclusive in practice — users configure one or the other.This is PR 2 of 3
This PR also includes the cross-type namespace validation fixes from PR 1 (needed on this branch since it diverges from main before PR 1 merges). #7500
Code Changes
src/fides/api/schemas/namespace_meta/google_cloud_sql_postgres_namespace_meta.py- New namespace meta schema with requiredschemafield, optionaldatabase_name, and empty fallback secret fields (GCS Postgres defaults to public schema)src/fides/api/service/connectors/query_configs/google_cloud_postgres_query_config.py- Addednamespace_meta_schema,generate_table_name(),get_formatted_query_string(), andget_update_stmt()to generate schema-qualified SQLsrc/fides/api/service/connectors/google_cloud_postgres_connector.py- Addedquery_config()override to fetch namespace meta from DB,get_qualified_table_name()for table existence checks, clarifying docstring onset_schemasrc/fides/api/service/connectors/sql_connector.py- AddedNoneguard inget_namespace_meta()for cases wheredbisNonesrc/fides/service/dataset/validation_steps/namespace_meta.py- Added cross-type namespace skip logic (explicitconnection_typecheck + field overlap check),Nonesecrets guardclients/admin-ui/src/features/integrations/integration-type-info/googleCloudSQLPostgresInfo.tsx- Added "DSR Automation" tagtests/ops/service/connectors/test_google_cloud_postgres_query_config.py- New integration tests: 5 parametrized query generation tests, 1 invalid meta test, 3 update statement teststests/service/dataset_service/test_namespace_meta_validation.py- Added 5 GCS Postgres validation tests + cross-type skip test, changed unsupported type test to usemariadbSteps to Confirm
1. Run the namespace meta validation unit tests (no Docker required):
nox -s "pytest(misc-unit)" -- tests/service/dataset_service/test_namespace_meta_validation.py -vExpected: All 19 tests pass, including the new GCS Postgres tests:
test_validate_gcs_postgres_without_namespace_or_schema— PASSEDtest_validate_gcs_postgres_with_valid_namespace— PASSEDtest_validate_gcs_postgres_with_connection_defaults— PASSEDtest_validate_gcs_postgres_with_invalid_namespace— PASSEDtest_validate_mismatched_namespace_skipped— PASSED2. Run the query config integration tests (requires Postgres via Docker):
nox -s "pytest(ops-integration)" -- tests/ops/service/connectors/test_google_cloud_postgres_query_config.py -vExpected: All 10 tests pass:
test_generate_query_with_namespace_meta(5 parametrized cases) — queries produce:schema="billing"→SELECT ... FROM "billing"."customer" WHERE (email = :email)database_name="prod_db", schema="billing"→SELECT ... FROM "prod_db"."billing"."customer" WHERE (email = :email){"schema": "billing"}→ same as first caseconnection_type→ same as first caseNone→SELECT ... FROM "customer" WHERE (email = :email)test_generate_query_with_invalid_namespace_meta— raisesValidationErrortest_generate_update_stmt→UPDATE "address" SET ...test_generate_namespaced_update_stmt→UPDATE "billing"."address" SET ...test_generate_namespaced_update_stmt_with_database→UPDATE "prod_db"."billing"."address" SET ...3. Quick local verification (no Docker, no nox):
Expected output:
Pre-Merge Checklist
CHANGELOG.mdupdatedmaindowngrade()migration is correct and works