Repository navigation
[#13945] fix(jdbc): Honor pool wait settings and validate MySQL table properties - #13946
Merged
Merged
Conversation
Code Coverage Report
Files
|
MySQL now rejects table properties it does not support, so the shared Flink ITs only send arbitrary WITH options to catalogs that accept them.
Apply the canonical pool wait only when it is configured or no DBCP maxWaitMillis bypass is present, matching maxIdle. List the supported keys in MySQL table property errors and document the pool wait setting and its precedence across the JDBC catalog docs.
CREATE TABLE ... LIKE copies the url, table-name, username and password options the connector adds for scans, and MySQL now rejects them as table properties. Strip them before forwarding table options, as Spark does for owner, and note that shared metadata keys are not applied to MySQL tables.
yuqi1129
added a commit
that referenced
this pull request
Oct 21, 2026
…ngs and validate MySQL table properties (#13946) (#14035) ### What changes were proposed in this pull request? Backport commit `3ce91977a2d137af39321716a9fb5813922afceb` (#13946) to `branch-1.3`: honor JDBC pool wait settings and validate MySQL table properties. Resolve the committed conflict markers while retaining the 1.3 validation query, connection timeout handling and Spark `getPGInstance()` API. Adapt tests to the existing 1.3 ALTER connection lifecycle. Pool max-idle and table credential-vending settings are not introduced by this backport. ### Why are the changes needed? Configured pool wait values could be lost or overwritten, and unsupported MySQL table options could reach generated SQL. This backport brings the fixes to 1.3. ### Does this PR introduce _any_ user-facing change? Yes. JDBC pool wait settings are honored, and unsupported or invalid MySQL table options are rejected before DDL is executed. ### How was this patch tested? - 70 targeted non-Docker tests passed across JDBC common, MySQL, Flink common and Spark common. - Formatting and conflict-marker checks passed. - Six mutation cases fail if the retained `SELECT 1` pool validation query is removed, then pass after restoration. - Docker tests were not run locally because the Docker daemon is unavailable; CI validation is pending. --------- Co-authored-by: Qi Yu <yuqi@datastrato.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
jdbc.pool.max-wait-msso it reaches the JDBC connection pool. As withjdbc.pool.max-idle, an explicit canonical value takes precedence overgravitino.bypass.maxWaitMillis; the canonical default no longer overrides a bypassedmaxWaitMillis.ownermetadata in the JDBC connector before forwarding table options, preserving normal Spark table creation under strict MySQL validation.url,table-name,username,password) before forwarding table options, soCREATE TABLE ... LIKEkeeps working and the catalog password is no longer sent as a table property. Same-named options written explicitly inWITHare dropped too; they never took effect, because scan options are always regenerated from the catalog connection.WITHoptions only to catalogs that accept custom table properties; the MySQL Flink IT opts out.jdbc.pool.max-wait-msand its bypass precedence in every JDBC catalog page (adds the missing Hologres row).Why are the changes needed?
The canonical pool-wait setting currently falls back to
30000ms. Unknown MySQL table properties are accepted and then discarded by mapping on both create and alter. This patch corrects these two property-handling paths; it does not claim that unknown properties bypass immutability through the public API.Fix: #13945
Does this PR introduce any user-facing change?
Configured pool wait times take effect, whether set through
jdbc.pool.max-wait-msorgravitino.bypass.maxWaitMillis. Values that were previously ignored now apply after upgrade: a stored non-positivejdbc.pool.max-wait-msfails catalog initialization, and a bypassedmaxWaitMillis=-1means an unbounded wait. MySQL create/alter requests with unknown table properties now fail with an error naming the key. Supported values are validated before DDL. Shared metadata keys that every entity accepts (the internal identifier and credential-vending keys) are still accepted and not turned into MySQL options, since they are not table options.engineandauto-increment-offsetremain immutable, and strict unknown-key validation is scoped to MySQL. Spark JDBC no longer forwards its reservedownermetadata as a table option; other user-supplied keys continue to reach the server for catalog-specific handling.How was this patch tested?
maxWaitMillisbypass combinations (6 cases); reverting the precedence guard fails the bypass-only case.CREATE TABLE ... LIKEcase inFlinkJdbcMysqlCatalogIT119fails without the fix (Unsupported MySQL table property: password) and passes with it.FlinkJdbcMysqlCatalogIT119with Docker MySQL: 30 tests, 0 failures, including the LIKE case and the four create-table cases that previously sent unsupported options.