Skip to content

[#14000] docs(test): Add the test container framework design - #14005

Open
nevzheng wants to merge 5 commits into
apache:mainfrom
nevzheng:claude/13900-tc-01-design
Open

nevzheng wants to merge 5 commits into
apache:mainfrom
nevzheng:claude/13900-tc-01-design

Conversation

@nevzheng

@nevzheng nevzheng commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

What changes were proposed in this pull request?

  • Adds design-docs/testing/containers/test-container-framework-design.md: background from the [Improvement] Audit and improve test container usage efficiency #13900 audit, goals, options considered, the proposal and a task breakdown that maps to this PR series. Appendix A has the measurements.
  • Adds design-docs/testing/.rumdl.toml (100-column wrap, aligned tables) for docs in this folder.

This PR includes the commits of the PRs below it. Review only its own commit: the whole PR.

Why are the changes needed?

The test container library has lifecycle bugs, fixed waits, restarts and leaks (#13900). This doc agrees the direction before the framework PRs (#14002) land.

Fix: #14000

Does this PR introduce any user-facing change?

No. Test code only.

How was this patch tested?

rumdl check -c design-docs/testing/.rumdl.toml design-docs/testing passes.

Stack: #14103 → #14083 → #14084 → #14085 → #14005 → #14006 → #14052 → #14076 → #14007 → #14008 → #14009 → #14010 → #14011 → #14012 → #14013 → #14079 → #14077 → #14078 → #14086 → #14087 → #14088 → #14089 → #14090 → #14091 → #14094

@nevzheng

nevzheng commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator Author

Timing results, 2026-10-20 (local, preliminary)

Measured on an Apple-silicon Mac (OrbStack Docker), images already pulled, one Gradle invocation per module, main and the stack alternating. Medians of 5 rounds per side; same tests and pass counts on both sides. Apache CI numbers are not in yet.

Main against the stack (#14005–#14012)

Module Tests main stack change
idp-basic 221 488 s 102 s −79 %
hive-metastore-common 52 394 s 175 s −56 %
flink-1.20 439 426 s 263 s −38 %
catalog-kafka 25 34 s 22 s −35 %
catalog-hive 308 578 s 395 s −32 %
catalog-fileset 436 253 s 237 s −6 %
spark-3.5 581 482 s 452 s −6 %

Modules that start few containers (fileset, Spark) gain little; most of their time is the tests themselves.

Disposable data (#14013), durable against disposable, 2–3 rounds

Module Containers durable disposable change
idp-basic MySQL, PostgreSQL 77 s 34 s ≈ −55 %
catalog-jdbc-mysql MySQL 55 s 43 s ≈ −22 %
hive-metastore-common Hive 173 s 171 s ≈ 0 %
catalog-hive Hive 391 s 388 s ≈ 0 %

What this includes: container start timing, the lifecycle and readiness fixes, the fixture framework and the caller migrations. The appendix of the design doc (#14005) has the per-fix numbers.

Not included yet: Apache CI runners and the Hive readiness change just pushed to #14006 (it drops two duplicate Hive CLI checks per start).

Other opportunities that could be explored (not committed to):

  • Wait strategies for Hive, Doris, Ranger, StarRocks and RustFS after the Testcontainers 2.x upgrade ([Improvement] Upgrade Testcontainers from 1.21.4 to 2.0.x #13958).
  • Disposable data for Doris and StarRocks, which needs image changes.
  • Measuring the same modules on Apache CI.
  • Sharding test classes across CI jobs, if a single job grows long enough for test time to dominate.
  • A shared per-test database reset outside core, if a module turns out to need one.

@nevzheng
nevzheng force-pushed the claude/13900-tc-01-design branch 2 times, most recently from 4db4a53 to 2d913ee Compare October 20, 2026 15:02
@github-actions

Copy link
Copy Markdown

Code Coverage Report

Overall Project 72.33% 🟢
Files changed No Java source files changed -

Module Coverage
aliyun 19.74% 🔴
api 55.09% 🟢
authorization-common 85.96% 🟢
authorization-ranger 14.44% 🔴
aws 60.78% 🟢
aws-bundle 0.0% 🔴
azure 68.04% 🟢
azure-bundle 0.0% 🔴
catalog-common 29.49% 🔴
catalog-fileset 82.37% 🟢
catalog-glue 73.24% 🟢
catalog-hive 84.5% 🟢
catalog-jdbc-common 58.22% 🟢
catalog-jdbc-doris 87.94% 🟢
catalog-jdbc-mysql 84.91% 🟢
catalog-jdbc-postgresql 83.76% 🟢
catalog-jdbc-starrocks 79.72% 🟢
catalog-kafka 75.21% 🟢
catalog-lakehouse-generic 86.91% 🟢
catalog-lakehouse-hudi 79.1% 🟢
catalog-lakehouse-iceberg 86.8% 🟢
catalog-lakehouse-paimon 84.36% 🟢
catalog-model 77.99% 🟢
cli 44.62% 🟢
client-java 79.44% 🟢
client-java-runtime 0.0% 🔴
common 60.9% 🟢
core 87.57% 🟢
filesystem-hadoop3 76.89% 🟢
flink 55.56% 🟢
flink-common 32.39% 🔴
flink-runtime 0.0% 🔴
gcp 32.2% 🔴
hadoop-auth 68.0% 🟢
hadoop-common 17.84% 🔴
hive-metastore-common 62.3% 🟢
hive-metastore3-libs 58.8% 🟢
iceberg-aliyun-bundle 0.0% 🔴
iceberg-common 67.14% 🟢
iceberg-rest-server 77.45% 🟢
idp-basic 87.03% 🟢
integration-test-common 0.0% 🔴
jobs 71.35% 🟢
k8s-job-executor 97.84% 🟢
lance-common 40.73% 🟢
lance-rest-server 68.04% 🟢
lineage 59.39% 🟢
optimizer 84.2% 🟢
optimizer-api 46.58% 🟢
server 91.44% 🟢
server-common 84.01% 🟢
spark 63.45% 🟢
tencent 81.78% 🟢
tms 80.04% 🟢
trino-connector 64.88% 🟢

@nevzheng
nevzheng force-pushed the claude/13900-tc-01-design branch from f350e94 to 1509507 Compare October 22, 2026 04:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Subtask] Design the test container framework

1 participant