Repository navigation
Conversation
This was referenced Oct 21, 2026
Open
Open
Open
Open
Open
[#14049] improvement(test): Replace the remaining container CLI calls with in-process clients
#14076
Draft
Draft
nevzheng
force-pushed
the
claude/13900-tc-13-per-run-names
branch
from
October 21, 2026 17:20
e8505c3 to
45faf64
Compare
This was referenced Oct 21, 2026
Draft
Draft
nevzheng
force-pushed
the
claude/13900-tc-13-per-run-names
branch
6 times, most recently
from
October 21, 2026 19:27
667a880 to
2faff71
Compare
nevzheng
force-pushed
the
claude/13900-tc-13-per-run-names
branch
9 times, most recently
from
October 22, 2026 03:28
ed6520e to
cebda95
Compare
…st libraries each module uses
…s built-in store cleanup
Design doc for apache#13900 and a rumdl config for design-docs/testing.
…e ContainerSuite lifecycle - Log every container start with pull, start and readiness times; add test log4j configs so the lines show. - Reset all ContainerSuite state in close(); key Hive by version; make the PostgreSQL retry wait; pin StarRocks; drop a dead start method. - Poll readiness with no initial delay; run cleanDisk() only on real starts. - idp-basic starts its containers once per run; Ryuk back on, unprivileged. - Drop the suite-wide closes in Hive2, Flink and Spark ITs.
…ve readiness without the CLI
… calls with in-process clients
nevzheng
force-pushed
the
claude/13900-tc-13-per-run-names
branch
from
October 22, 2026 04:29
cebda95 to
95725f9
Compare
…est containers (part 1/6) ContainerSpec, ContainerExtension and @FixtureContainer: a test declares a container with a fixture annotation and receives it as a parameter or field. Fixtures for Kafka, LocalStack, RustFS, StarRocks, OceanBase and Doris, with their callers migrated.
…o, Ranger and Hive (part 2/6) - TestImages declares every test image in one place. - BaseContainer subclasses can supply their own container (ready for the official Testcontainers modules). - Moto, Ranger and Hive (version, auth) fixtures. - A container whose start fails is closed instead of leaking.
…add ClickHouse fixtures and scope (part 3/6) - client-java, fileset, Hudi, Iceberg, Paimon, Iceberg REST, GVFS, Flink, Spark and Trino env tests use @HiveFixture. - ClickHouse (1 or 3 nodes) and ZooKeeper fixtures. - scope = RUN | CLASS | METHOD; HadoopUserAuthenticationIT gets a private Kerberos Hive.
…and PostgreSQL fixtures (part 4/6) - Review follow-ups from the first fixture changes. - A fixture annotation on a field picks that field's container. - startup = EAGER | LAZY; a warning for fixtures no test injects. - MySQL and PostgreSQL fixtures; SparkQueryRunner starts Hive through its spec. - Only the run's closer closes the container suite. - Split container start time into pull, start and readiness; give the Hive metastore check a real time budget.
…remaining Hive variants (part 5/6) - ContainerSpec.dependencies(): dependencies start first, deepest first, with a cycle check. - ClickHouseCluster injected for the three-node cluster. - Hive auth = RANGER and storage = S3 | GCS | ABS; the CatalogHive IT family and the Ranger ITs use them. - HiveUserAuthenticationIT on the Kerberos Hive fixture.
…th TrinoCluster (part 6/6) TrinoCluster starts the Trino stack through Testcontainers instead of launch.sh and docker compose, with real readiness checks and Ryuk cleanup. @TrinoFixture starts lazily and reads the Gravitino server port from the test.
…ners per fixture - durability = DISPOSABLE (default) | DURABLE on every fixture; each container applies its own recipe at start. - Hive keeps HDFS on tmpfs and its metastore MySQL skips per-commit flushes, for consistency (no measurable gain on a local Mac); ZooKeeper sets forceSync=no; MySQL and PostgreSQL use apache#13899's settings everywhere. - Ranger keeps its policies in the shared MySQL. - -PdurableTestContainers makes every container durable, for debugging. Builds on apache#13899.
…-private shared containers Every test now gets its containers through a spec. Test classes declare fixture annotations; code that cannot carry one (BaseIT's Gravitino backend, which the core database lanes use, idp-basic's backend parameter, RangerITEnv, SparkQueryRunner) calls Fixtures.get(spec), and tests that build a container of their own use Fixtures.network(). ContainerSuite becomes the package-private SharedContainers, used only by specs and ContainerExtension: the same images, environment, host names, network, one container per key per run, run-end close and timing lines. Its unused starts (the single Trino container, default Doris and PostgreSQL overloads) go away; disabled TrinoConnectorIT builds its Trino container itself. TestContainerSuite becomes TestSharedContainers.
…ork per-run names
nevzheng
force-pushed
the
claude/13900-tc-13-per-run-names
branch
from
October 22, 2026 04:46
95725f9 to
012780c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
gravitino-<service>-<runId>-<n>), using only characters valid in a host name./tmp/obdata.In-network host names and aliases are unchanged (the Kerberos image derives its principals from its host name). On macOS the connector still routes one fixed subnet, so a second run on the same Mac fails fast instead of breaking the first.
This PR includes the commits of the PRs below it. Review only its own commit: nevzheng/gravitino@claude/13900-tc-12-no-container-suite...claude/13900-tc-13-per-run-names
Why are the changes needed?
Fixed names make two runs on one Docker host collide, and Docker's generated names contain underscores, which Hadoop and Trino clients reject when they resolve a container's IP back to a name.
Fix: #14002
Does this PR introduce any user-facing change?
No. Test code only.
How was this patch tested?
Unit tests for the run id and names; Doris, Trino connector, Kerberos Hive, JDBC MySQL and OceanBase ITs. timing in progress; numbers follow
Stack: #14103 → #14083 → #14084 → #14085 → #14005 → #14006 → #14052 → #14076 → #14007 → #14008 → #14009 → #14010 → #14011 → #14012 → #14013 → #14079 → #14077 → #14078 → #14086 → #14087 → #14088 → #14089 → #14090 → #14091 → #14094