Repository navigation
Conversation
This was referenced Oct 21, 2026
Open
Open
[#14049] improvement(test): Replace the remaining container CLI calls with in-process clients
#14076
Draft
Open
Open
Open
Draft
Draft
Draft
nevzheng
force-pushed
the
claude/13900-tc-21-kerberos-principals
branch
2 times, most recently
from
October 21, 2026 21:16
04f29e9 to
f2f621c
Compare
nevzheng
force-pushed
the
claude/13900-tc-21-kerberos-principals
branch
3 times, most recently
from
October 21, 2026 21:40
8fc64c9 to
f501d92
Compare
nevzheng
force-pushed
the
claude/13900-tc-21-kerberos-principals
branch
from
October 21, 2026 23:56
6b272da to
d060f4f
Compare
nevzheng
force-pushed
the
claude/13900-tc-21-kerberos-principals
branch
from
October 22, 2026 03:28
d060f4f to
b0e3887
Compare
…st libraries each module uses
…s built-in store cleanup
Design doc for apache#13900 and a rumdl config for design-docs/testing.
…e ContainerSuite lifecycle - Log every container start with pull, start and readiness times; add test log4j configs so the lines show. - Reset all ContainerSuite state in close(); key Hive by version; make the PostgreSQL retry wait; pin StarRocks; drop a dead start method. - Poll readiness with no initial delay; run cleanDisk() only on real starts. - idp-basic starts its containers once per run; Ryuk back on, unprivileged. - Drop the suite-wide closes in Hive2, Flink and Spark ITs.
…ve readiness without the CLI
… calls with in-process clients
nevzheng
force-pushed
the
claude/13900-tc-21-kerberos-principals
branch
from
October 22, 2026 04:29
b0e3887 to
882e073
Compare
…est containers (part 1/6) ContainerSpec, ContainerExtension and @FixtureContainer: a test declares a container with a fixture annotation and receives it as a parameter or field. Fixtures for Kafka, LocalStack, RustFS, StarRocks, OceanBase and Doris, with their callers migrated.
…o, Ranger and Hive (part 2/6) - TestImages declares every test image in one place. - BaseContainer subclasses can supply their own container (ready for the official Testcontainers modules). - Moto, Ranger and Hive (version, auth) fixtures. - A container whose start fails is closed instead of leaking.
…add ClickHouse fixtures and scope (part 3/6) - client-java, fileset, Hudi, Iceberg, Paimon, Iceberg REST, GVFS, Flink, Spark and Trino env tests use @HiveFixture. - ClickHouse (1 or 3 nodes) and ZooKeeper fixtures. - scope = RUN | CLASS | METHOD; HadoopUserAuthenticationIT gets a private Kerberos Hive.
…and PostgreSQL fixtures (part 4/6) - Review follow-ups from the first fixture changes. - A fixture annotation on a field picks that field's container. - startup = EAGER | LAZY; a warning for fixtures no test injects. - MySQL and PostgreSQL fixtures; SparkQueryRunner starts Hive through its spec. - Only the run's closer closes the container suite. - Split container start time into pull, start and readiness; give the Hive metastore check a real time budget.
…remaining Hive variants (part 5/6) - ContainerSpec.dependencies(): dependencies start first, deepest first, with a cycle check. - ClickHouseCluster injected for the three-node cluster. - Hive auth = RANGER and storage = S3 | GCS | ABS; the CatalogHive IT family and the Ranger ITs use them. - HiveUserAuthenticationIT on the Kerberos Hive fixture.
…th TrinoCluster (part 6/6) TrinoCluster starts the Trino stack through Testcontainers instead of launch.sh and docker compose, with real readiness checks and Ryuk cleanup. @TrinoFixture starts lazily and reads the Gravitino server port from the test.
…ners per fixture - durability = DISPOSABLE (default) | DURABLE on every fixture; each container applies its own recipe at start. - Hive keeps HDFS on tmpfs and its metastore MySQL skips per-commit flushes, for consistency (no measurable gain on a local Mac); ZooKeeper sets forceSync=no; MySQL and PostgreSQL use apache#13899's settings everywhere. - Ranger keeps its policies in the shared MySQL. - -PdurableTestContainers makes every container durable, for debugging. Builds on apache#13899.
…-private shared containers Every test now gets its containers through a spec. Test classes declare fixture annotations; code that cannot carry one (BaseIT's Gravitino backend, which the core database lanes use, idp-basic's backend parameter, RangerITEnv, SparkQueryRunner) calls Fixtures.get(spec), and tests that build a container of their own use Fixtures.network(). ContainerSuite becomes the package-private SharedContainers, used only by specs and ContainerExtension: the same images, environment, host names, network, one container per key per run, run-end close and timing lines. Its unused starts (the single Trino container, default Doris and PostgreSQL overloads) go away; disabled TrinoConnectorIT builds its Trino container itself. TestContainerSuite becomes TestSharedContainers.
…ork per-run names
…containers wait strategies Move the post-start poll loops of Hive, Doris, Ranger, StarRocks, OceanBase, RustFS, ClickHouse, Kafka and Trino into wait strategies: the port or log check composed with a ProbeWaitStrategy for checks that need a client. Startup timeouts are the old port budget plus the old poll budget. Readiness time now shows in containerMs of the [CONTAINER-TIMING] line.
…nd OceanBase on official Testcontainers modules
…tainers onto fixtures
…ontainer and durable data
… for it without commands in the container
…ners for readiness and logs
…container start instead of with kadmin in the container
nevzheng
force-pushed
the
claude/13900-tc-21-kerberos-principals
branch
from
October 22, 2026 04:46
882e073 to
6b16816
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
@HiveFixture(kerberosPrincipals = {...})declares the extra Kerberos principals a test needs;HiveSpeccarries them (sorted, de-duplicated), so different principals mean a different container. They need Kerberos auth and a private container (scope = CLASSorMETHOD); the fixture fails at class discovery otherwise.start.shright after the KDC and the image's own principals exist and before HDFS starts, so readiness implies the keytabs exist. A missing hook point or a missing keytab fails the start.HadoopUserAuthenticationITdeclarescli_schemaandcli_filesetand copies their keytabs out withcopyFileFromContainer; its twokadmin.localcalls throughexecuteInContainerare gone.Why are the changes needed?
They were the last commands a test ran inside a container. Until the image adds the principals itself, creating them at start keeps the test free of
docker exec.Fix: #14049
Does this PR introduce any user-facing change?
No. Test code only.
How was this patch tested?
TestHiveSpec(8) andTestHiveContainer(5, including a check that the hook line exists indev/docker/kerberos-hive/start.sh);:integration-test-common:test(176);HadoopUserAuthenticationIT(4),HDFSKerberosIT,HiveUserAuthenticationITandMultipleHMSUserAuthenticationITon Docker.Stack: #14103 → #14083 → #14084 → #14085 → #14005 → #14006 → #14052 → #14076 → #14007 → #14008 → #14009 → #14010 → #14011 → #14012 → #14013 → #14079 → #14077 → #14078 → #14086 → #14087 → #14088 → #14089 → #14090 → #14091 → #14094