Skip to content

[Subtask] Run test SQL and readiness checks without CLIs in containers #14049

Open

Description

@nevzheng

Describe the subtask

Tests run Hive SQL, HDFS commands and readiness checks through docker exec CLIs. Each hive -e call starts a JVM with its own embedded metastore, which costs seconds per statement and per readiness poll. Replace them with in-process clients:

  • Hive SQL over JDBC to the HiveServer2 that already runs in the container.
  • Hive readiness through the NameNode's JMX and WebHDFS plus the metastore and HiveServer2 ports.
  • Kerberos HDFS commands through Hadoop FileSystem, database grants over JDBC, LocalStack setup over its HTTP API.

The Hive image runs everything as HADOOP_USER_NAME (anonymous in tests), which may not impersonate users, so HiveServer2 sessions fail. Until the image allows it, the test container adds the proxy-user rule before Hadoop starts.

Locally (3 runs, medians): catalog-hive 380 s → 235 s, CatalogHiveViewIT 122 s → 37 s, Hive readiness 8 s → 0.5 s per start.

Parent issue

#13900

Order and PRs

Blocked by #14003. PRs: two PRs stacked on #14013, linked here when opened. The image changes (proxy-user rule and warehouse directory in start.sh) follow in a separate issue.

Activity

  1. self-assigned this
    on Oct 21, 2026
  2. nevzheng commented on Oct 21, 2026

    @nevzheng
    CollaboratorAuthor

    PRs: #14052 (Hive SQL over JDBC and CLI-free Hive readiness). The Kerberos HDFS, database CLI and LocalStack execs follow in a second PR.

  3. added 20 commits that reference this issue on Oct 21, 2026
  4. added a commit that references this issue on Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

subtaskSubtasks of umbrella issue

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions