Skip to content

issue #7582 : fix IT write permissions and ExecProcess single-command exec#7584

Merged
mattcasters merged 2 commits into
apache:mainfrom
mattcasters:issue-7582
Jul 20, 2026
Merged

issue #7582 : fix IT write permissions and ExecProcess single-command exec#7584
mattcasters merged 2 commits into
apache:mainfrom
mattcasters:issue-7582

Conversation

@mattcasters

Copy link
Copy Markdown
Contributor

What

Fixes Jenkins integration-test permission failures after the Native Spark / surefire work, and restores ExecProcess single-command-line execution broken by #7565.

Closes #7582

Why

Build Hop-integration-tests #2247 failed 14 tests with AccessDeniedException / Could not delete under bind-mounted project trees. Permission hardening only covered output/ and surefire reports, while many older ITs still write under files/ or the project root. Separately, #7565 changed Runtime.exec(String) to Runtime.exec(String[]) for single-command mode, so command lines like /bin/echo hop-single no longer tokenize.

How

  • IT host/container prep (run-tests-docker.sh / run-tests.sh): re-own integration-tests/ to the container UID; world-writable output/ and files/; writable project roots; residual cleanup that never deletes git-tracked fixtures.
  • GCP key validation: stricter check before running Google Sheets ITs; clear SKIP_GOOGLE_SHEETS logging.
  • Generated artifacts → output/ for spreadsheet Excel writers, MDI JSON/Excel outputs, HTTP download target, transforms parquet prepare/read; update golden multi-file Excel dataset paths.
  • Untrack residual seeds (http-action-output, MDI generated outputs) and extend .gitignore.
  • ExecProcess: restore Runtime.exec(String) for non-arguments-in-fields mode; minor test setup hardening.

Verification

  • Local full IT suite green after the final spreadsheet golden path fix.
  • ExecProcessTest (and related unit tests) pass after the Runtime.exec restore.

…ommand exec

Harden integration-test Docker prep so project output/, files/, and roots
are writable under the container UID, and move generated spreadsheet/MDI/
parquet/http artifacts into world-writable output/ paths (with golden data
and gitignore updates). Also restore Runtime.exec(String) for ExecProcess
single-command mode so command lines are tokenized again (apache#7565 regression).
…pache#7581

Use explicit whitespace tokenization before Runtime.exec(String[]) for
single-command mode (avoids the deprecated String overload), and add a
unit test for tokenizeCommandLine. Credit: @leehaut in apache#7581.
@mattcasters

Copy link
Copy Markdown
Contributor Author

Adopted the ExecProcess tokenization approach from #7581 (thanks @leehaut) so we keep the non-deprecated Runtime.exec(String[]) path with explicit StringTokenizer splitting. IT permission/path fixes remain the rest of this PR.

@mattcasters
mattcasters merged commit ef9d648 into apache:main Jul 20, 2026
3 checks passed
mattcasters added a commit to mattcasters/hop that referenced this pull request Jul 20, 2026
Bring in ExecProcess single-command tokenization fix (apache#7581/apache#7584) and
other main updates to unblock PR build.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Some Jenkins integration tests have permissions issues

1 participant