Skip to content

fix(e2e): enable user provisioning on the E2E test instances during deploy - #1380

Merged
dylan-openhands merged 1 commit into
mainfrom
fix/e2e-enable-user-provisioning-on-test-instances
Oct 10, 2026
Merged

dylan-openhands merged 1 commit into
mainfrom
fix/e2e-enable-user-provisioning-on-test-instances

Conversation

@openhands-agent

Copy link
Copy Markdown
Contributor

Opened by an automated OpenHands agent.

Risk: 3/10 — the change touches only the deploy script for the internal test instances, and it changes one config value that is off by default.

Problem

Two E2E tests fail on every run because the test instances keep the user-provisioning feature turned off, and both tests need that feature to add a user to a new organization.

Description

The deploy script reads the instance config and writes it back during each upgrade. This PR makes the script set one config item, oem_user_creation_flow_enabled, to 1 in that write-back. Every other value stays the same. A new unit test checks that the script turns on this item and keeps the other values.

Evidence

  1. The E2E suite records its results in ReportPortal. The last 25 launches each report the same 2 failures.
  2. The failing tests are 006-org-management › create an org and provision the new user into it and 011-automations › owner in a fresh org can create a prompt automation.
  3. Both tests create an organization first. That step succeeds.
  4. Both tests then call POST /api/organizations/provision-user. The server answers 405 Method Not Allowed. A 405 here means that the server does not have this route.
  5. The unstable instance publishes its API list. The list does not include /api/organizations/provision-user.
  6. The app server adds this route only when USER_PROVISIONING_ENABLED is true. The Helm value userProvisioning.enabled sets that variable.
  7. On these instances, the Replicated config item oem_user_creation_flow_enabled sets that Helm value (replicated/openhands.yaml). The item defaults to 0.
  8. scripts/replicated_deploy.sh deploys every release to the internal test instances. It writes the config back unchanged, so the item stays 0.
  9. The tests have failed this way on every run since they started to run on 2026-10-01 (launches 200 to 262). So this failure is not a flake. The real fix is to turn on the feature that the tests need.
  10. Three more tests (008-managed-key-ownership, 013-super-admin-dashboard, and the second 006 test) use the same route. They are skipped or blocked today, so this change also unblocks them.

Work items

  1. This PR: Turn on user provisioning on the E2E test instances, so the 2 org tests can pass.
  2. Find out why 24 of the 45 tests are skipped on every run, for example the 11 @budgets tests and the @billing tests. Re-enable each test whose skip reason can be fixed.
  3. Confirm that every release-please merge starts a beta E2E run. Only 1 of the last 25 ReportPortal launches ran against beta.

Verification

After the next release deploys to unstable, the next unstable E2E launch in ReportPortal should show both tests as passed.

Helm Chart Checklist

  • No Helm chart was changed (this PR changes only scripts/).

…eploy

The org E2E specs call POST /api/organizations/provision-user, which the
app registers only when oem_user_creation_flow_enabled is on. The deploy
script now sets that item when it writes the instance config back.

Co-authored-by: openhands <openhands@all-hands.dev>
@openhands-agent openhands-agent added platform-team Issue falls under the platform team's responsibilities automated labels Oct 9, 2026
@openhands-agent openhands-agent added platform-team Issue falls under the platform team's responsibilities automated labels Oct 9, 2026
@github-actions github-actions Bot added the type: fix A bug fix label Oct 9, 2026
@dylan-openhands
dylan-openhands merged commit 0c928f9 into main Oct 10, 2026
28 checks passed
@dylan-openhands
dylan-openhands deleted the fix/e2e-enable-user-provisioning-on-test-instances branch October 10, 2026 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automated platform-team Issue falls under the platform team's responsibilities type: fix A bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants