Skip to content

Commit 2d81473

Browse files
committed
Keep AKS guide focused on customer setup
1 parent c30d7b8 commit 2d81473

1 file changed

Lines changed: 54 additions & 95 deletions

File tree

‎enterprise/k8s-install/aks.mdx‎

Lines changed: 54 additions & 95 deletions
Original file line numberDiff line numberDiff line change
@@ -13,14 +13,17 @@ Helm guide to deploy using the runtime values below. Skip
1313
or checks Sysbox, use the runc values and checks on this page instead.
1414

1515
<Warning>
16-
Sysbox is not yet supported for OpenHands Enterprise on AKS. This guide uses the
17-
node’s default runc runtime, evaluated with ordinary coding workflows (see Validation Scope). Docker builds
18-
and Docker Compose inside the sandbox are unavailable in this configuration.
19-
Other system-container workloads, such as systemd or nested containers, were not validated.
20-
Runc does not provide Sysbox’s additional system-container isolation; review the
21-
isolation requirements for your workloads before production adoption.
16+
Sysbox is not yet supported for OpenHands Enterprise on AKS. Use the runc
17+
configuration below. Docker builds and Docker Compose inside the sandbox are
18+
unavailable, and runc does not provide Sysbox's additional isolation. Confirm that
19+
these limits meet your workload requirements before choosing this configuration.
2220
</Warning>
2321

22+
<Note>
23+
The values below apply to Enterprise Helm chart `0.74.0` and Runtime API `0.10.0`.
24+
Check the values for your licensed release before installing or upgrading.
25+
</Note>
26+
2427
## Cluster Requirements
2528

2629
| Requirement | Recommendation |
@@ -33,8 +36,8 @@ isolation requirements for your workloads before production adoption.
3336

3437
Entra or Azure DevOps authentication alone does not establish subscription access.
3538
Check `az account list --all` and the selected subscription's permissions.
36-
Contributor was sufficient for the evaluated default managed-network deployment;
37-
Azure role assignments require additional permissions. Existing networks, private
39+
Grant the deployment identity access to create the cluster and its dependencies.
40+
Creating Azure role assignments requires additional permissions. Existing networks, private
3841
cluster access and workload identities need their own access checks.
3942

4043
Check VM SKU restrictions as well as quota before selecting a region or availability
@@ -50,16 +53,16 @@ Use separate pools for application services and sandbox workloads:
5053
`workload=openhands-sandbox` as a persistent node-pool label so new nodes match
5154
the Runtime API selector. Do not install the Sysbox DaemonSet or add its installer label.
5255

53-
The evaluation used Azure CNI overlay with Calico, two platform nodes and a sandbox
54-
pool with autoscaler bounds of one to two nodes. Size the pools using the
55-
[Sizing Guide](/enterprise/sizing-guide) and
56-
[Resource Limits](/enterprise/k8s-install/resource-limits), including node overhead,
57-
image storage and warm sandbox capacity. Validate scale-down behavior for active sessions.
56+
Size both pools using the [Sizing Guide](/enterprise/sizing-guide) and
57+
[Resource Limits](/enterprise/k8s-install/resource-limits). Leave capacity for
58+
Kubernetes services, existing workloads and concurrent sandboxes. Check
59+
PodDisruptionBudgets before draining nodes for maintenance or upgrades.
5860

59-
AKS rejected a manual node-pool scale-down with `UnsatisfiablePDB` because of a
60-
sandbox PodDisruptionBudget with `maxUnavailable: 0`. Node-image and Kubernetes
61-
upgrades, which drain nodes, were not tested. Validate maintenance before relying
62-
on it; do not delete or patch PodDisruptionBudgets to force it.
61+
<Warning>
62+
Keep sufficient sandbox capacity available before users start conversations.
63+
Provisioning a new AKS node can exceed the sandbox startup timeout. Do not rely
64+
on cold autoscaling, including scale-from-zero, for immediate conversation startup.
65+
</Warning>
6366

6467
### Configure runc Sandboxes
6568

@@ -83,57 +86,34 @@ install and upgrade, after your base values and feature overlays. For example:
8386
```bash
8487
helm upgrade --install openhands <licensed-chart-source> --version 0.74.0 \
8588
--namespace openhands --create-namespace \
86-
-f values.yaml -f values-automation.yaml -f values-aks-runc.yaml --timeout 10m
89+
-f values.yaml -f values-aks-runc.yaml --timeout 10m
8790
```
8891

89-
Include `values-automation.yaml` only when enabling automations. Follow the Helm
90-
guide for namespaces, Secrets and other prerequisites. Omitting these overrides
92+
Use the licensed chart source from the [Helm guide](/enterprise/k8s-install/installation),
93+
and complete its namespace and Secret setup first. When enabling optional features,
94+
place their values files before `values-aks-runc.yaml`. Omitting these overrides
9195
can restore the chart’s Sysbox defaults and prevent new sandboxes from starting.
9296
If you add a sandbox-pool taint, configure a matching toleration instead of the empty list.
9397

94-
After deployment, create a conversation and verify that its pod is in the
95-
sandbox pool with no `runtimeClassName` or `hostUsers` override:
98+
### Verify a Sandbox
9699

97-
```bash
98-
kubectl get pod <sandbox-pod> -n <runtime-namespace> \
99-
-o jsonpath='{.spec.nodeName}{"\t"}{.spec.runtimeClassName}{"\t"}{.spec.hostUsers}{"\n"}'
100-
```
100+
1. Sign in to OpenHands and start a conversation. Ask the agent to run `pwd`,
101+
write a file in its workspace and read it back.
102+
2. Confirm the sandbox pod runs on the sandbox pool using the default runtime:
101103

102-
Expect a sandbox-pool node name followed by two empty fields. Use the configured
103-
runtime namespace (`openhands-runtimes` in the evaluation). The evaluated
104-
sandboxes ran without privileged mode, a host Docker socket or added capabilities.
105-
Confirm the agent executes commands and writes and rereads a workspace marker.
106-
Python, Node/npm, Git (public clone; authenticated pushes not tested), browser
107-
navigation/screenshots, HTTP previews and workspace
108-
persistence passed the runc evaluation. Docker-in-sandbox failed; an installed
109-
Docker CLI or Compose binary does not establish daemon functionality.
110-
See [Docker in the agent sandbox](/enterprise/docker-in-sandbox) for that feature’s requirements.
111-
112-
### Keep Startup Capacity Ready
113-
114-
Keep enough Ready sandbox nodes with the agent image cached to accommodate the
115-
expected concurrent sessions. Pool minimum counts alone do not establish available
116-
capacity: account for existing pods, memory requests, disk limits and node overhead.
117-
The tested defaults requested 500m CPU and 3 GiB memory per sandbox; a tested
118-
4-vCPU/16-GiB node accommodated three such sandboxes after Kubernetes reservations.
119-
Size production capacity using your own resources and workload measurements.
120-
121-
In the October 6 evaluation, six starts on ready/cached capacity all succeeded in
122-
38–51 seconds. A separate three-start burst forcing a zero-node pool to scale up
123-
failed around 120 seconds: the new node became Ready at 143 seconds and sandbox
124-
pods at approximately 234–247 seconds. The failed startup tasks did not recover
125-
automatically after their pods became healthy. Three fresh starts on that same node
126-
with its image cached then all succeeded in 53–55 seconds.
127-
128-
This locates the observed failure before sandbox execution, in cold provisioning
129-
and startup orchestration. It is not a runc-versus-Sysbox performance benchmark.
130-
Nodes that were already Ready with the image cached avoided this failure in testing.
131-
Pre-pulling images onto newly added nodes was not tested and would not remove the
132-
node-readiness delay. No startup deadline or recovery fix was validated. Do not
133-
rely on the autoscaler adding nodes, including scale-from-zero, to absorb interactive
134-
startups in the tested release. Validate scale-out, node replacement and maintenance
135-
for your target release. Pause orphaned sandboxes from failed startups through
136-
supported OpenHands APIs or the UI’s Stop Runtime action before retrying.
104+
```bash
105+
kubectl get pod <sandbox-pod> -n <runtime-namespace> \
106+
-o jsonpath='{.spec.nodeName}{"\t"}{.spec.runtimeClassName}{"\t"}{.spec.hostUsers}{"\n"}'
107+
```
108+
109+
Expect a sandbox-pool node name followed by two empty fields. Use the runtime
110+
namespace configured in your Helm values, such as `openhands-runtimes`.
111+
3. Stop the conversation runtime, reopen the conversation and ask the agent to
112+
read the same file to confirm workspace persistence.
113+
114+
If a sandbox fails to start, follow the
115+
[Troubleshooting guide](/enterprise/troubleshooting) and collect the pod events
116+
before contacting OpenHands support.
137117

138118
## Persistent Storage
139119

@@ -143,12 +123,11 @@ Inspect the Azure Disk CSI StorageClass:
143123
kubectl get storageclass managed-csi -o yaml
144124
```
145125

146-
The evaluated class uses `disk.csi.azure.com`, `WaitForFirstConsumer`, reclaim
147-
policy `Delete` and volume expansion. The evaluated Azure Disk PVCs use
148-
`ReadWriteOnce`. Configure stateful components and workspace
149-
storage explicitly, including persistence for any in-cluster object store; the GKE
150-
`standard-rwo` and AWS gp3 examples do not apply. Add `STORAGE_CLASS` to the same
151-
`runtime-api.env` map as the runc values above; do not create duplicate YAML keys.
126+
Confirm that the class uses the Azure Disk CSI provisioner (`disk.csi.azure.com`)
127+
and supports the binding mode and expansion your workloads need. Azure Disk
128+
workspace PVCs use `ReadWriteOnce`. Configure storage explicitly for workspaces,
129+
PostgreSQL and any in-cluster object store. Merge the following settings into your
130+
base values file:
152131

153132
```yaml
154133
runtime-api:
@@ -162,9 +141,6 @@ postgresql:
162141

163142
Account for node disk-attachment limits and availability-zone topology. Validate
164143
mounting, persisted content after reattachment, and expansion on a disposable PVC.
165-
The runc evaluation passed workspace read/write, stop/reopen persistence and
166-
cross-node disk reattachment. Expansion was not validated on the runc configuration;
167-
test it on a disposable PVC.
168144
See [Azure Disk CSI provisioning](https://learn.microsoft.com/en-us/azure/aks/create-volume-azure-disk).
169145

170146
## Object Storage
@@ -173,9 +149,10 @@ Conversation/session storage is separate from workspace PVCs. Helm chart 0.74.0
173149
supports S3-compatible and GCS filestore configuration; it does not expose an Azure
174150
Blob backend. Do not substitute Azure Blob credentials into the S3 settings.
175151

176-
The tested configuration used the chart's optional RustFS store on Azure Disk CSI.
177-
Select an object store that meets your durability and availability requirements,
178-
and verify backup and restore for that configuration.
152+
For an in-cluster store, chart `0.74.0` includes optional RustFS. Configure its
153+
persistence with Azure Disk CSI, provide the object-store credential Secret, and
154+
create the conversation bucket before starting conversations. Select an object
155+
store that meets your durability requirements and configure backup and restore.
179156
When enabling automations, configure a separate automation package bucket. The
180157
automation service can create it at startup when bucket creation is enabled.
181158

@@ -184,12 +161,12 @@ automation service can create it at startup when bucket creation is enabled.
184161
Use the [External PostgreSQL](/enterprise/external-postgres) guide for managed
185162
database requirements and Helm values. If choosing Azure Database for PostgreSQL,
186163
verify compatibility, TLS and network access against those requirements before deployment.
187-
The tested configuration used bundled PostgreSQL.
164+
For bundled PostgreSQL, set its persistence storage class as shown above.
188165

189166
## Ingress
190167

191-
Run an ingress controller on the general pool. Traefik was validated with an Azure
192-
public LoadBalancer. In the Traefik Helm values (chart `41.6.0` was tested):
168+
Run an ingress controller on the general pool. For Traefik with an Azure public
169+
load balancer, use the following Traefik Helm values:
193170

194171
```yaml
195172
service:
@@ -204,24 +181,6 @@ Follow [DNS and TLS](/enterprise/k8s-install/dns-and-tls) for trusted certificat
204181
and hostname configuration. Use flat runtime hostnames with Traefik standard Ingress.
205182
If provisioning certificates manually, assign renewal and Secret-update ownership.
206183

207-
## Validation Scope
208-
209-
<Note>
210-
Validated October 6, 2026 with Helm chart `0.74.0`, OpenHands `1.67.0`, Runtime API
211-
`0.10.0` and agent-server `1.49.6-python` on AKS `1.35.8`, Ubuntu `24.04.5 LTS`
212-
and containerd `2.3.3-2`. The configuration used bundled PostgreSQL and RustFS.
213-
GitHub login, trusted HTTPS, ordinary coding/browser workflows, workspace persistence,
214-
cross-node disk movement and execution on an autoscaler-added node, after it
215-
became Ready, passed. Newly added nodes ran sandboxes without per-node runtime setup.
216-
217-
Cold autoscaling exceeded the startup deadline. An earlier resume-related HTTP 401
218-
and a PodDisruptionBudget issue blocking manual pool scale-down remain unresolved.
219-
Automations and PVC expansion were not validated on this runc configuration. Managed Azure PostgreSQL, Azure Blob, private
220-
endpoints, backup/restore, cross-zone recovery, tenant isolation and network
221-
isolation between sandboxes were not covered. No NetworkPolicy targeted the
222-
sandbox namespace in the evaluation. This is evaluation evidence, not production acceptance.
223-
</Note>
224-
225184
## Next Steps
226185

227186
<CardGroup cols={2}>
@@ -232,6 +191,6 @@ sandbox namespace in the evaluation. This is evaluation evidence, not production
232191
Deploy the application and validate a conversation.
233192
</Card>
234193
<Card title="Enable Automations" icon="clock" href="/enterprise/k8s-install/automations">
235-
Configure automation storage and verify a completed run.
194+
Configure optional automation services and storage.
236195
</Card>
237196
</CardGroup>

0 commit comments

Comments
 (0)