Add the GLM-5.2 744B x terminal-bench-2 Daytona example

[yueming-yuan]

Author: Miles team

Fully-async agentic RL on 16 GB300 nodes (4 GPUs each): 8 training nodes (TP2/CP4/PP4/EP8, optimizer state streamed to node-local disk) and 8 inference nodes (one 4-GPU dp-attention fp8 sglang engine per node). Every episode is a multi-turn terminal agent solving one terminal-bench-2 task inside its own Daytona sandbox built from that task's official image; scoring is the task's canonical tests/test.sh.

Three files under examples/experimental/openenv/glm52_tbench2/, next to the shared openenv agent/reward/data modules the recipe imports:

• run_glm5_2_744b_a40b_daytona.py — the recipe. Its defaults are the reference configuration: python3 run_glm5_2_744b_a40b_daytona.py train --num-nodes 16 reproduces the reference run. Deviations (smoke sizes, engine shape, checkpoint scoring) are CLI flags, not launcher edits.
• launch_16node_slurm.sh — a ~60-line site adapter: container + Ray bring-up only, forwards its own CLI args to the recipe. No experiment settings live here.
• README.md — environment contract, preparation steps, what the config does and why.

Performance Tuning

This PR includes several recent optimizations:

Memory

At the beginning, we was not able to train GLM 5.2 in 1 GB300 rack. After 3&4&5, we made it trainable with 16 GB300 nodes in colocate mode. Then, with 1&2, we make the async version trainable with 8 rollout nodes and 8 train nodes.

1. (save all the optimizer memory from GPU/CPU) Disk offload feat(offload): support disk target for training-actor offload #1575
2. (reduce the GPU peak memory from whole optimizer to bucket) NVME streaming optimizer feat(optimizer): NVMe optimizer-state streaming as a miles plugin #1793
3. (save 100+ GB CPU memory) For colocate mode, it use rematerialization to save CPU mem [optim]--rematerialize-param-from-master-weight: save the bf16 weight backup in colocate #1572
4. (save 20+ GB GPU memory, allow larger static-mem-fraction) Use gloo group in megatron disk ckpt [optim] run plan/metadata coordination over a gloo group Megatron-LM#62
5. (save 10+ GB GPU memory) fix DSA cuda graph page table in TMS Put the DSA cuda-graph page table in the pausable memory region sgl-project/sglang#33479

Speed

All the performance here is for 8 nodes rollout + 8 nodes train async mode. We use miles dashboard for all observation and perf tuning. At the beginning, we see slow rollout generation and large bubbles. After all the tuning, we make it almost fully overlapped and fully utilize the GPU resource.

1. Concurrency: Sample-level async submission [feat] support sample rollout submission granularity to keep fully async concurrency #1673 & Set to 2x batch size generation concurrency. Add --async-max-concurrent-samples to decouple fully-async generation concurrency from batch size #1677
   Before this, we see running_reqs dropping over time and the rollout concurrency is too low to be balanced with training, so large bubble exists. These PRs fixed the issue.
   Concurrency before:

2. Router: Default router config fixes [fix] fix session server hash_consistent mode #1657, Auto-enable router dp_aware routing when DP attention is on #1351, and set router mode to hash-consistent + min_load [router] set manual policy (sticky + min_load) as default agentic routing policy #1690
   Only with the above PRs, we still see large bubbles caused by an imbalance between engines. The hash-consistent + min_load fix it.
   Concurrency before and after:

3. Engine config: Tune SGLang config to be balanced Add the GLM-5.2 744B x terminal-bench-2 Daytona example #2220

and more...

Result

Routing
prefix-cache hit rate ≈0.96. Concurrency stable (as the right figure above)

Step-time
5-6min per step.

Schedule
Rollout and training are well-overlapped.

[Shi-Dong]

LGTM!

[Shi-Dong]

Both SGLang PRs are merged, so I think we can drop this now?

[yushengsu-thu]

awesome