feat: add MiniMax H3 t2va Flow-GRPO support

[niehen6174]

Summary

• Add MiniMax H3 t2va Flow-GRPO training (LoRA + LoRA-IPC) against sglang-diffusion rollout.
• Pin diffusers to the main commit that merged H3 (not in a release yet).
• Add 2-GPU recipe: scripts/run_diffusion_grpo_h3_t2va_2gpu.py.
• Small prerequisites H3 exposed: modular-pipeline component specs, lazy num_train_timesteps, LoRA test fixture sglang_tp_size.

H3-specific bits (sampling target, SDE scorer, LoRA name/layout map) go through TrainPipelineConfig hooks instead of if family == "h3" in generic code.

Dependencies

1. diffusers — from this PR’s requirements.txt (git pin f53d5520). Force-reinstall if pip keeps an older wheel:

   pip install --force-reinstall -r requirements.txt

2. sglang-diffusion H3 RL — sgl-project/sglang#34365. Install that branch (or wait until it merges):

   pip install -e "python[all]"   # from the PR checkout
   export PYTHONPATH=/path/to/sglang/python:$PYTHONPATH

3. GPUs — at least 2 for the smoke/alignment path; more for the recommended batch=8 / group=16 scale. HF access to MiniMaxAI/MiniMax-H3, dataset rockdu/miles-diffusion-datasets (flowgrpo_pickscore; auto-downloaded by the recipe).

Reproduce

We only had 2 GPUs, so the long run used rollout_batch_size=4, n_samples_per_prompt=16. That is enough to train, but the GRPO group is smaller than we would like. rollout_batch_size=8 / group 16 is the better recipe when more GPUs are available (e.g. 8).

# alignment check on 2 GPUs (optimizer frozen; small batch)
python3 scripts/run_diffusion_grpo_h3_t2va_2gpu.py \
  --cuda-visible-devices 0,1 \
  --num-rollout 2 --eval-interval 0 --debug-alignment

# short train on 2 GPUs
python3 scripts/run_diffusion_grpo_h3_t2va_2gpu.py \
  --cuda-visible-devices 0,1 --num-rollout 5 --eval-interval 0

# this PR's 2-GPU long-run scale (resource-limited)
python3 scripts/run_diffusion_grpo_h3_t2va_2gpu.py \
  --rollout-batch-size 4 --n-samples-per-prompt 16 \
  --cuda-visible-devices 0,1

# recommended scale when more GPUs are available
python3 scripts/run_diffusion_grpo_h3_t2va_2gpu.py \
  --rollout-batch-size 8 --n-samples-per-prompt 16 \
  --cuda-visible-devices 0,1,2,3,4,5,6,7

On 2 GPUs, expect log_prob_mean_abs_diff around 1e-5 .. 8e-5, and LoRA IPC with skipped 0.

Notes:

• SDE window is 1,4 (skip step 0 from the recipe: σ=1 is singular for H3 flow-SDE). The train-side scorer is the generic flow-SDE backend.
• H3 requires --use-lora --lora-ipc-weight-sync.
• Geometry is aspect_ratio + duration_seconds (not width/height); sglang fixes short_edge=768.

Results

log_prob_diff stays in 1.4e-5 .. 7.1e-5 (well under 1e-4), so the train-side scorer matches the rollout log-probs at this scale:

Eval PickScore (every 10 rollouts) moves 0.806 → 0.814, peak 0.816 at step 59. The lift is small; this is the batch-4 curve, not the recommended batch-8 recipe:

Train rollout reward is noisier (64 samples / step) and sits around ~0.79 raw PickScore:

Test plan

• Install this branch + sglang#34365
• Alignment run: log_prob_mean_abs_diff < 1e-4
• Short train for a few rollouts
• Existing non-H3 CI still green