Skip to content

predict: move bench service to deepbook-services#1051

Merged
pei-mysten merged 1 commit into
mainfrom
pei/move-predict-bench
May 29, 2026
Merged

predict: move bench service to deepbook-services#1051
pei-mysten merged 1 commit into
mainfrom
pei/move-predict-bench

Conversation

@pei-mysten

@pei-mysten pei-mysten commented May 29, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Removes crates/bench/ and docker/predict-bench/ — the bench service moves to deepbook-services alongside the other deepbook-* services
  • Removes crates/bench from the workspace Cargo.toml; Cargo.lock regenerated without bench's transitive deps
  • Keeps docker/predict-sim/ and packages/predict/simulations/ here — the sim container's entrypoint clones packages/predict/simulations/ at the requested SHA per benchmark run, so they are tightly coupled to the protocol code by design

Key decisions

  • Why split bench from sim: bench is pure ops infra (Axum service + k8s Job orchestration), sim is protocol-coupled (clones the source tree). Different release cadences, different reviewers, different concerns
  • predict-bench.yml workflow stays here — it only references the deployed bench service by URL, no local paths
  • Cargo.lock changes are mechanical (removing bench's transitive deps from the lock)
  • sim.ts is unchanged in this PR. The loadScenario(process.env.SCENARIO_PATH) change needed to let the sim pick up the GCS-downloaded scenario CSV is being handled in a separate PR by another teammate who owns that area. Until that lands, benchmarks against this PR's commits will silently ignore the bucket CSV and use the in-repo scenario_mar6_1000mints.csv, which matches the pre-bench-move behavior — so this PR is fully backwards-compatible and safe to land independently

Coordination

Part of a coordinated move across three repos:

  • deepbook-services#7: adds the bench code
  • sui-operations#8080: switches Pulumi to build the bench image from deepbook-services (sim image still builds from here) + provisions GCS scenario bucket
  • this PR: removes the bench code here

Recommended landing order: deepbook-services first → sui-operations second → this PR last (avoids any broken-deploy window — the bench code exists in both repos briefly, sui-operations switches sources, then this PR cleans up). The companion sim.ts change is decoupled and will land on its own schedule.

Test plan

  • cargo check clean on the workspace after removal
  • No behavior change for existing benchmarks (sim.ts unchanged → falls through to the in-repo scenario CSV as before)
  • After all three PRs land, manual benchmark via the deployed service still succeeds end-to-end against the in-repo scenario

🤖 Generated with Claude Code

The bench API service is pure ops infrastructure with no dependency on the
Move code or simulation TypeScript. It now lives in deepbook-services
alongside the other deepbook-* services. See:
MystenLabs/deepbook-services#7

The companion predict-sim image stays here because its entrypoint clones
packages/predict/simulations/ at the requested SHA — they are tightly
coupled to the protocol code by design.

The sim.ts change to read SCENARIO_PATH from env (needed to point the
sim at the GCS-downloaded scenario CSV) is deferred to a separate PR
from another teammate who owns that area.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@pei-mysten pei-mysten force-pushed the pei/move-predict-bench branch from 5956715 to c49f7d2 Compare May 29, 2026 21:41
@pei-mysten pei-mysten marked this pull request as ready for review May 29, 2026 21:42
@pei-mysten pei-mysten merged commit b07a374 into main May 29, 2026
4 of 5 checks passed
@pei-mysten pei-mysten deleted the pei/move-predict-bench branch May 29, 2026 21:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants