Skip to content

Add GLM 5.3 to live bench - #7

Open
anthonypizzolato-abacus wants to merge 1 commit into
mainfrom
add-glm-5-3
Open

Add GLM 5.3 to live bench#7
anthonypizzolato-abacus wants to merge 1 commit into
mainfrom
add-glm-5-3

Conversation

@anthonypizzolato-abacus

Copy link
Copy Markdown

Add GLM 5.3 to live bench

  • Model: GLM-5.3 (Z.ai / Zhipu AI), released 2026-08-14 — flagship coding/agentic model, open-weights SOTA on Terminal-Bench 3.0 and Agents' Last Exam (CLI). Docs: https://docs.z.ai/guides/llm/glm-5.3
  • Provider API ID: glm-5.3 on the first-party Z.ai API (https://api.z.ai/api/paas/v4, OpenAI-compatible)
  • Context window: 1M tokens (1,048,576); max output 128K
  • Note: the standalone GLM-5.3 API is rolling out ("coming soon" per the docs page; currently available via the GLM Coding Plan) — runs will work as soon as the endpoint opens.

Changes

  • lcb_runner/lm_styles.py — new LMStyle.ZAIAPI member; LanguageModel("glm-5.3", "GLM-5.3", LMStyle.ZAIAPI, datetime(2026, 8, 14), link=...) added after the DeepSeek first-party API entries
  • lcb_runner/runner/zai_runner.py — new ZAIRunner (env key ZAI_API_KEY), mirroring DeepSeekRunner exactly
  • lcb_runner/runner/runner_utils.pybuild_runner branch for LMStyle.ZAIAPI
  • lcb_runner/prompts/code_generation.pyZAIAPI added to the generic-chat-prompt style list

Copy link
Copy Markdown
Author

🔄 Re-verified 2026-08-18: this PR is still open and merges cleanly into current main (no conflicts).

Availability update since the PR was opened: the GLM-5.3 API is now live — z.ai serves it to GLM Coding Plan users via https://api.z.ai/api/paas/v4 (the endpoint this PR's ZAIRunner targets), and OpenRouter also lists z-ai/glm-5.3 at the same official rates ($1.40/$4.40 per 1M, cached $0.26 — cross-checked against docs.z.ai/guides/overview/pricing). Bench runs should now work with a ZAI_API_KEY from a Coding-Plan account.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant