Skip to content

Commit 4143d3b

Browse files
committed
fix(server): null content in tool_calls log; bump mlx-swift-lm for turbo-kv fixes
- Streaming path: log content as null (not raw fullText) when tool_calls are present - mlx-swift-lm: TurboKV now compresses from token 1 (not 512) - mlx-swift-lm: head_dim guard prevents fatal crash on Qwen 122B (dim=256 != 128)
1 parent c323412 commit 4143d3b

2 files changed

Lines changed: 3 additions & 2 deletions

File tree

‎Sources/SwiftLM/Server.swift‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1194,10 +1194,11 @@ func handleChatStreaming(
11941194
print("") // end the real-time token stream line
11951195
let dur = Date().timeIntervalSince(genStart)
11961196
let tokPerSec = dur > 0 ? Double(completionTokenCount) / dur : 0
1197+
let logContent: Any = hasToolCalls ? NSNull() : fullText
11971198
let logResp: [String: Any] = [
11981199
"choices": [[
11991200
"index": 0,
1200-
"message": ["role": "assistant", "content": fullText],
1201+
"message": ["role": "assistant", "content": logContent],
12011202
"finish_reason": reason
12021203
]],
12031204
"usage": [

‎mlx-swift-lm‎

0 commit comments

Comments
 (0)