Commit c5f1568
fix: Resolve inferenceInProgress error in agentic tool calling loop
Root cause: chatCompletion() kept consuming the AsyncStream until
maxTokens even after a tool call was detected, because PythonService
.terminate() was a no-op with the native MLX backend. The follow-up
generateResponse() after tool execution hit isInferenceRunning = true.
Fix:
- MLXService.chatCompletion() now breaks out of the generation stream
loop as soon as </tool> or </tool_call> appears in the response,
returning immediately instead of running to maxTokens
- Removed all PythonService.shared.terminate() calls from ChatViewModel
(no-ops since Python was eliminated in v6.2.0)
- Simplified stopGeneration() to directly set flags without a Task
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>1 parent 3994bc8 commit c5f1568
2 files changed
Lines changed: 15 additions & 19 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
233 | 233 | | |
234 | 234 | | |
235 | 235 | | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
236 | 243 | | |
237 | 244 | | |
238 | 245 | | |
239 | 246 | | |
240 | 247 | | |
241 | 248 | | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
242 | 252 | | |
243 | 253 | | |
244 | 254 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
246 | 246 | | |
247 | 247 | | |
248 | 248 | | |
249 | | - | |
250 | | - | |
251 | | - | |
252 | | - | |
253 | | - | |
254 | | - | |
255 | | - | |
256 | | - | |
257 | | - | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
258 | 252 | | |
259 | 253 | | |
260 | 254 | | |
| |||
389 | 383 | | |
390 | 384 | | |
391 | 385 | | |
392 | | - | |
393 | | - | |
394 | | - | |
395 | | - | |
396 | | - | |
397 | | - | |
| 386 | + | |
| 387 | + | |
398 | 388 | | |
399 | 389 | | |
400 | 390 | | |
| |||
410 | 400 | | |
411 | 401 | | |
412 | 402 | | |
413 | | - | |
414 | | - | |
415 | 403 | | |
416 | 404 | | |
417 | 405 | | |
418 | 406 | | |
419 | 407 | | |
420 | 408 | | |
421 | 409 | | |
422 | | - | |
423 | 410 | | |
424 | 411 | | |
425 | 412 | | |
426 | 413 | | |
427 | 414 | | |
428 | 415 | | |
429 | | - | |
430 | 416 | | |
431 | 417 | | |
432 | 418 | | |
| |||
0 commit comments