feat(http_provider): enhance K2-Think model support #334
feat(http_provider): enhance K2-Think model support #334shauryr wants to merge 2 commits intosipeed:mainfrom
Conversation
Add support for reasoning models that return responses in the reasoning_content field instead of the standard content field. Changes: - Add reasoning_content field to response parsing - Fallback to reasoning_content if content is empty - Add reasoning_effort parameter for K2-Think models - Strip special tokens (<|im_end|>, <|endoftext|>, etc.) from output This enables compatibility with: - LLM360/K2-Think-V2 (with vLLM) - DeepSeek R1 - QwQ and other reasoning models Tested with K2-Think-V2 on vLLM with --tool-call-parser hermes. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
Tech document referencing the need for https://platform.moonshot.ai/docs/guide/use-kimi-k2-thinking-model#accessing-the-reasoning-content @shauryr have you test this with other models that are used by the HTTP Provider? |
|
Thanks for reviewing! Quick clarification; LLM360/K2-Think-V2 https://huggingface.co/LLM360/K2-Think-V2 what I tested and Moonshot Kimi K2 are different models. The changes are generic and support any reasoning model. Here are the specific change I made which should be generic (all models):
K2-Think specific:
Also the changes are backward compatible
I have done the testing for - K2-Think-V2 via vLLM (basic queries + tool calling) as well as multi-step reasoning with tools. Also this should Work With DeepSeek R1, QwQ, and other models using Happy to test with specific models if you have recommendations! |
Added reasoning_effort parameter and handling reasoning_content in responses