-
Notifications
You must be signed in to change notification settings - Fork 0
feat: token tracking & cost estimation for non-Anthropic providers #14
Copy link
Copy link
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Current Limitation
Token tracking (tokensIn, tokensOut, costUsd) only works for Anthropic providers. For all other providers — OpenAI, Google, Groq, Mistral, DeepSeek, Cohere, xAI, Together, OpenRouter, Ollama, LM Studio — tokens and cost show as 0 because their SSE usage events are not yet parsed.
Root Cause
In multiAgentEngine.ts, the onUsage? callback is only wired in streamAnthropic. The other streaming functions (streamGoogle, streamCohere, streamOllama, streamOpenAICompat) don't parse usage events and don't call onUsage.
What Needs to Be Done
Each provider returns usage data differently at the end of their stream:
- OpenAI-compatible (
openai,groq,mistral,deepseek,xai,together,openrouter,lmstudio) — SSE sends a final chunk with:{ "usage": { "prompt_tokens": 120, "completion_tokens": 340 } } - Google Gemini — final SSE chunk contains:
{ "usageMetadata": { "promptTokenCount": 120, "candidatesTokenCount": 340 } } - Cohere — final SSE event
"message-end"contains:{ "delta": { "usage": { "billed_units": { "input_tokens": 120, "output_tokens": 340 } } } } - Ollama / LM Studio — non-streaming final JSON body contains
prompt_eval_countandeval_count
Acceptance Criteria
-
streamOpenAICompatparsesusage.prompt_tokens/usage.completion_tokensfrom the final SSE chunk and callsonUsage -
streamGoogleparsesusageMetadataand callsonUsage -
streamCohereparsesmessage-enddelta usage and callsonUsage -
streamOllamaparsesprompt_eval_count/eval_countand callsonUsage -
TOKEN_PRICEStable extended with pricing for OpenAI, Google, Groq, Mistral, DeepSeek, Cohere, xAI, Together models - Header metrics bar shows correct token count and cost for all providers
Files to Modify
src/lib/multiAgentEngine.ts—streamOpenAICompat,streamGoogle,streamCohere,streamOllama,streamLLM,TOKEN_PRICES
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request