Skip to content

feat: token tracking & cost estimation for non-Anthropic providers #14

@Dhwanil25

Description

@Dhwanil25

Current Limitation

Token tracking (tokensIn, tokensOut, costUsd) only works for Anthropic providers. For all other providers — OpenAI, Google, Groq, Mistral, DeepSeek, Cohere, xAI, Together, OpenRouter, Ollama, LM Studio — tokens and cost show as 0 because their SSE usage events are not yet parsed.

Root Cause

In multiAgentEngine.ts, the onUsage? callback is only wired in streamAnthropic. The other streaming functions (streamGoogle, streamCohere, streamOllama, streamOpenAICompat) don't parse usage events and don't call onUsage.

What Needs to Be Done

Each provider returns usage data differently at the end of their stream:

  • OpenAI-compatible (openai, groq, mistral, deepseek, xai, together, openrouter, lmstudio) — SSE sends a final chunk with:
    { "usage": { "prompt_tokens": 120, "completion_tokens": 340 } }
  • Google Gemini — final SSE chunk contains:
    { "usageMetadata": { "promptTokenCount": 120, "candidatesTokenCount": 340 } }
  • Cohere — final SSE event "message-end" contains:
    { "delta": { "usage": { "billed_units": { "input_tokens": 120, "output_tokens": 340 } } } }
  • Ollama / LM Studio — non-streaming final JSON body contains prompt_eval_count and eval_count

Acceptance Criteria

  • streamOpenAICompat parses usage.prompt_tokens / usage.completion_tokens from the final SSE chunk and calls onUsage
  • streamGoogle parses usageMetadata and calls onUsage
  • streamCohere parses message-end delta usage and calls onUsage
  • streamOllama parses prompt_eval_count / eval_count and calls onUsage
  • TOKEN_PRICES table extended with pricing for OpenAI, Google, Groq, Mistral, DeepSeek, Cohere, xAI, Together models
  • Header metrics bar shows correct token count and cost for all providers

Files to Modify

  • src/lib/multiAgentEngine.tsstreamOpenAICompat, streamGoogle, streamCohere, streamOllama, streamLLM, TOKEN_PRICES

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions