-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Labels
area/benchmarkBenchmark methodology and automationBenchmark methodology and automationarea/llm-usabilityLLM generation reliability and token efficiencyLLM generation reliability and token efficiencytype/taskImplementation taskImplementation task
Description
Goal
Strengthen token-efficiency measurement so it predicts real LLM cost/latency decisions.
Scope
- Add provider-tokenizer-based counts (where available) alongside whitespace proxy.
- Report prompt, completion, and total tokens per task and per pass.
- Add cost projection mode (optional) using configurable token price table.
Acceptance
- Reports include both proxy and tokenizer-native counts.
- Token efficiency trends are charted in wiki.
- Methodology doc updated with caveats.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
area/benchmarkBenchmark methodology and automationBenchmark methodology and automationarea/llm-usabilityLLM generation reliability and token efficiencyLLM generation reliability and token efficiencytype/taskImplementation taskImplementation task