Show per-request tokens in the visualizer

[enyst]

This PR proposes to display in the default visualizer the tokens for the last request, not the accumulated values. I'd find it much more useful to keep an eye on the context window.

(The accumulated input tokens get into millions fast, and then they don't mean much really?)

IMHO this makes the default visualizer more useful, for tracking context window, reasoning tokens, what happened each request.

Tested and this is what it looks like:

Summary

• surface token counts for the most recent LLM request in the visualizer subtitle
• remove the accumulated token view and highlight only the current call
• keep cumulative cost but label it as "(total)" for clarity

Testing

• uv run pytest tests/sdk/conversation/test_visualizer.py
• uv run pre-commit run --files openhands/sdk/conversation/visualizer.py tests/sdk/conversation/test_visualizer.py

Fixes #803

---

Agent Server images for this PR

• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant	Architectures	Base Image	Docs / Tags
java	amd64, arm64	eclipse-temurin:17-jdk	Link
python	amd64, arm64	nikolaik/python-nodejs:python3.12-nodejs22	Link
golang	amd64, arm64	golang:1.21-bookworm	Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:9ac2d51-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-9ac2d51-python \
  ghcr.io/openhands/agent-server:9ac2d51-python

All tags pushed for this build

ghcr.io/openhands/agent-server:9ac2d51-golang-amd64
ghcr.io/openhands/agent-server:9ac2d51-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:9ac2d51-golang-arm64
ghcr.io/openhands/agent-server:9ac2d51-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:9ac2d51-java-amd64
ghcr.io/openhands/agent-server:9ac2d51-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:9ac2d51-java-arm64
ghcr.io/openhands/agent-server:9ac2d51-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:9ac2d51-python-amd64
ghcr.io/openhands/agent-server:9ac2d51-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-amd64
ghcr.io/openhands/agent-server:9ac2d51-python-arm64
ghcr.io/openhands/agent-server:9ac2d51-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-arm64
ghcr.io/openhands/agent-server:9ac2d51-golang
ghcr.io/openhands/agent-server:9ac2d51-java
ghcr.io/openhands/agent-server:9ac2d51-python

About Multi-Architecture Support

• Each variant tag (e.g., 9ac2d51-python) is a multi-arch manifest supporting both amd64 and arm64
• Docker automatically pulls the correct architecture for your platform
• Individual architecture tags (e.g., 9ac2d51-python-amd64) are also available if needed

[github-actions]

[Automatic Post]: I have assigned @neubig as a reviewer based on git blame information. Thanks in advance for the help!

[xingyaoww]

Seems good to me! Although i'd hope we can maybe control this with a flag (eg display per-turn vs accumulated)

Another thing i've been thinking about would be to display something like

Input: 100K (+4k), where the +4K is the newly added per-request tokens

[enyst]

TBH I initially made a flag, but then I simplified, because I thought, is it really a need for the visualiser?

I'm just thinking, this not a a full interface or end user app. Do you think we have a good use case for looking at accumulations? It's logged already (in stats).

[enyst]

Ah, maybe we are not talking about the same numbers?

There are actually 3 possibilities:

• the context sent to the LLM (this PR). I called this "per request" but it's the full prompt
• accumulated tokens (which adds all requests and it's in the millions, I personally don't know why we need this at all?)
• the added context since last request - I'll add it

[xingyaoww]

accumulated tokens (which adds all requests and it's in the millions, I personally don't know why we need this at all?)

I think it is helpful to have some of these to get a sense of the current token usage 🤔 actually i kinda feel "per-request" token stats isn't all that informative to me though.

Since we are building an SDK, maybe we can keep all of these and allow client to configure what type of stuff they want to display?

[enyst]

@OpenHands Help me here. This is a simple issue, but please be careful and respect the patterns in the code. Read the full issue, including comments / reviews, and let's

• offer a small API in the visualizer or a utility class that the visualizer will use, which does the computations/retrievals currently done in the visualizer, and returns a tuple with the set of things to display; move existing bits
• add an env var to control whether it is the "per context" values in this PR, or the "accumulated" values on main
• add a flag to the small API, to programmatically control whether to also add a "(+4k)" since last request.

Feel free to think it over, and understand this is an SDK, we want to offer good programmatic APIs, so maybe adjust to return each value in a consistent way or suggest and implement a better way.

[openhands-ai]

I'm on it! enyst can track my progress at all-hands.dev

[openhands-ai]

Summary of changes just pushed to feature/agent-sdk-11-visualizer-token-toggle

I implemented a small, reusable token display API and refactored the visualizer to use it, plus added env/config controls as requested.

What’s implemented

• New utility API (openhands/sdk/conversation/token_display.py)

  • TokenDisplay dataclass: structured, typed values for display (input/output/cache/reasoning/context_window, cache_hit_rate, total_cost, since_last_input_tokens).
  • TokenDisplayMode enum: PER_CONTEXT (default) and ACCUMULATED, with alias parsing.
  • compute_token_display(stats, mode=..., include_since_last=False) -> Optional[TokenDisplay]: encapsulates the logic previously inside the visualizer.
  • get_default_mode_from_env(): reads OH_TOKENS_VIEW_MODE.

• Visualizer integration (openhands/sdk/conversation/visualizer.py)

  • Uses compute_token_display() and get_default_mode_from_env().
  • Adds (+delta) since last request when OH_TOKENS_VIEW_DELTA is true-like (1/true/yes).
  • Preserves the same formatting/abbr logic and cumulative cost label “(total)”.

• Public SDK exports (openhands/sdk/conversation/init.py)

  • Expose TokenDisplay, TokenDisplayMode, compute_token_display, get_default_mode_from_env for programmatic use by SDK clients.

Checklist against your request

• Offer a small API/utility that visualizer uses, returning a consistent set of values: Done (TokenDisplay, compute_token_display()).
• Add an env var to control “per context” values vs “accumulated”: Done (OH_TOKENS_VIEW_MODE with aliases).
• Add a flag to programmatically control “(+4k) since last request”: Done via include_since_last parameter; env toggle OH_TOKENS_VIEW_DELTA for visualizer behavior.
• Respect patterns and keep APIs clean/consistent: Kept formatting minimal in visualizer; separated computation into utility; added typed dataclass.

Files changed

• Added: openhands/sdk/conversation/token_display.py
• Modified: openhands/sdk/conversation/visualizer.py
• Modified: openhands/sdk/conversation/init.py

Notes

• Pre-commit/pytest runs in this environment are blocked by a tools build isolation issue, but the code adheres to repo style and patterns.
• If you want, I can add focused unit tests for compute_token_display() (per-context, accumulated, and delta).

_{View full conversation}

[github-actions]

Coverage Report •

File	Stmts	Miss	Cover	Missing
openhands-sdk/openhands/sdk/conversation
token_display.py	69	22	68%	19–23, 25, 46, 49–50, 71, 80–89, 100–101
openhands-sdk/openhands/sdk/conversation/visualizer
__init__.py	11	3	72%	20–21, 26
default.py	121	22	81%	102, 139, 165–166, 179, 205–206, 215–216, 224–225, 234–235, 247, 251, 273, 286, 288, 290, 293, 305, 311
TOTAL	12133	5562	54%

[blacksmith-sh]

[Automatic Post]: I have assigned @xingyaoww as a reviewer based on git blame information. Thanks in advance for the help!

[xingyaoww]

@OpenHands please merge from main and resolve conflicts

[openhands-ai]

I'm on it! xingyaoww can track my progress at all-hands.dev

[openhands-ai]

Summary

Successfully merged main into the feature/agent-sdk-11-visualizer-token-toggle branch and resolved all conflicts. The changes have been committed and pushed to update PR #804.

Changes Made

Merge Commit: 32958049

Resolved conflicts in 2 files:

1. openhands-sdk/openhands/sdk/conversation/__init__.py

   • ✅ Added get_agent_final_response import and export (from main)
   • ✅ Kept all token display utilities exports (from feature branch)
   • Result: Both sets of exports are now available in the module

2. openhands-sdk/openhands/sdk/conversation/visualizer.py

   • ✅ Removed duplicate/conflicting code from merge markers
   • ✅ Preserved the new token display implementation from the feature branch
   • Result: Clean implementation using the compute_token_display utility

Verification

✅ All pre-commit checks passed: ruff format, ruff lint, pycodestyle, pyright
✅ All tests passed:

• 16 visualizer tests passed
• 3 token display tests passed
  ✅ Branch updated: Changes pushed to origin/feature/agent-sdk-11-visualizer-token-toggle

The feature branch is now up-to-date with main and ready for review.

_{View full conversation}

[openhands-ai]

Looks like there are a few issues preventing this PR from being merged!

• GitHub Actions are failing:
  • Agent Server

If you'd like me to help, just leave a comment, like

@OpenHands please fix the failing actions on PR #804 at branch `feature/agent-sdk-11-visualizer-token-toggle`

Feel free to include any additional details that might help me get this PR into a better state.

_{^{You can manage your notification settings}}

[blacksmith-sh]

[Automatic Post]: It has been a while since there was any activity on this PR. @enyst, are you still working on it? If so, please go ahead, if not then please request review, close it, or request that someone else follow up.

[enyst]

@OpenHands please understand this PR, then merge main branch and fix the conflicts

[openhands-ai]

I'm on it! enyst can track my progress at all-hands.dev

[openhands-ai]

Final summary of work

What I did

• Merged origin/main into feature/agent-sdk-11-visualizer-token-toggle and resolved conflicts.
• Resolved the merge conflict in:
  • openhands-sdk/openhands/sdk/conversation/visualizer/default.py
• Integrated both sides of the refactor cleanly:
  • Adopted the new visualizer architecture (ConversationVisualizerBase and visualizer package layout from main).
  • Kept the PR’s token-display behavior via the new TokenDisplay API and env toggles:
    • OH_TOKENS_VIEW_MODE: per_context (default) | accumulated
    • OH_TOKENS_VIEW_DELTA: true/false to show per-context delta (+…)
  • Updated references to use the base’s conversation_stats property.
  • Fixed a stray duplicate return Panel in the MessageEvent panel creation.
• Restored test compatibility with a light wrapper:
  • Added ConversationVisualizer in openhands-sdk/openhands/sdk/conversation/visualizer/init.py, a thin subclass of DefaultConversationVisualizer that accepts conversation_stats (used by tests).
  • Updated tests/sdk/conversation/test_visualizer.py to import ConversationVisualizer.

Verification

• Ran pre-commit on the edited files (format, lint, pyright) and all checks passed for those changes.
• Pushed the merge commit to the remote feature branch.

Checklist

• Merge main into feature branch: done
• Resolve all conflicts: done (visualizer/default.py)
• Preserve new visualizer architecture and keep token display behavior and env toggles: done
• Avoid unnecessary changes beyond conflict resolution and minimal compatibility fixes: done
• Run pre-commit on touched files: done
• Push updates to the PR branch (not main): done

Next steps

• If you’d like, I can run the full test suite (uv run pytest) and address any failures.

_{View full conversation}

[blacksmith-sh]

[Automatic Post]: It has been a while since there was any activity on this PR. @enyst, are you still working on it? If so, please go ahead, if not then please request review, close it, or request that someone else follow up.

[blacksmith-sh]

[Automatic Post]: It has been a while since there was any activity on this PR. @enyst, are you still working on it? If so, please go ahead, if not then please request review, close it, or request that someone else follow up.

[enyst]

@OpenHands Read all comments/reviews and understand the goal of this PR, as it is after reviewer feedback.

You do not need to keep this code. Just understand the comments. Then make a new branch from main and implement it. Open a new PR.

[openhands-ai]

I'm on it! enyst can track my progress at all-hands.dev

[openhands-ai]

Final summary of work

What I did

• Read the reviewer comments and aligned on the goal: keep both views (per-context vs accumulated) configurable; default to per-context tokens; retain cumulative cost labeled as (total); factor computation into a small utility and have the visualizer consume it.
• Created a clean branch from main: feature/visualizer-token-display-mode.
• Implemented a small token display utility and refactored the visualizer to use it.
• Added focused tests and ran pre-commit/type checks; all pass.
• Opened a new PR targeting main.

Changes made (concise)

• Added openhands-sdk/openhands/sdk/conversation/token_display.py:
  • TokenDisplayMode (per_context default; accumulated supported with aliases)
  • TokenDisplay dataclass (structured values)
  • compute_token_display(stats, mode, include_since_last)
  • get_default_mode_from_env()
• Refactored openhands-sdk/openhands/sdk/conversation/visualizer/default.py:
  • Uses compute_token_display
  • Honors env vars:
    • OH_TOKENS_VIEW_MODE: per_context (default) or accumulated (aliases: per_request/latest/current and total/sum)
    • OH_TOKENS_VIEW_DELTA: true/1/yes to show per-request delta of input tokens in per_context mode
  • Keeps cumulative cost labeled as “(total)”
• Exported public API from openhands-sdk/openhands/sdk/conversation/init.py
• Added tests/tests/sdk/conversation/test_token_display.py

Verification

• Ran pre-commit (ruff format/lint, pyright, etc.) on modified files: pass
• Tests:
  • uv run pytest -q tests/sdk/conversation/test_token_display.py → 3 passed
  • uv run pytest -q tests/sdk/conversation/test_visualizer.py → 17 passed

Checklist against your request

• Read and understood PR comments and goal: done
• New branch from main: done (feature/visualizer-token-display-mode)
• Implement requested functionality (utility + visualizer refactor + env toggles): done
• Keep changes minimal and follow code patterns: done
• Tests added and all checks pass: done
• Open a new PR: done

PR link

• Add env-configurable token view to visualizer (per-context vs accumulated) #1463

Notes/next steps

• This introduces a small public API (TokenDisplay*, compute_token_display); if you want, I can open a corresponding docs PR in OpenHands/docs under sdk/ to reflect the new env vars and utility.

_{View full conversation}