Skip to content

Expose messageId and usage endpoints #642

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
shaunak99 wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Expose messageId and usage endpoints #642

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions packages/core/src/agent/DextoAgent.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -844,6 +844,7 @@ export class DextoAgent {
usage: usage as import('./types.js').TokenUsage,
toolCalls,
sessionId,
messageId: responseEvent.messageId,
};
}

Expand Down
1 change: 1 addition & 0 deletions packages/core/src/agent/types.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ export interface GenerateResponse {
usage: LLMTokenUsage;
toolCalls: AgentToolCall[];
sessionId: string;
messageId?: string;
}

/**
Expand Down
2 changes: 2 additions & 0 deletions packages/core/src/events/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -370,6 +370,8 @@ export interface AgentEventMap {
cacheReadTokens?: number;
cacheWriteTokens?: number;
};
/** Stable assistant message id for idempotency/billing */
messageId?: string;
Comment on lines +373 to +374
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Mirror messageId into the session-scoped llm:response type.

StreamProcessor now emits messageId on SessionEventBus, but SessionEventMap['llm:response'] still omits it. That leaves the session contract out of sync with the agent contract and hides the new field from session-level consumers.

Suggested fix 
 // SessionEventMap
     'llm:response': {
         content: string;
         reasoning?: string;
         provider?: LLMProvider;
         model?: string;
         /** Reasoning tuning variant used for this call, when the provider exposes it. */
         reasoningVariant?: ReasoningVariant;
         /** Reasoning budget tokens used for this call, when the provider exposes it. */
         reasoningBudgetTokens?: number;
         tokenUsage?: {
             inputTokens?: number;
             outputTokens?: number;
             reasoningTokens?: number;
             totalTokens?: number;
             cacheReadTokens?: number;
             cacheWriteTokens?: number;
         };
+        /** Stable assistant message id for idempotency/billing */
+        messageId?: string;
         /** Estimated input tokens before LLM call (for analytics/calibration) */
         estimatedInputTokens?: number;
         /** Finish reason: 'tool-calls' means more steps coming, others indicate completion */
         finishReason?: LLMFinishReason;
     };
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
/** Stable assistant message id for idempotency/billing */
messageId?: string;
'llm:response': {
content: string;
reasoning?: string;
provider?: LLMProvider;
model?: string;
/** Reasoning tuning variant used for this call, when the provider exposes it. */
reasoningVariant?: ReasoningVariant;
/** Reasoning budget tokens used for this call, when the provider exposes it. */
reasoningBudgetTokens?: number;
tokenUsage?: {
inputTokens?: number;
outputTokens?: number;
reasoningTokens?: number;
totalTokens?: number;
cacheReadTokens?: number;
cacheWriteTokens?: number;
};
/** Stable assistant message id for idempotency/billing */
messageId?: string;
/** Estimated input tokens before LLM call (for analytics/calibration) */
estimatedInputTokens?: number;
/** Finish reason: 'tool-calls' means more steps coming, others indicate completion */
finishReason?: LLMFinishReason;
};
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@packages/core/src/events/index.ts` around lines 373 - 374, Session-level
event type for "llm:response" is missing the new stable assistant messageId
emitted by StreamProcessor via SessionEventBus; update the session event map to
mirror this by adding messageId?: string to the SessionEventMap['llm:response']
/ the session-scoped "llm:response" type definition so session consumers receive
the same field StreamProcessor emits and keep the session contract in sync with
the agent contract.

/** Estimated input tokens before LLM call (for analytics/calibration) */
estimatedInputTokens?: number;
/** Finish reason: 'tool-calls' means more steps coming, others indicate completion */
Expand Down
1 change: 1 addition & 0 deletions packages/core/src/llm/executor/stream-processor.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -699,6 +699,7 @@ export class StreamProcessor {
model: this.config.model,
...this.getReasoningResponseFields(),
tokenUsage: config.tokenUsage,
...(this.assistantMessageId && { messageId: this.assistantMessageId }),
...(this.config.estimatedInputTokens !== undefined && {
estimatedInputTokens: this.config.estimatedInputTokens,
}),
Expand Down
2 changes: 2 additions & 0 deletions packages/server/src/events/a2a-sse-subscriber.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,10 +139,12 @@ export class A2ASseEventSubscriber {
role: 'agent',
content: [{ type: 'text', text: payload.content }],
timestamp: new Date().toISOString(),
...(payload.messageId && { messageId: payload.messageId }),
},
tokenUsage: payload.tokenUsage,
provider: payload.provider,
model: payload.model,
...(payload.messageId && { messageId: payload.messageId }),
});
},
{ signal }
Expand Down
6 changes: 6 additions & 0 deletions packages/server/src/hono/routes/messages.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,11 @@ export function createMessagesRouter(
sessionId: z.string().describe('Session ID used for this message'),
tokenUsage:
TokenUsageSchema.optional().describe('Token usage statistics'),
messageId: z
.string()
.uuid()
.optional()
.describe('Assistant message ID for idempotency'),
reasoning: z
.string()
.optional()
Expand Down Expand Up @@ -268,6 +273,7 @@ export function createMessagesRouter(
response: result.content,
sessionId: result.sessionId,
tokenUsage: result.usage,
messageId: result.messageId,
reasoning: result.reasoning,
model: llmConfig.model,
provider: llmConfig.provider,
Expand Down
122 changes: 122 additions & 0 deletions packages/server/src/hono/routes/sessions.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import {
SessionMetadataSchema,
InternalMessageSchema,
StandardErrorEnvelopeSchema,
UsageSummarySchema,
} from '../schemas/responses.js';
import type { GetAgentFn } from '../index.js';

Expand Down Expand Up @@ -196,6 +197,76 @@ export function createSessionsRouter(getAgent: GetAgentFn) {
},
});

const usageRoute = createRoute({
method: 'get',
path: '/sessions/{sessionId}/usage',
summary: 'Get Session Usage',
description: 'Returns aggregated token usage for a session',
tags: ['sessions'],
request: { params: z.object({ sessionId: z.string().describe('Session identifier') }) },
responses: {
200: {
description: 'Aggregated usage for session',
content: {
'application/json': {
schema: z
.object({
sessionId: z.string().describe('Session identifier'),
usage: UsageSummarySchema,
})
.strict(),
},
},
},
404: {
description: 'Session not found',
content: {
'application/json': {
schema: z.object({ error: z.string().describe('Error message') }).strict(),
},
},
Comment on lines +221 to +227
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Use the shared error envelope for these 404s.

Lines 225/263 document { error: string }, and Lines 541/569 return the same ad-hoc body. That introduces a second error contract for the sessions API and forces clients to special-case these endpoints.

Based on learnings, error responses must follow the standard middleware format from packages/server/src/hono/middleware/error.ts, and 404 responses should document that structure in the route definition.

Also applies to: 259-264, 541-542, 569-572

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@packages/server/src/hono/routes/sessions.ts` around lines 221 - 227, Replace
the ad-hoc 404 response schema (z.object({ error: z.string() })) in the sessions
route definitions with the shared error envelope exported by the error
middleware (use the middleware's exported schema, e.g., errorEnvelopeSchema /
errorResponseSchema) and import that symbol from the middleware module; update
all occurrences referenced in this review (the two 404 response docs in
sessions.ts and the ad-hoc 404 bodies returned at the endpoints) so the
OpenAPI/route docs and runtime responses use the same shared error envelope
shape.

},
},
});

const messageUsageRoute = createRoute({
method: 'get',
path: '/sessions/{sessionId}/messages/{messageId}/usage',
summary: 'Get Message Usage',
description: 'Returns token usage for a specific assistant message',
tags: ['sessions'],
request: {
params: z.object({
sessionId: z.string().describe('Session identifier'),
messageId: z.string().describe('Assistant message identifier'),
}),
},
responses: {
200: {
description: 'Usage for assistant message',
content: {
'application/json': {
schema: z
.object({
sessionId: z.string().describe('Session identifier'),
messageId: z.string().describe('Assistant message identifier'),
usage: UsageSummarySchema,
})
.strict(),
},
},
},
404: {
description: 'Session or message not found',
content: {
'application/json': {
schema: z.object({ error: z.string().describe('Error message') }).strict(),
},
},
},
},
});

const deleteRoute = createRoute({
method: 'delete',
path: '/sessions/{sessionId}',
Expand Down Expand Up @@ -462,6 +533,57 @@ export function createSessionsRouter(getAgent: GetAgentFn) {
isBusy,
});
})
.openapi(usageRoute, async (ctx) => {
const agent = await getAgent(ctx);
const { sessionId } = ctx.req.param();
const metadata = await agent.getSessionMetadata(sessionId);
if (!metadata) {
return ctx.json({ error: `Session not found: ${sessionId}` }, 404);
}
Comment on lines +539 to +542
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Catch thrown metadata lookups before returning 404.

Line 539 assumes agent.getSessionMetadata(sessionId) returns undefined for a missing session, but this file already wraps the same lookup in try/catch when stale session IDs are possible. If the lookup throws here, /sessions/{sessionId}/usage returns 500 instead of the documented 404.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@packages/server/src/hono/routes/sessions.ts` around lines 539 - 542, The
session metadata lookup using agent.getSessionMetadata(sessionId) can throw for
stale/nonexistent IDs; wrap that call in a try/catch in the
/sessions/{sessionId}/usage handler (around the existing
agent.getSessionMetadata call) and on error return the same 404 JSON response
(ctx.json({ error: `Session not found: ${sessionId}` }, 404)); keep the current
undefined check but ensure thrown errors are caught and handled the same way so
a thrown lookup doesn't produce a 500.


const usage = metadata.tokenUsage ?? {
inputTokens: 0,
outputTokens: 0,
totalTokens: 0,
reasoningTokens: 0,
cacheReadTokens: 0,
cacheWriteTokens: 0,
};

return ctx.json({
sessionId,
usage: {
promptTokens: usage.inputTokens ?? 0,
completionTokens: usage.outputTokens ?? 0,
totalTokens: usage.totalTokens ??
(usage.inputTokens ?? 0) + (usage.outputTokens ?? 0),
},
});
})
.openapi(messageUsageRoute, async (ctx) => {
const agent = await getAgent(ctx);
const { sessionId, messageId } = ctx.req.param();
const history = await agent.getSessionHistory(sessionId);
const message = history.find((item) => item.id === messageId);
if (!message || message.role !== 'assistant' || !message.tokenUsage) {
return ctx.json(
{ error: `Message usage not found for ${messageId}` },
404
);
}

const usage = message.tokenUsage;
return ctx.json({
sessionId,
messageId,
usage: {
promptTokens: usage.inputTokens ?? 0,
completionTokens: usage.outputTokens ?? 0,
totalTokens: usage.totalTokens ??
(usage.inputTokens ?? 0) + (usage.outputTokens ?? 0),
},
});
})
.openapi(deleteRoute, async (ctx) => {
const agent = await getAgent(ctx);
const { sessionId } = ctx.req.param();
Expand Down
13 changes: 13 additions & 0 deletions packages/server/src/hono/schemas/responses.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,19 @@ export const SessionTokenUsageSchema = z
.strict()
.describe('Session-level token usage (all fields required for cumulative totals)');

export const UsageSummarySchema = z
.object({
promptTokens: z.number().int().nonnegative().describe('Aggregated prompt/input tokens'),
completionTokens: z
.number()
.int()
.nonnegative()
.describe('Aggregated completion/output tokens'),
totalTokens: z.number().int().nonnegative().describe('Aggregated total tokens'),
})
.strict()
.describe('Usage summary for external billing');

export const ModelStatisticsSchema = z
.object({
provider: z.string().describe('LLM provider identifier'),
Expand Down
Loading