Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **`BaggageBuilder.sourceMetadataName()` renamed to `BaggageBuilder.channelName()`** — Fluent setter for the channel name baggage value.
- **`BaggageBuilder.sourceMetadataDescription()` renamed to `BaggageBuilder.channelLink()`** — Fluent setter for the channel link baggage value.

- **`OutputResponse.messages` type changed from `string[]` to `OutputMessages`** — The `OutputMessages` union type (`string[] | OutputMessage[]`) allows passing structured OTEL gen-ai `OutputMessage` objects with `finish_reason`, multi-modal parts, etc. Existing code passing `string[]` continues to work (auto-converted to OTEL format internally).
- **`recordInputMessages()` / `recordOutputMessages()` parameter type widened** — Methods now accept `InputMessages` (`string[] | ChatMessage[]`) and `OutputMessages` (`string[] | OutputMessage[]`). Plain `string[]` input is auto-wrapped to OTEL gen-ai format. These methods are no longer available on `ExecuteToolScope`.

### Added (`@microsoft/agents-a365-observability`)

- **OTEL Gen-AI Message Format types** — New types aligned with [OpenTelemetry Gen-AI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/): `MessageRole`, `FinishReason`, `Modality`, `ChatMessage`, `OutputMessage`, `InputMessages`, `OutputMessages`, and discriminated `MessagePart` union (`TextPart`, `ToolCallRequestPart`, `ToolCallResponsePart`, `ReasoningPart`, `BlobPart`, `FilePart`, `UriPart`, `ServerToolCallPart`, `ServerToolCallResponsePart`, `GenericPart`).
- **`SpanDetails`** — New interface grouping `parentContext`, `startTime`, `endTime`, `spanKind` for scope construction.
- **`CallerDetails`** — New interface wrapping `userDetails` and `callerAgentDetails` for `InvokeAgentScope`.
- **`Request`** — Unified request context interface (`conversationId`, `channel`, `content`, `sessionId`) used across all scopes.
Expand Down
103 changes: 101 additions & 2 deletions packages/agents-a365-observability/docs/design.md
Original file line number Diff line number Diff line change
Expand Up @@ -211,6 +211,103 @@ using scope = ExecuteToolScope.start(
scope.recordResponse('Tool result');
```

#### OutputScope ([OutputScope.ts](../src/tracing/scopes/OutputScope.ts))

Traces outgoing agent output messages:

```typescript
import { OutputScope, OutputResponse } from '@microsoft/agents-a365-observability';

const response: OutputResponse = { messages: ['Hello!', 'How can I help?'] };

using scope = OutputScope.start(
{ conversationId: 'conv-123', channel: { name: 'Teams' } },
response,
agentDetails // Must include tenantId
);

scope.recordOutputMessages(['Additional response']);
// Messages are flushed to the span attribute on dispose
```

### Message Format (OTEL Gen-AI Semantic Conventions)

The SDK uses [OpenTelemetry Gen-AI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/) for message tracing. All `recordInputMessages`/`recordOutputMessages` methods accept both plain strings and structured OTEL message objects.

#### Message Types ([contracts.ts](../src/tracing/contracts.ts))

| Type | Description |
|------|-------------|
| `ChatMessage` | Input message with `role`, `parts[]`, and optional `name` |
| `OutputMessage` | Output message extending `ChatMessage` with `finish_reason` |
| `InputMessages` | Union: `string[] \| ChatMessage[]` |
| `OutputMessages` | Union: `string[] \| OutputMessage[]` |
| `MessageRole` | Enum: `system`, `user`, `assistant`, `tool` |
| `FinishReason` | Enum: `stop`, `length`, `content_filter`, `tool_call`, `error` |
| `MessagePart` | Discriminated union of all content part types |

#### Message Part Types

| Part Type | `type` Discriminator | Purpose |
|-----------|---------------------|---------|
| `TextPart` | `text` | Plain text content |
| `ToolCallRequestPart` | `tool_call` | Tool invocation by the model |
| `ToolCallResponsePart` | `tool_call_response` | Tool execution result |
| `ReasoningPart` | `reasoning` | Chain-of-thought / reasoning content |
| `BlobPart` | `blob` | Inline base64 binary data (image, audio, video) |
| `FilePart` | `file` | Reference to a pre-uploaded file |
| `UriPart` | `uri` | External URI reference |
| `ServerToolCallPart` | `server_tool_call` | Server-side tool invocation |
| `ServerToolCallResponsePart` | `server_tool_call_response` | Server-side tool response |
| `GenericPart` | *(custom)* | Extensible part for future types |

#### Auto-Wrapping Behavior

Plain `string[]` input is automatically wrapped to OTEL format:
- Input strings become `ChatMessage` with `role: 'user'` and a single `TextPart`
- Output strings become `OutputMessage` with `role: 'assistant'` and a single `TextPart`

#### Structured Message Example

```typescript
import { ChatMessage, OutputMessage, MessageRole, FinishReason } from '@microsoft/agents-a365-observability';

// Structured input with system prompt and user message
const input: ChatMessage[] = [
{ role: MessageRole.SYSTEM, parts: [{ type: 'text', content: 'You are a helpful assistant.' }] },
{ role: MessageRole.USER, parts: [{ type: 'text', content: 'What is the weather?' }] }
];
scope.recordInputMessages(input);

// Structured output with tool call and finish reason
const output: OutputMessage[] = [{
role: MessageRole.ASSISTANT,
parts: [
{ type: 'text', content: 'Let me check that for you.' },
{ type: 'tool_call', name: 'get_weather', id: 'call_1', arguments: { city: 'Seattle' } }
],
finish_reason: FinishReason.TOOL_CALL
}];
scope.recordOutputMessages(output);
```

#### Message Serialization and Truncation ([message-utils.ts](../src/tracing/message-utils.ts))

Messages are serialized to JSON and stored as span attributes. When the serialized output exceeds `MAX_ATTRIBUTE_LENGTH` (8192 chars), a binary-search algorithm finds the maximum number of leading messages that fit, appending a sentinel message indicating how many were dropped. Single messages exceeding the limit fall back to string truncation.

#### Scope Visibility

`recordInputMessages`/`recordOutputMessages` are `protected` on the base `OpenTelemetryScope` class and exposed as `public` only on scopes where they are semantically appropriate:

| Scope | `recordInputMessages` | `recordOutputMessages` |
|-------|----------------------|----------------------|
| `InvokeAgentScope` | public | public |
| `InferenceScope` | public | public |
| `OutputScope` | — | public (accumulating) |
| `ExecuteToolScope` | — | — |

`ExecuteToolScope` records tool input/output via `ToolCallDetails.arguments` and `recordResponse()` instead.

### BaggageBuilder ([BaggageBuilder.ts](../src/tracing/middleware/BaggageBuilder.ts))

Fluent API for setting OpenTelemetry baggage:
Expand Down Expand Up @@ -387,12 +484,14 @@ src/
├── ObservabilityBuilder.ts # Configuration builder
├── tracing/
│ ├── constants.ts # OpenTelemetry attribute keys
│ ├── contracts.ts # Data interfaces and enums
│ ├── contracts.ts # Data interfaces, enums, OTEL message types
│ ├── message-utils.ts # Message conversion and serialization
│ ├── scopes/
│ │ ├── OpenTelemetryScope.ts # Base scope class
│ │ ├── InvokeAgentScope.ts # Agent invocation tracing
│ │ ├── InferenceScope.ts # LLM inference tracing
│ │ └── ExecuteToolScope.ts # Tool execution tracing
│ │ ├── ExecuteToolScope.ts # Tool execution tracing
│ │ └── OutputScope.ts # Output message tracing
│ ├── middleware/
│ │ └── BaggageBuilder.ts # Baggage context builder
│ ├── processors/
Expand Down
20 changes: 20 additions & 0 deletions packages/agents-a365-observability/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,13 +39,33 @@ export {
InvokeAgentScopeDetails,
UserDetails,
CallerDetails,
// eslint-disable-next-line @typescript-eslint/no-deprecated -- intentional re-export for backward compatibility
EnhancedAgentDetails,
ServiceEndpoint,
InferenceDetails,
InferenceOperationType,
InferenceResponse,
OutputResponse,
SpanDetails,
// OTEL gen-ai message format types
MessageRole,
FinishReason,
Modality,
TextPart,
ToolCallRequestPart,
ToolCallResponsePart,
ReasoningPart,
BlobPart,
FilePart,
UriPart,
ServerToolCallPart,
ServerToolCallResponsePart,
GenericPart,
MessagePart,
ChatMessage,
OutputMessage,
InputMessages,
OutputMessages,
} from './tracing/contracts';

// Scopes
Expand Down
161 changes: 152 additions & 9 deletions packages/agents-a365-observability/src/tracing/contracts.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ export enum ExecutionType {
Unknown = 'Unknown'
}


/**
* Represents different roles that can invoke an agent
*/
Expand All @@ -43,12 +42,11 @@ export enum InvocationRole {
* Represents different operation for types for model inference
*/
export enum InferenceOperationType {
CHAT = 'Chat',
TEXT_COMPLETION = 'TextCompletion',
GENERATE_CONTENT = 'GenerateContent'
CHAT = 'Chat',
TEXT_COMPLETION = 'TextCompletion',
GENERATE_CONTENT = 'GenerateContent'
}


/**
* Represents channel for an invocation
*/
Expand Down Expand Up @@ -192,7 +190,7 @@ export interface CallerDetails {
callerAgentDetails?: AgentDetails;
}

/*
/**
* @deprecated Use AgentDetails. EnhancedAgentDetails is now an alias of AgentDetails.
*/
export type EnhancedAgentDetails = AgentDetails;
Expand All @@ -209,7 +207,6 @@ export interface ServiceEndpoint {

/** The protocol (e.g., http, https) */
protocol?: string;

}

/**
Expand Down Expand Up @@ -267,16 +264,16 @@ export interface InferenceResponse {

/** Number of output tokens generated */
outputTokens?: number;

}

/**
* Represents a response containing output messages from an agent.
* Used with OutputScope for output message tracing.
* Accepts plain strings or structured OTEL OutputMessage objects.
*/
export interface OutputResponse {
/** The output messages from the agent */
messages: string[];
messages: OutputMessages;
}

/**
Expand All @@ -303,3 +300,149 @@ export interface SpanDetails {
spanLinks?: Link[];
}

// ---------------------------------------------------------------------------
// OpenTelemetry Semantic Convention – Gen-AI Message Format
// https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-input-messages.json
// https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-output-messages.json
// ---------------------------------------------------------------------------

/**
* Role of a message participant per OTEL gen-ai semantic conventions.
*/
export enum MessageRole {
SYSTEM = 'system',
USER = 'user',
ASSISTANT = 'assistant',
TOOL = 'tool'
}

/**
* Reason a model stopped generating per OTEL gen-ai semantic conventions.
*/
export enum FinishReason {
STOP = 'stop',
LENGTH = 'length',
CONTENT_FILTER = 'content_filter',
TOOL_CALL = 'tool_call',
ERROR = 'error'
}

/**
* Media modality for blob, file, and URI parts.
*/
export enum Modality {
IMAGE = 'image',
VIDEO = 'video',
AUDIO = 'audio'
}

// ---- Message part types (discriminated union on `type`) --------------------

/** Plain text content. */
export interface TextPart {
type: 'text';
content: string;
}

/** A tool call requested by the model. */
export interface ToolCallRequestPart {
type: 'tool_call';
name: string;
id?: string;
arguments?: unknown;
}

/** Result of a tool call. */
export interface ToolCallResponsePart {
type: 'tool_call_response';
id?: string;
response?: unknown;
}

/** Model reasoning / chain-of-thought content. */
export interface ReasoningPart {
type: 'reasoning';
content: string;
}

/** Inline binary data (base64-encoded). */
export interface BlobPart {
type: 'blob';
modality: Modality | string;
mime_type?: string;
content: string;
}

/** Reference to a pre-uploaded file. */
export interface FilePart {
type: 'file';
modality: Modality | string;
mime_type?: string;
file_id: string;
}

/** External URI reference. */
export interface UriPart {
type: 'uri';
modality: Modality | string;
mime_type?: string;
uri: string;
}

/** Server-side tool invocation. */
export interface ServerToolCallPart {
type: 'server_tool_call';
name: string;
id?: string;
server_tool_call: Record<string, unknown>;
}

/** Server-side tool response. */
export interface ServerToolCallResponsePart {
type: 'server_tool_call_response';
id?: string;
server_tool_call_response: Record<string, unknown>;
}

/** Extensible part for custom / future types. */
export interface GenericPart {
type: string;
[key: string]: unknown;
}

/**
* Union of all message part types per OTEL gen-ai semantic conventions.
*/
export type MessagePart =
| TextPart
| ToolCallRequestPart
| ToolCallResponsePart
| ReasoningPart
| BlobPart
| FilePart
| UriPart
| ServerToolCallPart
| ServerToolCallResponsePart
| GenericPart;

/**
* An input message sent to a model (OTEL gen-ai semantic conventions).
*/
export interface ChatMessage {
role: MessageRole | string;
parts: MessagePart[];
name?: string;
}

/**
* An output message produced by a model (OTEL gen-ai semantic conventions).
*/
export interface OutputMessage extends ChatMessage {
finish_reason?: FinishReason | string;
}

/** Accepted input for `recordInputMessages`. */
export type InputMessages = string[] | ChatMessage[];

/** Accepted input for `recordOutputMessages`. */
export type OutputMessages = string[] | OutputMessage[];
Loading
Loading