diff --git a/src/pages/docs/ai-transport/messaging/accepting-user-input.mdx b/src/pages/docs/ai-transport/messaging/accepting-user-input.mdx index f13c8e3c70..2a66b7f43d 100644 --- a/src/pages/docs/ai-transport/messaging/accepting-user-input.mdx +++ b/src/pages/docs/ai-transport/messaging/accepting-user-input.mdx @@ -109,6 +109,10 @@ await channel.publish('user-input', { ``` + + ## Subscribe to user input The agent subscribes to a channel to receive messages from users. When a user publishes a message to the channel, the agent receives it through the subscription callback. @@ -191,6 +195,10 @@ await channel.subscribe('agent-response', (message) => { ``` + + ## Stream responses For longer AI responses, you'll typically want to stream tokens back to the user rather than waiting for the complete response. The `promptId` correlation allows users to associate streamed tokens with their original prompt. diff --git a/src/pages/docs/ai-transport/messaging/chain-of-thought.mdx b/src/pages/docs/ai-transport/messaging/chain-of-thought.mdx index 49a9bf1aaa..7c76eecea3 100644 --- a/src/pages/docs/ai-transport/messaging/chain-of-thought.mdx +++ b/src/pages/docs/ai-transport/messaging/chain-of-thought.mdx @@ -82,6 +82,10 @@ for await (const event of stream) { To learn how to stream individual tokens as they are generated, see the [token streaming](/docs/ai-transport/token-streaming) documentation. + + #### Subscribing Subscribe to both reasoning and model output messages on the same channel. @@ -204,6 +208,10 @@ for await (const event of stream) { To learn how to stream individual tokens as they are generated, see the [token streaming](/docs/ai-transport/token-streaming) documentation. + + #### Subscribing Subscribe to the main conversation channel to receive control messages and model output. Subscribe to the reasoning channel on demand, for example in response to a click event. diff --git a/src/pages/docs/ai-transport/messaging/citations.mdx b/src/pages/docs/ai-transport/messaging/citations.mdx index a71ae51795..1944200959 100644 --- a/src/pages/docs/ai-transport/messaging/citations.mdx +++ b/src/pages/docs/ai-transport/messaging/citations.mdx @@ -140,7 +140,11 @@ When streaming response tokens using the [message-per-response](/docs/ai-transpo + + ## Subscribing to summaries diff --git a/src/pages/docs/ai-transport/messaging/human-in-the-loop.mdx b/src/pages/docs/ai-transport/messaging/human-in-the-loop.mdx index b896366da3..3b9c075259 100644 --- a/src/pages/docs/ai-transport/messaging/human-in-the-loop.mdx +++ b/src/pages/docs/ai-transport/messaging/human-in-the-loop.mdx @@ -52,6 +52,10 @@ async function requestHumanApproval(toolCall) { ``` + + ## Review and decide Authorized humans subscribe to approval requests on the conversation channel and publish their decisions. The `requestId` correlates the response with the original request. @@ -99,6 +103,10 @@ async function reject(requestId) { ``` + + ## Process the decision The agent listens for human decisions and acts accordingly. When a response arrives, the agent retrieves the pending request using the `requestId`, verifies that the user is permitted to approve that specific action, and either executes the action or handles the rejection. diff --git a/src/pages/docs/ai-transport/messaging/tool-calls.mdx b/src/pages/docs/ai-transport/messaging/tool-calls.mdx index dddbcfc7eb..d987d2bb26 100644 --- a/src/pages/docs/ai-transport/messaging/tool-calls.mdx +++ b/src/pages/docs/ai-transport/messaging/tool-calls.mdx @@ -94,6 +94,10 @@ Model APIs like OpenAI's [Responses API](https://platform.openai.com/docs/api-re To learn how to stream individual tokens as they are generated, see the [token streaming](/docs/ai-transport/token-streaming) documentation. + + ## Subscribing to tool calls Subscribe to tool call and model output messages on the channel. @@ -239,6 +243,10 @@ await channel.subscribe('tool_call', async (message) => { Client-side tools often require user permission to access device APIs. These permissions are managed by the device operating system, not the agent. Handle permission denials gracefully by publishing an error tool result so the AI can respond appropriately. + + The agent subscribes to tool results to continue processing. The `toolCallId` correlates the result back to the original request: diff --git a/src/pages/docs/ai-transport/token-streaming/message-per-response.mdx b/src/pages/docs/ai-transport/token-streaming/message-per-response.mdx index d37c2ca6d2..3442300f18 100644 --- a/src/pages/docs/ai-transport/token-streaming/message-per-response.mdx +++ b/src/pages/docs/ai-transport/token-streaming/message-per-response.mdx @@ -96,6 +96,10 @@ for await (const event of stream) { Append only supports concatenating data of the same type as the original message. For example, if the initial message data is a string, all appended tokens must also be strings. If the initial message data is binary, all appended tokens must be binary. + + This pattern allows publishing append operations for multiple concurrent model responses on the same channel. As long as you append to the correct message serial, tokens from different responses will not interfere with each other, and the final concatenated message for each response will contain only the tokens from that response. ### Configuring rollup behaviour diff --git a/src/pages/docs/ai-transport/token-streaming/message-per-token.mdx b/src/pages/docs/ai-transport/token-streaming/message-per-token.mdx index 9bddcd8645..434aa6aa57 100644 --- a/src/pages/docs/ai-transport/token-streaming/message-per-token.mdx +++ b/src/pages/docs/ai-transport/token-streaming/message-per-token.mdx @@ -52,6 +52,10 @@ This approach maximizes throughput while maintaining ordering guarantees, allowi Unlike the [message-per-response](/docs/ai-transport/features/token-streaming/message-per-response) pattern, the message-per-token pattern requires you to [manage rate limits directly](/docs/ai-transport/features/token-streaming/token-rate-limits#per-token). + + ## Streaming patterns Ably is a pub/sub messaging platform, so you can structure your messages however works best for your application. Below are common patterns for streaming tokens, each showing both agent-side publishing and client-side subscription. Choose the approach that fits your use case, or create your own variation. diff --git a/src/pages/docs/guides/ai-transport/anthropic-message-per-response.mdx b/src/pages/docs/guides/ai-transport/anthropic-message-per-response.mdx index b8ac5f7c9f..1ce3260825 100644 --- a/src/pages/docs/guides/ai-transport/anthropic-message-per-response.mdx +++ b/src/pages/docs/guides/ai-transport/anthropic-message-per-response.mdx @@ -175,7 +175,10 @@ Add the Ably client initialization to your `publisher.mjs` file: import Ably from 'ably'; // Initialize Ably Realtime client -const realtime = new Ably.Realtime({ key: '{{API_KEY}}' }); +const realtime = new Ably.Realtime({ + key: '{{API_KEY}}', + echoMessages: false +}); // Create a channel for publishing streamed AI responses const channel = realtime.channels.get('ai:{{RANDOM_CHANNEL_NAME}}'); @@ -184,6 +187,10 @@ const channel = realtime.channels.get('ai:{{RANDOM_CHANNEL_NAME}}'); The Ably Realtime client maintains a persistent connection to the Ably service, which allows you to publish tokens at high message rates with low latency. + + ### Publish initial message and append tokens When a new response begins, publish an initial message to create it. Ably assigns a [`serial`](/docs/messages#properties) identifier to the message. Use this `serial` to append each token to the message as it arrives from the Anthropic model. diff --git a/src/pages/docs/guides/ai-transport/anthropic-message-per-token.mdx b/src/pages/docs/guides/ai-transport/anthropic-message-per-token.mdx index 9ccc2ddf50..3b99dd66ca 100644 --- a/src/pages/docs/guides/ai-transport/anthropic-message-per-token.mdx +++ b/src/pages/docs/guides/ai-transport/anthropic-message-per-token.mdx @@ -152,7 +152,10 @@ Add the Ably client initialization to your `publisher.mjs` file: import Ably from 'ably'; // Initialize Ably Realtime client -const realtime = new Ably.Realtime({ key: '{{API_KEY}}' }); +const realtime = new Ably.Realtime({ + key: '{{API_KEY}}', + echoMessages: false +}); // Create a channel for publishing streamed AI responses const channel = realtime.channels.get('{{RANDOM_CHANNEL_NAME}}'); @@ -161,6 +164,10 @@ const channel = realtime.channels.get('{{RANDOM_CHANNEL_NAME}}'); The Ably Realtime client maintains a persistent connection to the Ably service, which allows you to publish tokens at high message rates with low latency. + + ### Map Anthropic streaming events to Ably messages Choose how to map [Anthropic streaming events](#understand-streaming-events) to Ably messages. You can choose any mapping strategy that suits your application's needs. This guide uses the following pattern as an example: diff --git a/src/pages/docs/guides/ai-transport/openai-message-per-response.mdx b/src/pages/docs/guides/ai-transport/openai-message-per-response.mdx index 3cecee670c..77afb9745d 100644 --- a/src/pages/docs/guides/ai-transport/openai-message-per-response.mdx +++ b/src/pages/docs/guides/ai-transport/openai-message-per-response.mdx @@ -189,7 +189,10 @@ Add the Ably client initialization to your `publisher.mjs` file: import Ably from 'ably'; // Initialize Ably Realtime client -const realtime = new Ably.Realtime({ key: '{{API_KEY}}' }); +const realtime = new Ably.Realtime({ + key: '{{API_KEY}}', + echoMessages: false +}); // Create a channel for publishing streamed AI responses const channel = realtime.channels.get('ai:{{RANDOM_CHANNEL_NAME}}'); @@ -198,6 +201,10 @@ const channel = realtime.channels.get('ai:{{RANDOM_CHANNEL_NAME}}'); The Ably Realtime client maintains a persistent connection to the Ably service, which allows you to publish tokens at high message rates with low latency. + + ### Publish initial message and append tokens When a new response begins, publish an initial message to create it. Ably assigns a [`serial`](/docs/messages#properties) identifier to the message. Use this `serial` to append each token to the message as it arrives from the OpenAI model. diff --git a/src/pages/docs/guides/ai-transport/openai-message-per-token.mdx b/src/pages/docs/guides/ai-transport/openai-message-per-token.mdx index 669f4dd246..7573bcbd3f 100644 --- a/src/pages/docs/guides/ai-transport/openai-message-per-token.mdx +++ b/src/pages/docs/guides/ai-transport/openai-message-per-token.mdx @@ -166,7 +166,10 @@ Add the Ably client initialization to your `publisher.mjs` file: import Ably from 'ably'; // Initialize Ably Realtime client -const realtime = new Ably.Realtime({ key: '{{API_KEY}}' }); +const realtime = new Ably.Realtime({ + key: '{{API_KEY}}', + echoMessages: false +}); // Create a channel for publishing streamed AI responses const channel = realtime.channels.get('{{RANDOM_CHANNEL_NAME}}'); @@ -175,6 +178,10 @@ const channel = realtime.channels.get('{{RANDOM_CHANNEL_NAME}}'); The Ably Realtime client maintains a persistent connection to the Ably service, which allows you to publish tokens at high message rates with low latency. + + ### Map OpenAI streaming events to Ably messages Choose how to map [OpenAI streaming events](#understand-streaming-events) to Ably messages. You can choose any mapping strategy that suits your application's needs. This guide uses the following pattern as an example: