Tools can also return DocumentBlock, ImageBlock, VideoBlock by dpruessner · Pull Request #396 · strands-agents/sdk-typescript

dpruessner · 2026-01-13T00:54:46Z

Description

Enables tools to return DocumentBlock, ImageBlock, and VideoBlock content directly to multi-modal models.

Previously, tools could only return strings or JSON. This PR adds support for rich media blocks, allowing more efficient processing of documents, images, and videos through the Bedrock Converse API.

Key Changes:

Added ToolReturnValue type supporting media blocks
Enhanced tool() helper to accept DocumentBlock/ImageBlock/VideoBlock returns
Updated Bedrock formatting to handle media content natively
Maintained full backward compatibility

Related Issues

Closes #395

Documentation PR

Type of Change

New feature

Testing

How have you tested the change?

I ran npm run check

Checklist

I have read the CONTRIBUTING document
I have added any necessary tests that prove my fix is effective or my feature works
I have updated the documentation accordingly
I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
My changes generate no new warnings
Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

… string and JSONValue

strands-agent · 2026-01-13T04:18:42Z

👋 Welcome @dpruessner and thanks for this contribution!

This looks like an interesting enhancement allowing tools to return richer content types (DocumentBlock, ImageBlock, VideoBlock). This would enable more sophisticated tool outputs beyond simple text.

Hoping maintainers can take a look when they have a chance! 👀

_{🤖 This comment was generated by an AI agent using strands-agents. Workflow Run: 20944495454}

github-actions · 2026-01-26T16:36:46Z

src/tools/__tests__/zod-tool.test.ts

+        expect(result.type).toBe('documentBlock')
+        expect(result.name).toBe('RESULT')
+        expect(result.format).toBe('md')
+      })


Issue: Test coverage is incomplete for media block returns.

Only DocumentBlock return is tested. Consider adding tests for ImageBlock and VideoBlock returns to ensure consistent behavior across all media types.

Suggestion: Add similar tests for ImageBlock and VideoBlock:

it('handles ImageBlock return', async () => { const { ImageBlock } = await import('../../types/media.js') const imgTool = tool({ name: 'create_image', description: 'Creates an image', inputSchema: z.object({ data: z.string() }), callback: (input) => { return new ImageBlock({ format: 'png', source: { bytes: new TextEncoder().encode(input.data) }, }) }, }) const result = await imgTool.invoke({ data: 'test' }) expect(result.type).toBe('imageBlock') expect(result.format).toBe('png') })

github-actions · 2026-01-26T16:36:47Z

Review Summary

Assessment: Comment (Request minor changes before approval)

Key Themes

Strengths:

Clean implementation that extends the existing type system naturally
Proper handling in Bedrock model with _formatDocumentSource and _formatMediaSource
Good backward compatibility maintained
TSDoc documentation is present

Areas Needing Attention:

OpenAI Compatibility: The OpenAI model now silently ignores media blocks in tool results, returning empty strings. This could lead to unexpected behavior when users switch between providers.
Test Coverage: Only DocumentBlock return type is tested. Adding tests for ImageBlock and VideoBlock would improve confidence in the implementation.

Overall

This is a valuable enhancement that enables richer tool outputs. The core implementation in Bedrock and the type system changes look solid. The main suggestions focus on cross-provider compatibility and test coverage to ensure a robust feature.

Nice work on this contribution! 🎉

Move tests to `tests_integ` as `tests-integ` is not a proper module name. Also extract all provider ignoring to a new providers file which centralizes the environment variables needed.

Unshure

Can you also add support for the other model providers at a part of this pr?

Unshure · 2026-02-12T18:59:21Z

src/models/bedrock.ts

+              // eslint-disable-next-line @typescript-eslint/no-explicit-any
+              return { text: `[Unsupported content type: ${(content as any).type}]` }


nit: Lets just remove the default case so we implicitly skip if an unsupported content type is provided. Or we can warn and skip to be explicit.

Unshure · 2026-02-12T19:03:35Z

src/tools/function-tool.ts

  private _wrapInToolResult(value: unknown, toolUseId: string): ToolResultBlock {
    try {
+      // Handle DocumentBlock, ImageBlock, VideoBlock directly
+      if (value instanceof DocumentBlock || value instanceof ImageBlock || value instanceof VideoBlock) {


Can we just do an instance check against ToolResultContent instead? That way TextBlock and JsonBlock are also valid here
https://github.com/strands-agents/sdk-typescript/blob/main/src/types/messages.ts#L195

Suggested change

if (value instanceof DocumentBlock || value instanceof ImageBlock || value instanceof VideoBlock) {

if (value instanceof ToolResultContent) {

Unshure · 2026-02-12T19:06:09Z

src/tools/zod-tool.ts

+ * Valid return types for tool callbacks.
+ * Includes JSON-serializable values and media blocks.
+ */
+export type ToolReturnValue = JSONValue | DocumentBlock | ImageBlock | VideoBlock


Suggested change

export type ToolReturnValue = JSONValue | DocumentBlock | ImageBlock | VideoBlock

export type ToolReturnValue = JSONValue | ToolResultContent

Unshure · 2026-02-12T19:08:57Z

src/types/messages.ts

 * Tool result content block.
 */
-export class ToolResultBlock implements ToolResultBlockData {
+export class ToolResultBlock {


nit: Why no longer implement ToolResultBlockData?

Unshure · 2026-02-12T19:11:34Z

src/tools/__tests__/zod-tool.test.ts

        expect(result).toBe(3)
      })
+
+      it('handles DocumentBlock return', async () => {


Can you add a similar test to the agent.test.ts file to add a tool which returns a document, and create a mock model to invoke that tool, then ensure the loop completes successfully?

https://github.com/strands-agents/sdk-typescript/blob/main/src/agent/__tests__/agent.test.ts#L78

mehtarac

We have additional model providers in the repo now as well such as openai, anthropic, and gemini. Will be beneficial to add implemenatation for all model providers.

mehtarac · 2026-02-12T21:16:42Z

src/tools/function-tool.ts

  private _wrapInToolResult(value: unknown, toolUseId: string): ToolResultBlock {
    try {
+      // Handle DocumentBlock, ImageBlock, VideoBlock directly
+      if (value instanceof DocumentBlock || value instanceof ImageBlock || value instanceof VideoBlock) {


The instanceof check happens before the null check, but media blocks could theoretically be null in edge cases.

Suggestion: Keep the null check first (line 231) before the media block check

mehtarac · 2026-02-12T21:19:16Z

src/types/messages.ts

+        } else if ('image' in contentItem) {
+          return new ImageBlock(contentItem.image as ImageBlockData)
+        } else if ('video' in contentItem) {
+          return new VideoBlock(contentItem.video as VideoBlockData)


I think we can remove the type assertions and let TypeScript infer the types to ensure the data matches the expected shape

mehtarac · 2026-02-12T21:20:43Z

src/tools/__tests__/zod-tool.test.ts

        expect(result).toBe(3)
      })
+
+      it('handles DocumentBlock return', async () => {


Test only verifies the structure but doesn't validate the actual bytes content was preserved. Consider adding assertion to decode and verify the bytes match the input:

const decoded = new TextDecoder().decode(result.source.bytes) expect(decoded).toBe('Hello World!')

Tools can return DocumentBlock, ImageBlock, VideoBlock in addition to…

469c8f2

… string and JSONValue

dpruessner had a problem deploying to manual-approval January 13, 2026 00:54 — with GitHub Actions Failure

github-actions bot reviewed Jan 26, 2026

View reviewed changes

github-actions bot mentioned this pull request Jan 26, 2026

[FEATURE] AgentLoop - Messages - Solidify plan for ToolUse and ToolResponse messages #440

Closed

This comment was marked as off-topic.

Sign in to view

github-actions bot mentioned this pull request Jan 26, 2026

[FEATURE] AgentLoop - Messages - Solidify plan for ToolUse and ToolResponse messages #443

Closed

Unshure self-assigned this Feb 10, 2026

Unshure reviewed Feb 12, 2026

View reviewed changes

dpruessner temporarily deployed to manual-approval February 12, 2026 19:12 — with GitHub Actions Inactive

mehtarac reviewed Feb 12, 2026

View reviewed changes

mehtarac self-assigned this Feb 12, 2026

Unshure removed their assignment Feb 20, 2026

		// eslint-disable-next-line @typescript-eslint/no-explicit-any
		return { text: `[Unsupported content type: ${(content as any).type}]` }

	if (value instanceof DocumentBlock \|\| value instanceof ImageBlock \|\| value instanceof VideoBlock) {
	if (value instanceof ToolResultContent) {

	export type ToolReturnValue = JSONValue \| DocumentBlock \| ImageBlock \| VideoBlock
	export type ToolReturnValue = JSONValue \| ToolResultContent

Conversation

dpruessner commented Jan 13, 2026

Description

Related Issues

Documentation PR

Type of Change

Testing

Checklist

Uh oh!

strands-agent commented Jan 13, 2026

Uh oh!

github-actions bot Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jan 26, 2026

Review Summary

Key Themes

Overall

Uh oh!

This comment was marked as off-topic.

Unshure left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mehtarac left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants