Skip to content

fix: Support 'args' key in response tool call argument extraction#138

Merged
ericapisani merged 1 commit intomainfrom
ep/py-2281-update-tool-call-args-check-5yx
Apr 10, 2026
Merged

fix: Support 'args' key in response tool call argument extraction#138
ericapisani merged 1 commit intomainfrom
ep/py-2281-update-tool-call-args-check-5yx

Conversation

@ericapisani
Copy link
Copy Markdown
Member

Add foundCall.args as a fallback when extracting arguments from tool
call response data in checkResponseToolCalls.

Some frameworks (e.g. LangGraph Python) return tool calls with an args
key rather than arguments:

{"name": "read_file", "args": {"path": "/nonexistent/file.txt"}, "id": "call_...", "type": "tool_call"}

The lookup chain is now: argumentsargsfunction.arguments.

Fixes PY-2281

Some frameworks (e.g. LangGraph Python) return tool calls with an
'args' key instead of 'arguments'. Add 'args' as a fallback in
checkResponseToolCalls so these tool calls are validated correctly.

Refs PY-2281
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@linear-code
Copy link
Copy Markdown

linear-code bot commented Apr 8, 2026

@ericapisani ericapisani marked this pull request as ready for review April 8, 2026 18:20
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 8, 2026

🔴 AI SDK Integration Test Results

Status: 8 regressions detected

Summary

Metric main PR Change
Total Tests 667 667
Passed 458 457 -1 ⚠️
Failed 201 194 -7 ✅

🔴 Regressions

These tests were passing on main but are now failing:

cloudflare/langchain :: Basic LLM Test (blocking)

Error: Test execution failed: Wrangler exited with code 1

Test execution failed: Wrangler exited with code 1
stdout: 
 ⛅️ wrangler 4.81.0
───────────────────
Using secrets defined in .dev.vars
Your Worker has access to the following bindings:
Binding                                                                  Resource                  Mode
env.SENTRY_DSN ("http://public@localhost:35691/41215802")                Environment Variable      local
env.RUN_ID ("run-1775672397416-d6b47l9")                                 Environment Variable      local
env.OPENAI_API_KEY ("(hidden)")                                          Environment Variable      local
env.ANTHROPIC_API_KEY ("(hidden)")                                       Environment Variable      local
env.GOOGLE_GENAI_API_KEY ("(hidden)")                                    Environment Variable      local

*** Fatal uncaught kj::Exception: kj/async-io-unix.c++:945: failed: ::bind(sockfd, &addr.generic, addrlen): Address already in use; toString() = [::1]:41020
stack: /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@5143ab6 /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@5143889 /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@5141d7d /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@200db9c /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@200e501 /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@200f02d /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@20106aa /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1f64289 /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@5177765 /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@5177c88 /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@517574e /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@517554e /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1f4bd15 /lib/x86_64-linux-gnu/libc.so.6@2a1c9 /lib/x86_64-linux-gnu/libc.so.6@2a28a /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1f4b024


stderr: �[31m✘ �[41;31m[�[41;97mERROR�[41;31m]�[0m �[1mAddress already in use ([::1]:41020). Please check that you are not already running a server on this address or specify a different port with --port.�[0m


🪵  Logs were written to "/home/runner/.config/.wrangler/logs/wrangler-2026-04-08_18-36-51_255.log"

cloudflare/langchain :: Multi-Turn LLM Test (blocking)

Error: Test execution failed: Wrangler exited with code 1

Test execution failed: Wrangler exited with code 1
stdout: 
 ⛅️ wrangler 4.81.0
───────────────────
Using secrets defined in .dev.vars
Your Worker has access to the following bindings:
Binding                                                                   Resource                  Mode
env.SENTRY_DSN ("http://public@localhost:35691/951992546")                Environment Variable      local
env.RUN_ID ("run-1775672397416-c5fjnxx")                                  Environment Variable      local
env.OPENAI_API_KEY ("(hidden)")                                           Environment Variable      local
env.ANTHROPIC_API_KEY ("(hidden)")                                        Environment Variable      local
env.GOOGLE_GENAI_API_KEY ("(hidden)")                                     Environment Variable      local

*** Fatal uncaught kj::Exception: kj/async-io-unix.c++:945: failed: ::bind(sockfd, &addr.generic, addrlen): Address already in use; toString() = [::1]:41022
stack: /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@5143ab6 /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@5143889 /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@5141d7d /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@200db9c /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@200e501 /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@200f02d /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@20106aa /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1f64289 /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@5177765 /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@5177c88 /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@517574e /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@517554e /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1f4bd15 /lib/x86_64-linux-gnu/libc.so.6@2a1c9 /lib/x86_64-linux-gnu/libc.so.6@2a28a /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1f4b024


stderr: �[31m✘ �[41;31m[�[41;97mERROR�[41;31m]�[0m �[1mAddress already in use ([::1]:41022). Please check that you are not already running a server on this address or specify a different port with --port.�[0m


🪵  Logs were written to "/home/runner/.config/.wrangler/logs/wrangler-2026-04-08_18-36-53_218.log"

cloudflare/langchain :: Basic Error LLM Test (blocking)

Error: Test execution failed: Wrangler exited with code 1

Test execution failed: Wrangler exited with code 1
stdout: 
 ⛅️ wrangler 4.81.0
───────────────────
Using secrets defined in .dev.vars
Your Worker has access to the following bindings:
Binding                                                                    Resource                  Mode
env.SENTRY_DSN ("http://public@localhost:35691/1636814...")                Environment Variable      local
env.RUN_ID ("run-1775672397416-tfkky35")                                   Environment Variable      local
env.OPENAI_API_KEY ("(hidden)")                                            Environment Variable      local
env.ANTHROPIC_API_KEY ("(hidden)")                                         Environment Variable      local
env.GOOGLE_GENAI_API_KEY ("(hidden)")                                      Environment Variable      local

*** Fatal uncaught kj::Exception: kj/async-io-unix.c++:945: failed: ::bind(sockfd, &addr.generic, addrlen): Address already in use; toString() = [::1]:41024
stack: /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@5143ab6 /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@5143889 /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@5141d7d /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@200db9c /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@200e501 /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@200f02d /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@20106aa /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1f64289 /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@5177765 /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@5177c88 /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@517574e /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@517554e /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1f4bd15 /lib/x86_64-linux-gnu/libc.so.6@2a1c9 /lib/x86_64-linux-gnu/libc.so.6@2a28a /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1f4b024


stderr: �[31m✘ �[41;31m[�[41;97mERROR�[41;31m]�[0m �[1mAddress already in use ([::1]:41024). Please check that you are not already running a server on this address or specify a different port with --port.�[0m


🪵  Logs were written to "/home/runner/.config/.wrangler/logs/wrangler-2026-04-08_18-36-55_305.log"

cloudflare/langchain :: Vision LLM Test (blocking)

Error: Test execution failed: Wrangler exited with code 1

Test execution failed: Wrangler exited with code 1
stdout: 
 ⛅️ wrangler 4.81.0
───────────────────
Using secrets defined in .dev.vars
Your Worker has access to the following bindings:
Binding                                                                 Resource                  Mode
env.SENTRY_DSN ("http://public@localhost:35691/3366433")                Environment Variable      local
env.RUN_ID ("run-1775672397416-i44nf71")                                Environment Variable      local
env.OPENAI_API_KEY ("(hidden)")                                         Environment Variable      local
env.ANTHROPIC_API_KEY ("(hidden)")                                      Environment Variable      local
env.GOOGLE_GENAI_API_KEY ("(hidden)")                                   Environment Variable      local

*** Fatal uncaught kj::Exception: kj/async-io-unix.c++:945: failed: ::bind(sockfd, &addr.generic, addrlen): Address already in use; toString() = [::1]:41026
stack: /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@5143ab6 /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@5143889 /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@5141d7d /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@200db9c /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@200e501 /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@200f02d /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@20106aa /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1f64289 /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@5177765 /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@5177c88 /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@517574e /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@517554e /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1f4bd15 /lib/x86_64-linux-gnu/libc.so.6@2a1c9 /lib/x86_64-linux-gnu/libc.so.6@2a28a /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/cloudflare/langchain-llm-0.3.24-sentry-latest/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1f4b024


stderr: �[31m✘ �[41;31m[�[41;97mERROR�[41;31m]�[0m �[1mAddress already in use ([::1]:41026). Please check that you are not already running a server on this address or specify a different port with --port.�[0m


🪵  Logs were written to "/home/runner/.config/.wrangler/logs/wrangler-2026-04-08_18-36-58_465.log"

python/google-genai :: Basic LLM Test (sync, blocking)

Error: 2 check(s) failed:

2 check(s) failed:
Attribute validation failed:
  Span 84f1f7f9: Attribute 'gen_ai.usage.input_tokens' must exist but is missing
  Span 84f1f7f9: Attribute 'gen_ai.usage.output_tokens' must exist but is missing
Token usage validation failed:
  input_tokens must exist
  output_tokens must exist
  total_tokens must exist
gen_ai.response.model is missing (optional but recommended)
python/fastmcp :: MCP Resource Read Test (async, blocking, sse)

Error: Test execution failed: Command failed: /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/fastmcp-mcp-2.3.4-sentry-latest/.venv/bin/python /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/fastmcp-mcp-2.3.4-sentry-latest/test-mcp-resource-read-test-async-blocking-sse.py

Test execution failed: Command failed: /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/fastmcp-mcp-2.3.4-sentry-latest/.venv/bin/python /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/fastmcp-mcp-2.3.4-sentry-latest/test-mcp-resource-read-test-async-blocking-sse.py
[04/08/26 18:39:58] INFO     Starting server "test-server"...      server.py:214
INFO:     Started server process [21007]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
ERROR:    [Errno 98] error while attempting to bind on address ('127.0.0.1', 41046): [errno 98] address already in use
INFO:     Waiting for application shutdown.
INFO:     Application shutdown complete.
Traceback (most recent call last):
  File "/home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/fastmcp-mcp-2.3.4-sentry-latest/test-mcp-resource-read-test-async-blocking-sse.py", line 66, in <module>
    asyncio.run(main())
  File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/asyncio/runners.py", line 195, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/asyncio/base_events.py", line 691, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/fastmcp-mcp-2.3.4-sentry-latest/test-mcp-resource-read-test-async-blocking-sse.py", line 56, in main
    _wait_for_port(SSE_PORT)
  File "/home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/fastmcp-mcp-2.3.4-sentry-latest/test-mcp-resource-read-test-async-blocking-sse.py", line 44, in _wait_for_port
    raise RuntimeError(f"Server on {host}:{port} did not start within {timeout}s")
RuntimeError: Server on 127.0.0.1:41046 did not start within 10s

[04/08/26 18:39:58] INFO     Starting server "test-server"...      server.py:214
INFO:     Started server process [21007]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
ERROR:    [Errno 98] error while attempting to bind on address ('127.0.0.1', 41046): [errno 98] address already in use
INFO:     Waiting for application shutdown.
INFO:     Application shutdown complete.
Traceback (most recent call last):
  File "/home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/fastmcp-mcp-2.3.4-sentry-latest/test-mcp-resource-read-test-async-blocking-sse.py", line 66, in <module>
    asyncio.run(main())
  File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/asyncio/runners.py", line 195, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/asyncio/base_events.py", line 691, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/fastmcp-mcp-2.3.4-sentry-latest/test-mcp-resource-read-test-async-blocking-sse.py", line 56, in main
    _wait_for_port(SSE_PORT)
  File "/home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/fastmcp-mcp-2.3.4-sentry-latest/test-mcp-resource-read-test-async-blocking-sse.py", line 44, in _wait_for_port
    raise RuntimeError(f"Server on {host}:{port} did not start within {timeout}s")
RuntimeError: Server on 127.0.0.1:41046 did not start within 10s

python/mcp :: MCP Multiple Tool Calls Test (async, blocking, sse, highlevel)

Error: Test execution failed: Command failed: /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/mcp-mcp-1.9.4-sentry-latest/.venv/bin/python /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/mcp-mcp-1.9.4-sentry-latest/test-mcp-multiple-tool-calls-test-async-blocking-sse-highlevel.py

Test execution failed: Command failed: /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/mcp-mcp-1.9.4-sentry-latest/.venv/bin/python /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/mcp-mcp-1.9.4-sentry-latest/test-mcp-multiple-tool-calls-test-async-blocking-sse-highlevel.py
INFO:     Started server process [21088]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
ERROR:    [Errno 98] error while attempting to bind on address ('127.0.0.1', 41052): [errno 98] address already in use
INFO:     Waiting for application shutdown.
INFO:     Application shutdown complete.
Traceback (most recent call last):
  File "/home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/mcp-mcp-1.9.4-sentry-latest/test-mcp-multiple-tool-calls-test-async-blocking-sse-highlevel.py", line 89, in <module>
    asyncio.run(main())
  File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/asyncio/runners.py", line 195, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/asyncio/base_events.py", line 691, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/mcp-mcp-1.9.4-sentry-latest/test-mcp-multiple-tool-calls-test-async-blocking-sse-highlevel.py", line 69, in main
    _wait_for_port(SSE_PORT)
  File "/home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/mcp-mcp-1.9.4-sentry-latest/test-mcp-multiple-tool-calls-test-async-blocking-sse-highlevel.py", line 55, in _wait_for_port
    raise RuntimeError(f"Server on {host}:{port} did not start within {timeout}s")
RuntimeError: Server on 127.0.0.1:41052 did not start within 10s

INFO:     Started server process [21088]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
ERROR:    [Errno 98] error while attempting to bind on address ('127.0.0.1', 41052): [errno 98] address already in use
INFO:     Waiting for application shutdown.
INFO:     Application shutdown complete.
Traceback (most recent call last):
  File "/home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/mcp-mcp-1.9.4-sentry-latest/test-mcp-multiple-tool-calls-test-async-blocking-sse-highlevel.py", line 89, in <module>
    asyncio.run(main())
  File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/asyncio/runners.py", line 195, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/asyncio/base_events.py", line 691, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/mcp-mcp-1.9.4-sentry-latest/test-mcp-multiple-tool-calls-test-async-blocking-sse-highlevel.py", line 69, in main
    _wait_for_port(SSE_PORT)
  File "/home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/mcp-mcp-1.9.4-sentry-latest/test-mcp-multiple-tool-calls-test-async-blocking-sse-highlevel.py", line 55, in _wait_for_port
    raise RuntimeError(f"Server on {host}:{port} did not start within {timeout}s")
RuntimeError: Server on 127.0.0.1:41052 did not start within 10s

python/mcp :: MCP Resource Read Test (async, blocking, sse, highlevel)

Error: Test execution failed: Command failed: /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/mcp-mcp-1.9.4-sentry-latest/.venv/bin/python /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/mcp-mcp-1.9.4-sentry-latest/test-mcp-resource-read-test-async-blocking-sse-highlevel.py

Test execution failed: Command failed: /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/mcp-mcp-1.9.4-sentry-latest/.venv/bin/python /home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/mcp-mcp-1.9.4-sentry-latest/test-mcp-resource-read-test-async-blocking-sse-highlevel.py
INFO:     Started server process [21097]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
ERROR:    [Errno 98] error while attempting to bind on address ('127.0.0.1', 41054): [errno 98] address already in use
INFO:     Waiting for application shutdown.
INFO:     Application shutdown complete.
Traceback (most recent call last):
  File "/home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/mcp-mcp-1.9.4-sentry-latest/test-mcp-resource-read-test-async-blocking-sse-highlevel.py", line 76, in <module>
    asyncio.run(main())
  File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/asyncio/runners.py", line 195, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/asyncio/base_events.py", line 691, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/mcp-mcp-1.9.4-sentry-latest/test-mcp-resource-read-test-async-blocking-sse-highlevel.py", line 64, in main
    _wait_for_port(SSE_PORT)
  File "/home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/mcp-mcp-1.9.4-sentry-latest/test-mcp-resource-read-test-async-blocking-sse-highlevel.py", line 50, in _wait_for_port
    raise RuntimeError(f"Server on {host}:{port} did not start within {timeout}s")
RuntimeError: Server on 127.0.0.1:41054 did not start within 10s

INFO:     Started server process [21097]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
ERROR:    [Errno 98] error while attempting to bind on address ('127.0.0.1', 41054): [errno 98] address already in use
INFO:     Waiting for application shutdown.
INFO:     Application shutdown complete.
Traceback (most recent call last):
  File "/home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/mcp-mcp-1.9.4-sentry-latest/test-mcp-resource-read-test-async-blocking-sse-highlevel.py", line 76, in <module>
    asyncio.run(main())
  File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/asyncio/runners.py", line 195, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/asyncio/base_events.py", line 691, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/mcp-mcp-1.9.4-sentry-latest/test-mcp-resource-read-test-async-blocking-sse-highlevel.py", line 64, in main
    _wait_for_port(SSE_PORT)
  File "/home/runner/work/testing-ai-sdk-integrations/testing-ai-sdk-integrations/runs/python/mcp-mcp-1.9.4-sentry-latest/test-mcp-resource-read-test-async-blocking-sse-highlevel.py", line 50, in _wait_for_port
    raise RuntimeError(f"Server on {host}:{port} did not start within {timeout}s")
RuntimeError: Server on 127.0.0.1:41054 did not start within 10s

✅ Fixed

These tests were failing on main but are now passing:

  • cloudflare/vercel :: Conversation ID Agent Test (streaming, function, openai)
  • cloudflare/vercel :: Conversation ID Agent Test (streaming, function, anthropic)
  • cloudflare/google-genai :: Basic Embeddings Test (blocking)
  • browser/langchain :: Multi-Turn LLM Test (blocking)
  • browser/openai :: Multi-Turn LLM Test (blocking)
  • python/google-genai :: Multi-Turn LLM Test (sync, blocking)
  • python/google-genai :: Conversation ID LLM Test (sync, blocking)

Test Matrix

Agent Tests

SDK Basic Agent Test Conversation ID Agent Test Long Input Agent Test Tool Call Agent Test Tool Error Agent Test Vision Agent Test
browser/langgraph blk, combinedblk, compiledblk, custom-stateblk, graphblk, langchainstr, combinedstr, compiledstr, custom-statestr, graphstr, langchain blk, combinedblk, compiledblk, custom-stateblk, graphblk, langchainstr, combinedstr, compiledstr, custom-statestr, graphstr, langchain blk, combinedblk, compiledblk, custom-stateblk, graphblk, langchainstr, combinedstr, compiledstr, custom-statestr, graphstr, langchain blk, combinedblk, compiledblk, custom-stateblk, graphblk, langchainstr, combinedstr, compiledstr, custom-statestr, graphstr, langchain blk, combinedblk, compiledblk, custom-stateblk, graphblk, langchainstr, combinedstr, compiledstr, custom-statestr, graphstr, langchain blk, combinedblk, compiledblk, custom-stateblk, graphblk, langchainstr, combinedstr, compiledstr, custom-statestr, graphstr, langchain
cloudflare/langgraph
cloudflare/vercel blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openai ✅🔧str, function, anthropic ✅🔧str, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai
nextjs/mastra
nextjs/vercel blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai
node/langgraph
node/manual
node/mastra
node/vercel blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai
python/langgraph as as as as as as
python/manual as as as as as as
python/openai-agents
python/pydantic-ai a, fallbacka, single a, fallbacka, single a, fallbacka, single a, fallbacka, single a, fallbacka, single a, fallbacka, single

Embedding Tests

SDK Basic Embeddings Test
browser/google-genai
browser/langchain
browser/openai
cloudflare/google-genai ✅🔧
cloudflare/langchain
cloudflare/openai
cloudflare/vercel
nextjs/google-genai
nextjs/langchain
nextjs/openai
nextjs/vercel
node/google-genai
node/langchain
node/openai
node/vercel
python/google-genai a, blks, blk
python/langchain a, blks, blk
python/litellm a, blks, blk
python/manual a, blks, blk
python/openai a, blks, blk

LLM Tests

SDK Basic Error LLM Test Basic LLM Test Conversation ID LLM Test Long Input LLM Test Multi-Turn LLM Test Vision LLM Test
browser/anthropic blkstr blkstr blkstr blkstr blkstr blkstr
browser/google-genai blkstr blkstr blkstr blkstr blkstr blkstr
browser/langchain blkstr blkstr blkstr blkstr ✅🔧blkstr blkstr
browser/openai blkstr blkstr blkstr blkstr ✅🔧blkstr blkstr
cloudflare/anthropic blkstr blkstr blkstr blkstr blkstr blkstr
cloudflare/google-genai blkstr blkstr blkstr blkstr blkstr blkstr
cloudflare/langchain ❌📉blkstr ❌📉blkstr blkstr blkstr ❌📉blkstr ❌📉blkstr
cloudflare/openai blkstr blkstr blkstr blkstr blkstr blkstr
nextjs/anthropic blkstr blkstr blkstr blkstr blkstr blkstr
nextjs/google-genai blkstr blkstr blkstr blkstr blkstr blkstr
nextjs/langchain blkstr blkstr blkstr blkstr blkstr blkstr
nextjs/openai blkstr blkstr blkstr blkstr blkstr blkstr
node/anthropic blkstr blkstr blkstr blkstr blkstr blkstr
node/google-genai blkstr blkstr blkstr blkstr blkstr blkstr
node/langchain blkstr blkstr blkstr blkstr blkstr blkstr
node/manual
node/openai blkstr blkstr blkstr blkstr blkstr blkstr
python/anthropic a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str
python/google-genai a, blka, strs, blks, str a, blka, str ❌📉s, blks, str a, blka, str ✅🔧s, blks, str a, blka, strs, blks, str a, blka, str ✅🔧s, blks, str a, blka, strs, blks, str
python/langchain a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str
python/litellm a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str
python/manual a, blks, blk a, blks, blk a, blks, blk a, blks, blk a, blks, blk
python/openai a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str

MCP Tests

SDK Basic MCP Tool Call Test MCP Multiple Tool Calls Test MCP Prompt Get Test MCP Resource Read Test MCP Tool Error Test
node/mcp sseio sseio sseio sseio sseio
python/fastmcp a, blk, ssea, blk, io a, blk, ssea, blk, io a, blk, ssea, blk, io ❌📉a, blk, ssea, blk, io a, blk, ssea, blk, io
python/mcp a, blk, sse, hia, blk, sse, loa, blk, io, hia, blk, io, lo ❌📉a, blk, sse, hia, blk, sse, loa, blk, io, hia, blk, io, lo a, blk, sse, hia, blk, sse, loa, blk, io, hia, blk, io, lo ❌📉a, blk, sse, hia, blk, sse, loa, blk, io, hia, blk, io, lo a, blk, sse, hia, blk, sse, loa, blk, io, hia, blk, io, lo

Legend: ✅ Pass | ❌ Fail | ✅🔧 Fixed | ❌📉 Regressed | ✅🆕 New (pass) | ❌🆕 New (fail) | 🗑️ Removed | str=streaming blk=blocking a=async s=sync io=stdio sse=sse hi=highlevel lo=lowlevel


Generated by AI SDK Integration Tests

@constantinius
Copy link
Copy Markdown
Collaborator

constantinius commented Apr 8, 2026

We the same issue for JS langgraph. My suggestion would have been to align in the SDK.

EDIT: we decided to do this anyways

@ericapisani ericapisani merged commit 1dd9ee2 into main Apr 10, 2026
10 of 12 checks passed
@ericapisani ericapisani deleted the ep/py-2281-update-tool-call-args-check-5yx branch April 10, 2026 11:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants