ref: Remove gen_ai.tool.type assertions from test framework

[ericapisani]

Remove all references to the gen_ai.tool.type attribute from the testing framework.

This attribute is no longer part of the span conventions, so assertions on it
should be removed to avoid false failures as SDKs stop emitting it.

Changes:

• Remove gen_ai.tool.type from checkToolSpanAttributes assertion
• Remove type field from ExpectedToolCall interface and checkToolCalls validation
• Remove type: "function" from tool-call test expectations
• Remove gen_ai.tool.type setAttribute/set_data from manual agent templates (node + python)

Fixes PY-2284

[linear-code]

PY-2284 Remove assertions on `gen_ai.tool.type` within the AI testing framework

[github-actions]

🔴 AI SDK Integration Test Results

Status: 2 regressions detected

Summary

Metric	main	PR	Change
Total Tests	667	667	—
Passed	464	463	-1 ⚠️
Failed	195	196	+1 ⚠️

🔴 Regressions

These tests were passing on main but are now failing:

cloudflare/vercel :: Conversation ID Agent Test (blocking, function, openai)

Error: 1 check(s) failed:

1 check(s) failed:
Chat span 3 gen_ai.conversation.id should be "conv-b" but is "conv-a"

browser/langchain :: Multi-Turn LLM Test (streaming)

Error: Browser test timed out (60s)

Browser test timed out (60s)

✅ Fixed

These tests were failing on main but are now passing:

• cloudflare/vercel :: Basic Agent Test (streaming, function, openai)

Test Matrix

Agent Tests

SDK	Basic Agent Test	Conversation ID Agent Test	Long Input Agent Test	Tool Call Agent Test	Tool Error Agent Test	Vision Agent Test
browser/langgraph	❌blk, combined ❌blk, compiled ❌blk, custom-state ❌blk, graph ❌blk, langchain ❌str, combined ❌str, compiled ❌str, custom-state ❌str, graph ❌str, langchain	❌blk, combined ❌blk, compiled ❌blk, custom-state ❌blk, graph ❌blk, langchain ❌str, combined ❌str, compiled ❌str, custom-state ❌str, graph ❌str, langchain	❌blk, combined ❌blk, compiled ❌blk, custom-state ❌blk, graph ❌blk, langchain ❌str, combined ❌str, compiled ❌str, custom-state ❌str, graph ❌str, langchain	❌blk, combined ❌blk, compiled ❌blk, custom-state ❌blk, graph ❌blk, langchain ❌str, combined ❌str, compiled ❌str, custom-state ❌str, graph ❌str, langchain	❌blk, combined ❌blk, compiled ❌blk, custom-state ❌blk, graph ❌blk, langchain ❌str, combined ❌str, compiled ❌str, custom-state ❌str, graph ❌str, langchain	❌blk, combined ❌blk, compiled ❌blk, custom-state ❌blk, graph ❌blk, langchain ❌str, combined ❌str, compiled ❌str, custom-state ❌str, graph ❌str, langchain
cloudflare/langgraph	❌	❌	❌	❌	❌	❌
cloudflare/vercel	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅🔧str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ❌📉blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai
nextjs/mastra	✅	❌	—	✅	✅	✅
nextjs/vercel	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai
node/langgraph	❌	❌	❌	❌	❌	❌
node/manual	✅	✅	✅	✅	✅	✅
node/mastra	✅	❌	❌	❌	❌	❌
node/vercel	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai	❌blk, class, anthropic ❌blk, class, openai ✅blk, function, anthropic ✅blk, function, openai ❌str, class, anthropic ❌str, class, openai ✅str, function, anthropic ✅str, function, openai
python/langgraph	❌a ❌s	❌a ❌s	❌a ❌s	❌a ❌s	❌a ❌s	❌a ❌s
python/manual	✅a ✅s	✅a ✅s	✅a ✅s	✅a ✅s	✅a ✅s	✅a ✅s
python/openai-agents	✅	✅	✅	✅	✅	❌
python/pydantic-ai	✅a, fallback ✅a, single	✅a, fallback ✅a, single	✅a, fallback ✅a, single	✅a, fallback ✅a, single	✅a, fallback ✅a, single	✅a, fallback ✅a, single

Embedding Tests

SDK	Basic Embeddings Test
browser/google-genai	✅
browser/langchain	❌
browser/openai	✅
cloudflare/google-genai	✅
cloudflare/langchain	❌
cloudflare/openai	✅
cloudflare/vercel	✅
nextjs/google-genai	✅
nextjs/langchain	❌
nextjs/openai	✅
nextjs/vercel	✅
node/google-genai	✅
node/langchain	❌
node/openai	✅
node/vercel	✅
python/google-genai	✅a, blk ✅s, blk
python/langchain	❌a, blk ❌s, blk
python/litellm	❌a, blk ✅s, blk
python/manual	✅a, blk ✅s, blk
python/openai	✅a, blk ✅s, blk

LLM Tests

SDK	Basic Error LLM Test	Basic LLM Test	Conversation ID LLM Test	Long Input LLM Test	Multi-Turn LLM Test	Vision LLM Test
browser/anthropic	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
browser/google-genai	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
browser/langchain	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ❌📉str	✅blk ✅str
browser/openai	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
cloudflare/anthropic	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
cloudflare/google-genai	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
cloudflare/langchain	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
cloudflare/openai	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
nextjs/anthropic	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
nextjs/google-genai	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
nextjs/langchain	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
nextjs/openai	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
node/anthropic	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
node/google-genai	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
node/langchain	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
node/manual	—	✅	✅	✅	✅	✅
node/openai	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str	✅blk ✅str
python/anthropic	✅a, blk ❌a, str ✅s, blk ❌s, str	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str
python/google-genai	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str	❌a, blk ❌a, str ❌s, blk ❌s, str
python/langchain	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ✅a, str ✅s, blk ✅s, str
python/litellm	❌a, blk ❌a, str ❌s, blk ❌s, str	❌a, blk ❌a, str ✅s, blk ✅s, str	❌a, blk ❌a, str ✅s, blk ✅s, str	❌a, blk ❌a, str ✅s, blk ✅s, str	❌a, blk ❌a, str ✅s, blk ✅s, str	❌a, blk ❌a, str ✅s, blk ✅s, str
python/manual	—	✅a, blk ✅s, blk	✅a, blk ✅s, blk	✅a, blk ✅s, blk	✅a, blk ✅s, blk	✅a, blk ✅s, blk
python/openai	✅a, blk ✅a, str ✅s, blk ✅s, str	✅a, blk ❌a, str ✅s, blk ❌s, str	✅a, blk ❌a, str ✅s, blk ❌s, str	✅a, blk ❌a, str ✅s, blk ❌s, str	✅a, blk ❌a, str ✅s, blk ❌s, str	❌a, blk ❌a, str ❌s, blk ❌s, str

MCP Tests

SDK	Basic MCP Tool Call Test	MCP Multiple Tool Calls Test	MCP Prompt Get Test	MCP Resource Read Test	MCP Tool Error Test
node/mcp	✅sse ✅io	✅sse ✅io	✅sse ✅io	✅sse ✅io	✅sse ✅io
python/fastmcp	✅a, blk, sse ✅a, blk, io	✅a, blk, sse ✅a, blk, io	✅a, blk, sse ✅a, blk, io	✅a, blk, sse ✅a, blk, io	✅a, blk, sse ✅a, blk, io
python/mcp	✅a, blk, sse, hi ✅a, blk, sse, lo ✅a, blk, io, hi ✅a, blk, io, lo	✅a, blk, sse, hi ✅a, blk, sse, lo ✅a, blk, io, hi ✅a, blk, io, lo	✅a, blk, sse, hi ✅a, blk, sse, lo ✅a, blk, io, hi ✅a, blk, io, lo	✅a, blk, sse, hi ✅a, blk, sse, lo ✅a, blk, io, hi ✅a, blk, io, lo	✅a, blk, sse, hi ✅a, blk, sse, lo ✅a, blk, io, hi ✅a, blk, io, lo

Legend: ✅ Pass | ❌ Fail | ✅🔧 Fixed | ❌📉 Regressed | ✅🆕 New (pass) | ❌🆕 New (fail) | 🗑️ Removed | _str=streaming _blk=blocking _a=async _s=sync _io=stdio _sse=sse _hi=highlevel _lo=lowlevel

---

Generated by AI SDK Integration Tests