Add lingua support for streaming tool calls #81
Conversation
| "target": "*", | ||
| "fields": [ | ||
| { "pattern": "choices[*].delta.refusal", "reason": "ChatCompletions refusal field has no equivalent in other providers" }, | ||
| { "pattern": "choices[*].delta.tool_calls", "reason": "Streaming tool_calls transformation not yet implemented" } |
There was a problem hiding this comment.
this is the big one to remove choices[*].delta.tool_calls and what was blocking loop anthropic from working.
There was a problem hiding this comment.
the way the testing works here is the following:
we have coverage-report binary that produces an HTML matrix report -> https://github.com/braintrustdata/lingua/actions/runs/21727047150?pr=81
^ the HTML report is helpful for when u want the AI to crank at supporting smth.
in addition we run https://github.com/braintrustdata/lingua/blob/main/crates/coverage-report/tests/cross_provider_test.rs
which runs cargo test on the coverage-report matrix and basically it'll fail if we see an "unexpected diff" where it's not listed in crates/coverage-report/src/streaming_expected_differences.json.
curerntly we don't have full support for all providers so we are flagging them in one by one, the "strict" mode tests only run for:
const REQUIRED_PROVIDERS: &[ProviderFormat] = &[
ProviderFormat::Responses,
ProviderFormat::OpenAI, // ChatCompletions
ProviderFormat::Anthropic,
];
This stack of pull requests is managed by Graphite. Learn more about stacking. |
knjiang
changed the title
|
looks like there are some code duplication between adapter.rs and responses_adapter.rs, should we add some helpers ? |
| if let Some(delta) = &choice.delta { | ||
| // Check for tool_calls in the delta | ||
| if let Some(tool_calls) = delta.get("tool_calls").and_then(Value::as_array) { | ||
| if let Some(tc) = tool_calls.first() { |
There was a problem hiding this comment.
what if there are more than one tool call ?
| if let Some(delta) = &choice.delta { | ||
| // Check for tool_calls in the delta | ||
| if let Some(tool_calls) = delta.get("tool_calls").and_then(Value::as_array) { | ||
| if let Some(tc) = tool_calls.first() { |
There was a problem hiding this comment.
same, what if there are more than one tool call ?
| // Role-only delta or null content without tool_calls - return empty text_delta | ||
| let content_is_missing_or_null = | ||
| delta.get("content").is_none() || delta.get("content") == Some(&Value::Null); | ||
| let has_tool_calls = delta.get("tool_calls").is_some(); |
There was a problem hiding this comment.
shouldn't we check that tool calls are not empty ?
| let is_initial_metadata = | ||
| (chunk.model.is_some() || chunk.id.is_some() || chunk.usage.is_some()) | ||
| && !has_finish | ||
| && !has_tool_calls |
There was a problem hiding this comment.
When the first chunk from ChatCompletions carries both metadata and tool_calls, this skips message_start entirely and id/model would be lost, is that expected?
2 participants
This PR adds support for responses and anthropic streaming tool calls.