Skip to content

feat(2/n): agent return turn awaiting input for tool calls#1187

Merged
yanxi0830 merged 17 commits intoagents-unify-toolsfrom
agents-unify-tools-2
Feb 21, 2025
Merged

feat(2/n): agent return turn awaiting input for tool calls#1187
yanxi0830 merged 17 commits intoagents-unify-toolsfrom
agents-unify-tools-2

Conversation

@yanxi0830
Copy link
Copy Markdown
Contributor

@yanxi0830 yanxi0830 commented Feb 20, 2025

What does this PR do?

This is a backward compat change. Default allow_resume_turn=False to work with client v0.1.3.

Expected Behaviour

  1. User sends out agent.create_turn. When user receives a custom client_tool_call. The last chunk should be
chunk = AgentTurnResponseStreamChunk(
        event=AgentTurnResponseEvent(
        payload=AgentTurnResponseTurnAwaitingInputPayload(
            turn=turn,
       )
    )
)

There should be a tool_execution step.

  1. When user sees turn_pending as the event_type or if there's tool call in the response. They will execute the client tools, and submits back the tool response. E.g.
client.continue_turn(
    [ToolResponseMessage(...)]
)

The agent will then send back & record tool_execution step finish on server. and continue with the execution.

NOTE: This PR handles (1). Next PR handle (2).

Test Plan

  • Testing w/ Client SDK 0.1.3
LLAMA_STACK_BASE_URL=http://localhost:8321 pytest -v tests/client-sdk/agents/test_agents.py --inference-model meta-llama/Llama-3.1-8B-Instruct
image

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Meta Open Source bot. label Feb 20, 2025
@yanxi0830 yanxi0830 changed the base branch from main to agents-unify-tools February 20, 2025 22:19
@yanxi0830 yanxi0830 changed the title wip(2/n) - agent submit_tool_response step feat(2/n): agent return turn awaiting input for tool calls Feb 21, 2025
@yanxi0830 yanxi0830 marked this pull request as ready for review February 21, 2025 00:50
@ashwinb
Copy link
Copy Markdown
Contributor

ashwinb commented Feb 21, 2025

This is a backward compat change.

You mean this is a backward incompatible change, right?

@yanxi0830
Copy link
Copy Markdown
Contributor Author

yanxi0830 commented Feb 21, 2025

This is a backward compat change.

You mean this is a backward incompatible change, right?

Actually, I was referring to backward compat in that this change won't impact anything if client wants to create a separate turn. But I agree that the turn_awaiting_input may not be available for client with older versions.

@yanxi0830 yanxi0830 requested a review from ashwinb February 21, 2025 05:56
# What does this PR do?
- #1178
- #1187
- #1194

**client changes**
- llamastack/llama-stack-client-python#157
- llamastack/llama-stack-client-python#158

## Test Plan
```
LLAMA_STACK_BASE_URL=http://localhost:8321 pytest -v tests/client-sdk/agents/test_agents.py --inference-model meta-llama/Llama-3.1-8B-Instruct
```

```
LLAMA_STACK_CONFIG=fireworks pytest -v tests/client-sdk/agents/test_agents.py --inference-model meta-llama/Llama-3.1-8B-Instruct
```

<img width="1080" alt="image"
src="https://github.com/user-attachments/assets/6c48e062-5d00-4fed-98de-ecca627e5195"
/>

**llama-stack-apps**
```
python -m examples.agents.react_agent localhost 8321
```

- Test with script:
https://gist.github.com/yanxi0830/f2e407527f468998a700cd29fd271b15

**Output Before**: we have 2 `turn_id` with 2 turns
- https://gist.github.com/yanxi0830/9fbd7a80fcddc784a28c59d4a9c1d943

**Output After**: we have 1 `turn_id`, the final turn have all 3 steps
- https://gist.github.com/yanxi0830/17754d56d08ccbeaec419b693137500c

<img width="622" alt="image"
src="https://github.com/user-attachments/assets/ed7f42ca-41c8-4ee9-b6fa-2299901cafb4"
/>

**Telemetry**
<img width="1389" alt="image"
src="https://github.com/user-attachments/assets/031bf6ab-959c-46a7-aa85-70a511cba296"
/>


[//]: # (## Documentation)
@yanxi0830 yanxi0830 merged commit 36a1e8b into agents-unify-tools Feb 21, 2025
3 checks passed
@yanxi0830 yanxi0830 deleted the agents-unify-tools-2 branch February 21, 2025 19:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants