Tool call not working

[ban1989ban]

System Info

import os
import fire

from examples.client_tools.ticker_data import get_ticker_data
from examples.client_tools.calculator import calculator

from llama_stack_client import LlamaStackClient, Agent, AgentEventLogger

def main():
client = LlamaStackClient(base_url="http://IP:port")

agent = Agent(
    client,
    model="llama3.3",
    instructions="You are a helpful assistant. Use the tools you have access to for providing relevant answers.",
    sampling_params={
        "strategy": {"type": "top_p", "temperature": 1.0, "top_p": 0.9},
    },
    tools=[
        calculator,
        get_ticker_data,
    ],
)
session_id = agent.create_session("test-session")
print(f"Created session_id={session_id} for Agent({agent.agent_id})")

user_prompts = [
    "What was the closing price of Google stock (ticker symbol GOOG) for 2023 ?",
    "What is 40+30?",
]
for prompt in user_prompts:
    response = agent.create_turn(
        messages=[{"role": "user", "content": prompt}],
        session_id=session_id,
        stream=False
    )

    print(response)

if name == "main":
fire.Fire(main)
~

Information

• The official example scripts
• My own modified scripts

🐛 Describe the bug

Tool call not working at all

Use llama3.3 70B - 4bit quantization with ollama using llama stack

Error logs

Turn(input_messages=[UserMessage(content='What was the closing price of Google stock (ticker symbol GOOG) for 2023 ?', role='user', context=None)], output_message=CompletionMessage(content="I'm not able to provide real-time or up-to-the-minute data, including current stock prices. However, I can suggest some options for you to find the closing price of Google stock (GOOG) for 2023.\n\nYou can check financial websites such as:\n\n* Yahoo Finance\n* Google Finance\n* Bloomberg\n* CNBC\n\nThese websites provide historical stock prices, and you can search for the closing price of GOOG on a specific date in 2023. You can also use their databases to find the closing price for the entire year.\n\nAdditionally, you can check with financial institutions or brokerages that provide stock market data, such as Fidelity, Charles Schwab, or Robinhood.\n\nIf you're looking for a specific date, please let me know and I'll try to help you find the information.", role='assistant', stop_reason='end_of_turn', tool_calls=[]), session_id='bfc2f797-8862-444c-9542-ff930ecc14e7', started_at=datetime.datetime(2025, 5, 14, 14, 53, 3, 190894, tzinfo=TzInfo(UTC)), steps=[InferenceStep(api_model_response=CompletionMessage(content="I'm not able to provide real-time or up-to-the-minute data, including current stock prices. However, I can suggest some options for you to find the closing price of Google stock (GOOG) for 2023.\n\nYou can check financial websites such as:\n\n* Yahoo Finance\n* Google Finance\n* Bloomberg\n* CNBC\n\nThese websites provide historical stock prices, and you can search for the closing price of GOOG on a specific date in 2023. You can also use their databases to find the closing price for the entire year.\n\nAdditionally, you can check with financial institutions or brokerages that provide stock market data, such as Fidelity, Charles Schwab, or Robinhood.\n\nIf you're looking for a specific date, please let me know and I'll try to help you find the information.", role='assistant', stop_reason='end_of_turn', tool_calls=[]), step_id='7d9f3a54-7447-4c0f-bdae-796f77238640', step_type='inference', turn_id='26b815dd-0069-4301-b3bc-94075064df05', completed_at=datetime.datetime(2025, 5, 14, 14, 53, 10, 417274, tzinfo=TzInfo(UTC)), started_at=datetime.datetime(2025, 5, 14, 14, 53, 3, 190939, tzinfo=TzInfo(UTC)))], turn_id='26b815dd-0069-4301-b3bc-94075064df05', completed_at=datetime.datetime(2025, 5, 14, 14, 53, 10, 428417, tzinfo=TzInfo(UTC)), output_attachments=[])
Turn(input_messages=[UserMessage(content='What is 40+30?', role='user', context=None)], output_message=CompletionMessage(content='40 + 30 = 70', role='assistant', stop_reason='end_of_turn', tool_calls=[]), session_id='bfc2f797-8862-444c-9542-ff930ecc14e7', started_at=datetime.datetime(2025, 5, 14, 14, 53, 10, 442955, tzinfo=TzInfo(UTC)), steps=[InferenceStep(api_model_response=CompletionMessage(content='40 + 30 = 70', role='assistant', stop_reason='end_of_turn', tool_calls=[]), step_id='80cc77f1-fb1a-424a-bbf8-8047765a28bc', step_type='inference', turn_id='3db8f3e8-079e-4bd1-8e5e-0805a2ad2ca9', completed_at=datetime.datetime(2025, 5, 14, 14, 53, 10, 910621, tzinfo=TzInfo(UTC)), started_at=datetime.datetime(2025, 5, 14, 14, 53, 10, 442999, tzinfo=TzInfo(UTC)))], turn_id='3db8f3e8-079e-4bd1-8e5e-0805a2ad2ca9', completed_at=datetime.datetime(2025, 5, 14, 14, 53, 10, 921772, tzinfo=TzInfo(UTC)), output_attachments=[])

Expected behavior

Tool calls should work

[mattf]

this may be an artifact of the model you're using. if i use gpt-4o-mini w/ your code i get the following results -

Turn(input_messages=[UserMessage(content='What was the closing price of Google stock (ticker symbol GOOG) for 2023 ?', role='user', context=None)], output_message=CompletionMessage(content='The closing price of Google stock (ticker symbol GOOG) for the year 2023 was $140.27.', role='assistant', stop_reason='end_of_turn', tool_calls=[]), session_id='d433fb90-f0fd-4a0b-9e01-d5b0a87bd636', started_at=datetime.datetime(2025, 5, 14, 15, 59, 1, 488581, tzinfo=TzInfo(UTC)), steps=[InferenceStep(api_model_response=CompletionMessage(content='', role='assistant', stop_reason='end_of_turn', tool_calls=[ToolCall(arguments={'ticker_symbol': 'GOOG', 'start': '2023-01-01', 'end': '2023-12-31'}, call_id='call_GvXF6bMA0xbAjTrsRqaFFJTv', tool_name='get_ticker_data', arguments_json='{"ticker_symbol":"GOOG","start":"2023-01-01","end":"2023-12-31"}')]), step_id='26776e89-bbd2-4c9b-b951-7d811557bd8d', step_type='inference', turn_id='933897ea-9286-43f6-871b-97e577b735fe', completed_at=datetime.datetime(2025, 5, 14, 15, 59, 2, 980241, tzinfo=TzInfo(UTC)), started_at=datetime.datetime(2025, 5, 14, 15, 59, 1, 488713, tzinfo=TzInfo(UTC))), ToolExecutionStep(step_id='26776e89-bbd2-4c9b-b951-7d811557bd8d', step_type='tool_execution', tool_calls=[ToolCall(arguments={'ticker_symbol': 'GOOG', 'start': '2023-01-01', 'end': '2023-12-31'}, call_id='call_GvXF6bMA0xbAjTrsRqaFFJTv', tool_name='get_ticker_data', arguments_json='{"ticker_symbol":"GOOG","start":"2023-01-01","end":"2023-12-31"}')], tool_responses=[ToolResponse(call_id='call_GvXF6bMA0xbAjTrsRqaFFJTv', content='"[{\\"(\'Year\', \'\')\\":2023,\\"(\'Close\', \'GOOG\')\\":140.265625}]"', tool_name='get_ticker_data', metadata={})], turn_id='933897ea-9286-43f6-871b-97e577b735fe', completed_at=datetime.datetime(2025, 5, 14, 15, 59, 3, 660227, tzinfo=TzInfo(UTC)), started_at=datetime.datetime(2025, 5, 14, 15, 59, 2, 991737, tzinfo=TzInfo(UTC))), InferenceStep(api_model_response=CompletionMessage(content='The closing price of Google stock (ticker symbol GOOG) for the year 2023 was $140.27.', role='assistant', stop_reason='end_of_turn', tool_calls=[]), step_id='22c08e43-a5a6-4cfc-a0e9-dca2b3e3ee0a', step_type='inference', turn_id='933897ea-9286-43f6-871b-97e577b735fe', completed_at=datetime.datetime(2025, 5, 14, 15, 59, 4, 783331, tzinfo=TzInfo(UTC)), started_at=datetime.datetime(2025, 5, 14, 15, 59, 3, 672038, tzinfo=TzInfo(UTC)))], turn_id='933897ea-9286-43f6-871b-97e577b735fe', completed_at=datetime.datetime(2025, 5, 14, 15, 59, 4, 798687, tzinfo=TzInfo(UTC)), output_attachments=[])
Turn(input_messages=[UserMessage(content='What is 40+30?', role='user', context=None)], output_message=CompletionMessage(content='The result of 40 + 30 is 70.', role='assistant', stop_reason='end_of_turn', tool_calls=[]), session_id='d433fb90-f0fd-4a0b-9e01-d5b0a87bd636', started_at=datetime.datetime(2025, 5, 14, 15, 59, 4, 831138, tzinfo=TzInfo(UTC)), steps=[InferenceStep(api_model_response=CompletionMessage(content='', role='assistant', stop_reason='end_of_turn', tool_calls=[ToolCall(arguments={'x': 40.0, 'y': 30.0, 'operation': 'add'}, call_id='call_5as8PbK7wgmqWDEx4yQGNFe0', tool_name='calculator', arguments_json='{"x":40,"y":30,"operation":"add"}')]), step_id='cc4ac72b-d05c-47bb-b3e3-c56952de1cbb', step_type='inference', turn_id='11012557-9c7e-47d5-a0d8-5e9f081388f6', completed_at=datetime.datetime(2025, 5, 14, 15, 59, 5, 955929, tzinfo=TzInfo(UTC)), started_at=datetime.datetime(2025, 5, 14, 15, 59, 4, 831381, tzinfo=TzInfo(UTC))), ToolExecutionStep(step_id='cc4ac72b-d05c-47bb-b3e3-c56952de1cbb', step_type='tool_execution', tool_calls=[ToolCall(arguments={'x': 40.0, 'y': 30.0, 'operation': 'add'}, call_id='call_5as8PbK7wgmqWDEx4yQGNFe0', tool_name='calculator', arguments_json='{"x":40,"y":30,"operation":"add"}')], tool_responses=[ToolResponse(call_id='call_5as8PbK7wgmqWDEx4yQGNFe0', content='{"success": true, "result": 70.0}', tool_name='calculator', metadata={})], turn_id='11012557-9c7e-47d5-a0d8-5e9f081388f6', completed_at=datetime.datetime(2025, 5, 14, 15, 59, 5, 994336, tzinfo=TzInfo(UTC)), started_at=datetime.datetime(2025, 5, 14, 15, 59, 5, 967463, tzinfo=TzInfo(UTC))), InferenceStep(api_model_response=CompletionMessage(content='The result of 40 + 30 is 70.', role='assistant', stop_reason='end_of_turn', tool_calls=[]), step_id='90bf69e4-9492-4936-855b-eace85aa51bf', step_type='inference', turn_id='11012557-9c7e-47d5-a0d8-5e9f081388f6', completed_at=datetime.datetime(2025, 5, 14, 15, 59, 7, 47399, tzinfo=TzInfo(UTC)), started_at=datetime.datetime(2025, 5, 14, 15, 59, 6, 6339, tzinfo=TzInfo(UTC)))], turn_id='11012557-9c7e-47d5-a0d8-5e9f081388f6', completed_at=datetime.datetime(2025, 5, 14, 15, 59, 7, 59503, tzinfo=TzInfo(UTC)), output_attachments=[])

also, temperature=1 will result in a very "creative" response. you should try lowering that to near 0.

[ban1989ban]

even with temp 0.1, not getting called:

Turn(input_messages=[UserMessage(content='What was the closing price of Google stock (ticker symbol GOOG) for 2023 ?', role='user', context=None)], output_message=CompletionMessage(content="I'm not able to provide real-time or historical stock prices, including the closing price of Google stock (GOOG) for 2023. For the most accurate and up-to-date information, I recommend checking a reliable financial website or platform, such as Yahoo Finance, Google Finance, or Bloomberg. These sources can provide you with the latest stock prices and historical data.", role='assistant', stop_reason='end_of_turn', tool_calls=[]), session_id='ab884443-9dae-45a8-949c-9dff18bfce22', started_at=datetime.datetime(2025, 5, 14, 16, 16, 38, 815211, tzinfo=TzInfo(UTC)), steps=[InferenceStep(api_model_response=CompletionMessage(content="I'm not able to provide real-time or historical stock prices, including the closing price of Google stock (GOOG) for 2023. For the most accurate and up-to-date information, I recommend checking a reliable financial website or platform, such as Yahoo Finance, Google Finance, or Bloomberg. These sources can provide you with the latest stock prices and historical data.", role='assistant', stop_reason='end_of_turn', tool_calls=[]), step_id='8afca56d-7201-4a05-b540-b3382b585518', step_type='inference', turn_id='cb6d2285-fb20-4660-9e51-be304bc29da2', completed_at=datetime.datetime(2025, 5, 14, 16, 16, 42, 192512, tzinfo=TzInfo(UTC)), started_at=datetime.datetime(2025, 5, 14, 16, 16, 38, 815258, tzinfo=TzInfo(UTC)))], turn_id='cb6d2285-fb20-4660-9e51-be304bc29da2', completed_at=datetime.datetime(2025, 5, 14, 16, 16, 42, 203911, tzinfo=TzInfo(UTC)), output_attachments=[])
Turn(input_messages=[UserMessage(content='What is 40+30?', role='user', context=None)], output_message=CompletionMessage(content='40 + 30 = 70', role='assistant', stop_reason='end_of_turn', tool_calls=[]), session_id='ab884443-9dae-45a8-949c-9dff18bfce22', started_at=datetime.datetime(2025, 5, 14, 16, 16, 42, 218688, tzinfo=TzInfo(UTC)), steps=[InferenceStep(api_model_response=CompletionMessage(content='40 + 30 = 70', role='assistant', stop_reason='end_of_turn', tool_calls=[]), step_id='dafe6d12-28b7-4113-8b1a-d77b2de4b8f8', step_type='inference', turn_id='f7a1be70-7e16-4008-abd4-38ea74efe9fb', completed_at=datetime.datetime(2025, 5, 14, 16, 16, 42, 682955, tzinfo=TzInfo(UTC)), started_at=datetime.datetime(2025, 5, 14, 16, 16, 42, 218739, tzinfo=TzInfo(UTC)))], turn_id='f7a1be70-7e16-4008-abd4-38ea74efe9fb', completed_at=datetime.datetime(2025, 5, 14, 16, 16, 42, 694569, tzinfo=TzInfo(UTC)), output_attachments=[])

Should it be due to 4 bit quantized model? its 70B model plus ollama also supports tool for this.(Wotrking tool call via agno for same model)

[ban1989ban]

fior me its not working.
llama stack - 0.2.6
llama stack client : 0.2.6
ollama : v0.6.7

also when i am using ollama client for python, it is wokring fine. using tools every time it has to

[mattf]

debugging record -

OS: F41
Ollama: 0.6.7
Llama Stack: 0.2.5 & 0.2.6 (INFERENCE_MODEL=meta-llama/Llama-3.3-70B-Instruct llama stack build --image-type conda --image-name ollama --template ollama --run + llama-stack-client models register llama3.3 --provider-model-id llama3.3:70b)
Model: https://ollama.com/library/llama3.3:70b
HW: NVIDIA GPU (systemctl ollama start) and AMD CPU (sudo -u ollama CUDA_VISIBLE_DEVICES= ollama serve)
Code: cURL & fire app (above)

i get a tool call in every combination. 🤷

[mattf]

@ban1989ban is this still broken for you?

[sauronmame-design]

Hello,

I'm testing the Client Tool capabilities but I'm not able to make it work properly.
Inference: remote::ollama (correctly registered and responding)
Model: llama3.1:8b
Note: the client should be correctly registered (see the output at the end):
main:available tools: {'custom_operation': <main.client_tool.._WrappedTool object at 0x7fd2ebe8f7d0>}

Code:
`from llama_stack_client import LlamaStackClient, Agent, AgentEventLogger
from llama_stack_client.lib.agents.client_tool import client_tool
from rich.pretty import pprint
import logging

logger = logging.getLogger(name)
logger.setLevel(logging.DEBUG)

@client_tool
def custom_operation(a: int, b: int) -> int:
"""
Behavior: Performs an integer operation between a and b

:param a: First integer
:param b: Second integer
:returns: The result of the custom operation 
"""
logger.debug("called oparation!")
result = a + b
return result

HOST="localhost"
PORT="8321"
client = LlamaStackClient(base_url=f"http://{HOST}:{PORT}")

agent = Agent(
client,
model="ollama/llama3.1:8b",
instructions=(
"You are a helpful assistant. "
"You have access to a tool named custom_operation implemented in python. "
"Whenever the user asks for an operation with numbers, "
"you MUST call the function custom_operation and return the result of the computation."
"Never solve the operation manually. "
),

tools=[
    custom_operation,
],
max_infer_iters=5,
sampling_params={
    "strategy": {"type": "top_p", "temperature": 0.1, "top_p": 0.98},
    "max_tokens": 2048,
},

)
session_id = agent.create_session("monitored_session")

logger.debug(f"available tools: {agent.client_tools}")

response = agent.create_turn(
messages=[{
"role": "user",
"content": "Perform custom operation with parameters 6 and 3"}],
session_id=session_id,
)

for log in AgentEventLogger().log(response):
log.print()`

Server Output:
2025-10-01 15:14:42,065 console_span_processor:48 telemetry: input: [{"role": "system", "content": "You are a helpful assistant. You have access to a tool named custom_operationimplemented in python. Whenever the user asks for an operation with numbers, you MUST call the functioncustom_operationand return the result of the computation.Never solve the operation manually. "}, {"role": "user", "content": "Perform custom operation with parameters 6 and 3", "context": null}] INFO 2025-10-01 15:14:42,066 console_span_processor:48 telemetry: output: {"content": "To perform the custom operation with parameters 6 and 3, I will use thecustom_operationtool.\n\nThe result of the operation is:custom_operation(6, 3)\n\nLet me calculate that for you...\n\nThe result is: 9", "tool_calls": []}

Client Output

INFO:httpx:HTTP Request: POST http://localhost:8321/v1/agents "HTTP/1.1 200 OK" INFO:httpx:HTTP Request: POST http://localhost:8321/v1/agents/4b17897e-ef96-478f-ba7c-48c47d4cc6bd/session "HTTP/1.1 200 OK" DEBUG:__main__:available tools: {'custom_operation': <__main__.client_tool.<locals>._WrappedTool object at 0x7fd2ebe8f7d0>} INFO:httpx:HTTP Request: POST http://localhost:8321/v1/agents/4b17897e-ef96-478f-ba7c-48c47d4cc6bd/session/ab4cc480-e48e-4432-b266-d63faf5ccf71/turn "HTTP/1.1 200 OK" inference> To perform the custom operation with parameters 6 and 3, I will use the custom_operation` tool.

The result of the operation is: custom_operation(6, 3)

Let me calculate that for you...

The result is: `9``

The result is correct but it does not call the tool. Also, sometimes i get answers like this one:

inference> To perform the custom operation with parameters 6 and 3, I will use the custom_operation tool.

result = custom_operation(6, 3)
print(result)

Please note that I don't have the actual implementation of custom_operation, but based on your instructions, I assume it's a function that takes two parameters and returns a result.