Skip to content

Single-step completions #598

@mbmjertan

Description

@mbmjertan

Currently, every provider call handler implements tool call handling logic directly by calling the tools if the finish reason is a tool call, attaching a ToolResultMessage to the conversation and repeating a request. An example of this implementation is in the OpenAI text handler class: https://github.com/prism-php/prism/blob/7020d47b561f531e30f9a55aa7b9a81f45f657a2/src/Providers/OpenAI/Handlers/Text.php#L74C1-L103

        return match ($this->mapFinishReason($data)) {
            FinishReason::ToolCalls => $this->handleToolCalls($data, $request, $response),
            FinishReason::Stop => $this->handleStop($data, $request, $response),
            FinishReason::Length => throw new PrismException('OpenAI: max tokens excceded'),
            default => throw new PrismException('OpenAI: unknown finish reason'),
        };

However, I don't want this to happen. Some tool calls on a project I'm working on are computationally heavy and take a long time, so I want to enqueue them, block the conversation while the tools are running and continue afterwards.

Therefore, I'd like to be able to configure the handler using the builder interface in a way that Prism only wraps a single completion request, and doesn't continue, but instead just returns the Response object directly. I haven't found a lower-level interface for doing this in Prism, and by digging through the code, I don't think such a mechanism exists.

I'm open to making a PR for this if you are okay with that, but I'm not sure what the API should be.

Would this be okay? Something else?

$response = Prism::text()
    ->using(Provider::Anthropic, 'claude-3-5-sonnet-latest')
    ->withMaxSteps(2)
    ->withPrompt('What is the weather like in Paris?')
    ->withTools([$weatherTool])
    ->withoutToolExecution()
    ->asText();

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions