Multi-turn conversation

[soukicz]

I am trying to migrate project from direct api calls to llm-chain and I can't find anywhere how should I use it for multi-turn conversation.

I have this specific example: LLM is tasked with generating something and hits maximum response limit. I can check that with stop_reason, add "continue" as next message and call API again. How should I do this with lllm-chain? There isn't stop_reason in response and I also cannot add last message for MessageBag because there tool messages abstracted away.

Is there a way to continue chain of conversation that I am missing? Or is my general approach wrong and this type of task should be handled somehow through tools?

Similar issue is with feedback loop where I want to check result (for example syntax check) and return it for fixes but that should probably be done with mandatory tools and not multi-turn chat.

Thank you for any help. I really like how llm-chain works (and that it even exists in PHP) but I just cannot wrap my head around this specific problem.

[chr-hertel]

Good one - i'm afraid the abstraction is currently too restrictive about that - one solution could be to attach the "rawResponse" into the response - you could than implement your own OutputProcessor or similar.

WDYT? would be open for a PR about the raw response thing if you think that's helpful and you have the time for that :)

[soukicz]

Of course, I was mainly asking if I am missing something :)

I will try to port pattern I have been using so far - optional feedback loop callable argument. This callable receives final result of LLM (what call() now returns), stop reason and atempt counter. This callable could either return nothing and call() finishes as usual. Or it can return UserMessage and llm-chain would add this message to message bag, sends it back to LLM (like it does with tools) and repeats this process until callable returns nothing.

It should be relatively simple add this and is nice to use in this way - the most complicated part is as usual to figure out where to add to current api to keep it all nice and tidy. It is relevant to input MessageBag so probably same way as output_structure is passed as argument?

[soukicz]

And I just realized that it doesn't work here. This would solve validation because it forces LLM to go back, try again and return correct response. But it doesn't solve continuation because I would still get only last response and whole conversation. But maybe that is its own problem that shouldn't be solved by same thing.

[chr-hertel]

You are referring to finish_reason=length are you? or maybe i'm still missing the point

Otherwise, with "usual" finish_reason=stop i would just implement that loop like here:
https://github.com/php-llm/symfony-demo/blob/main/src/Blog/Chat.php

or a bit like this?

llm-chain/examples/persistent-chat.php

Lines 24 to 34 in a133112

	$chat = new Chat($chain, new InMemoryStore());
	$messages = new MessageBag(
	Message::forSystem('You are a helpful assistant. You only answer with short sentences.'),
	);
	$chat->start($messages);
	$chat->submit(Message::ofUser('My name is Christopher.'));
	$message = $chat->submit(Message::ofUser('What is my name?'));
	echo $message->content.PHP_EOL;