Allow to access and modify the actual inputs before the underlying LLM call

[daavoo]

Currently, users can access and modify the attributes of the span (Context.current_span) that will be exposed in the tracing.

It would be also useful to do the same with whatever object is actually being used by each framework, so callbacks could impact the workflow in different ways beyond stopping execution.

[njbrake]

Makes sense. Just to make sure the topic is clear, a simple example:

1. User asks agent "please search the web for restaurants"
2. Callback for post_tool_call summarizes the web search tool results to condense the long html output into a small text block, replaces the initial tool call output with this content to reduce the context length required for future llm calls
3. Further LLM calls take place, and the LLM never sees that long html results, it only sees the neat text output that was postprocessed by the post tool call callback.

[javiermtorres]

IMHO this kind of summarization is a very useful example 👍🏼 I'd assume that we'd keep the original trace, with the original result, as well as the working context, which is what we actually feed the LLM, right?

[daavoo]

I did an initial exploration on how to implement this in 2fba0bd .

I don't expect it to be a breaking API change, just an extension of the current Context with a new messages or something that users can manipulate.

Giving that and the fact that I feel like more input is needed on what are the expected use cases for this, I am dropping this from v1.0 milestone

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.