Discussion: "compact "feature

[njbrake]

I was using Claude code as well as open code this weekend, and I noticed they have a very nice feature for context engineering called "compact ". It's this tool you can use to summarize all of your previous context. I wonder if it might be useful to add some sort of feature to do this in any agent at least open code has a auto compact feature, where the compact function will automatically run when they reach a certain threshold of context length something like this might be useful to help people who are developing their agents, I'm not sure

[inventivepotter]

I think few utility functions such as "context condensation" would absolutely help the community managing their context better but doing it every iteration would cost negatively. It is important that this has to done once a condition met such as context length is almost 70% of supported context length for the model used. It is proven that once you start reaching towards the limit of context length, llm performance and accuracy tend to go south exponentially.

I think as any agent community there is quite bit of improvements that we can make about memory management or context engineering. The most advanced would be a memory manager agent (custom agent per user needs) that helps what to keep in the context and what can be concised. This agent either can run per 'n' iterations/turns or once context reach certain length.

[inventivepotter]

Also, I think we should provide an unopinionated way on how we handle message list instead of just allowing prompt string. There is a value in keeping few interactions between ai and user while condensing them per chat session etc.

[inventivepotter]

@njbrake @daavoo I'd love to hear your thoughts about this.

[njbrake]

Thanks for your thoughts, @inventivepotter . I am thinking about this and don't have an informed opinion yet, but I am thinking about this as possibly a 'context management' class/interface in any-agent that people can easily plug and play in their applications. For our agentic loop we could allow for this as a callback for before_llm_call, utilizing the new callback edit feature that we're currently working on: #852.

I guess I'm thinking about this in terms of whether we write the code and put it into any-agent, or if we write a guide/tutorial and leave it to the user to implement it themselves.

[inventivepotter]

It makes sense to write code only if we are adding more capabilities to context engineering and memory management. If not, guiding users in best possible direction should do, especially with the capability of updating messages from callbacks is useful, if not user can plugin before calling the run method.