Proposal: agent as composed of immutable instances

[enyst]

Problem:

We see quite a number of bug reports on restoring conversations with different settings. Many of those I’ve seen, I think, are LLM settings like model name, set encrypted reasoning, etc, but also MCP add/remove, Skills modification between sessions, and others.

The issues come from agent immutability, which seemed to imply LLM immutability, agent context immutability (like skills), condenser llm etc. The interesting part is, this “freezing” of the agent was one of the design principles of V1.

It’s useful for reproducibility, but it’s maybe not so good for user experience. As of now, we seem to patch things one by one in places where they’re reported, e.g. in metadata, in skills, in security, in condenser, in reasoning attributes.

Proposed solution:

Conceptualize the Agent as:

• composable, that is composed of
• immutable parts (llm, condenser, initial context, tools, MCP)

This is the minimal change I can think of, to keep a lot of the guarantees of immutability, and also unblock development of flexible user interfaces. And make these bug reports a thing of the past.

I think in practice, we already do it - sort of. We relaxed in multiple places, via multiple bug fixes for settings or entire components, which were due to the absolute freezing. agent_context is completely replaced with runtime value.

Code Design Detail

This proposal implies that LLM (and AgentContext, etc), can be immutable instances, but we have a principled way to

• copy+update => to a new LLM instance
• in a single place in the code
• all the rest of the codebase works on an immutable LLM instance.

What place?

• for LLM, I think LLMRegistry
• another option is conversation_state. If we like this one, maybe keep the Agent state components in there? Then it would be easy to enforce the rule: have a single, principled way to copy/update/instantiate/switch. The tradeoffs seem to be 1) that LLM doesn’t quite fit, it should be a LLMRegistry responsibility, and, 2) managing all in state puts a lot of responsibility on ConversationState (I tried it, and it feels a bit god-like class)

How will it work?

• keep LLM immutable in Pydantic implementation
• however, a new LLM instance can be created, with user-triggered or automatic or restoration updates, freely, as freely as we can, and switch the LLM instance fully
• in one place in the code, so that everything else keeps the guarantees.

Definitions

• flexibility: the user/client developer can change any/many settings and just do stuff
• composition/composability: the simple idea that an entity (e.g. agent) is composed of parts that can, themselves, be changed or switched, in place or at restore or when “teleporting”.

Note: keeping the components frozen is maybe not as flexible as it could be; but the tradeoffs here are the guarantees from pydantic and from single point where we switch instances (not attributes), which I’d suggest is less error-prone and may be worth it.

References:

1. We actually touched this from the PR we did it as an experiment in how to make the agent stateless - and how much statelessness is possible, without affecting usability… 😅
   But it does, and badly IMO.
2. Xingyao actually foresaw us ending this practice here.
3. Example bug: BUG: cannot resume conversation with different settings for MCP OpenHands-CLI#240
4. Example bug: BUG: cannot resume conversation with different settings for enable_encrypted_reasoning OpenHands-CLI#238
5. Example bug fix: Fix nested LLM reconciliation in agent deserialization #517
6. Example bug fix: Fix agent reconciliation to allow agent_context updates #1371

[jpshackelford]

@xingyaoww @malhotra5 Would love your feedback on this as the usability impact of these bugs on resuming conversations in the CLI is severe and likely to be frequently encountered.

Happily the openhands agent can go to the home directory and reread its own trajectories to get some context in new conversations but new users might not know to prompt for that.

[xingyaoww]

@OpenHands please review this proposal and list the pros and cons for me

[openhands-ai]

I'm on it! xingyaoww can track my progress at all-hands.dev

[openhands-ai]

Review of Proposal: "Agent as Composed of Immutable Instances" (Issue #1451)

After reviewing the codebase and the proposal, here is my analysis of the pros and cons:

---

PROS

1. Unified Approach - Instead of patching individual components (LLM settings, MCP, skills, condenser, security) one by one as bugs are reported, this provides a single principled way to handle component updates.

2. Better User Experience - Users can change settings between sessions (model name, reasoning settings, MCP servers, skills) without breaking conversation resume.

3. Maintains Immutability Guarantees - Components remain frozen/immutable during execution (preserving reproducibility), but can be replaced wholesale at well-defined points.

4. Reduces Bug Surface - A single point of component switching means fewer places for bugs to occur. The current approach has led to multiple bug fixes (PR Fix nested LLM reconciliation in agent deserialization #517 for nested LLM reconciliation, PR Fix agent reconciliation to allow agent_context updates #1371 for agent_context reconciliation).

5. Enables Flexible UIs - Client developers can change settings freely without worrying about breaking conversation state.

6. Already Partially Implemented - The codebase already has foundations that align with this pattern:

   • resolve_diff_from_deserialized() methods in AgentBase and LLM
   • LLMRegistry class for managing LLM instances by usage ID
   • ConversationState.create() handles reconciliation during resume

7. Clear Ownership - LLMRegistry for LLM instances is a natural fit; ConversationState could manage other components.

---

CONS

1. Implementation Complexity - Need to define clear boundaries for what can be switched and when. The proposal mentions tradeoffs between LLMRegistry vs ConversationState ownership.

2. Potential for Subtle Bugs - Switching components mid-conversation could lead to inconsistent state if not done carefully (e.g., condenser depends on LLM).

3. Testing Complexity - Need to test all combinations of component switches across resume scenarios.

4. Migration Effort - Existing code that relies on absolute immutability may need updates. The frozen=True on AgentBase would need careful consideration.

5. Documentation Burden - Need to clearly document when and how components can be switched, and what the lifecycle guarantees are.

6. Risk of Over-Engineering - The current patch-by-patch approach, while not elegant, is working. The proposal acknowledges this: "we already do it - sort of."

7. Backward Compatibility - Existing integrations may rely on current behavior where certain fields cause resume failures.

---

Key Design Questions to Resolve

1. Where should switching happen? - LLMRegistry for LLM (proposed), but what about AgentContext, tools, MCP, condenser?

2. When can components be switched? - Only at conversation resume? Between agent steps? At specific checkpoints?

3. What components should be switchable? - LLM, Condenser (and its LLM), AgentContext (skills, secrets), Tools/MCP, Security analyzer?

4. How to handle component dependencies? - Condenser depends on LLM; Agent depends on all components.

---

Recommendation

The proposal is sound and well-motivated. The current codebase already has the foundations (resolve_diff_from_deserialized, LLMRegistry) that support this pattern. The main work would be:

1. Formalizing the component switching interface
2. Consolidating the reconciliation logic into a single place (rather than scattered across AgentBase.resolve_diff_from_deserialized)
3. Documenting the component lifecycle clearly

The proposal correctly identifies that the current approach of "patching things one by one" is unsustainable and leads to recurring bugs. A principled approach would provide long-term benefits for maintainability and user experience.

_{View full conversation}

[ixchio]

@xingyaoww @enyst @jpshackelford

Took a look at how the SDK handles this and honestly the pattern is already there - ConversationState.create() does the reconciliation, agents/llms are frozen, theres a
resolve_diff_from_deserialized
method that copies over secrets and stuff.

One thing I noticed tho - tools have to match exactly or the whole resume fails. Makes sense for consistency but feels kinda strict? Like if I just want to add a new tool to an existing convo, why should that break things. Maybe only fail if a tool used in history is missing, not if a new one is added.

The condenser-llm handling is a bit gnarly with all the nested model_copy calls but works.

Anyway +1 on making this a proper pattern instead of ad-hoc fixes everywhere.

[xingyaoww]

One thing I noticed tho - tools have to match exactly or the whole resume fails. Makes sense for consistency but feels kinda strict? Like if I just want to add a new tool to an existing convo, why should that break things. Maybe only fail if a tool used in history is missing, not if a new one is added.

I think this make sense, @OpenHands please create a new github issue tracking this

[openhands-ai]

I'm on it! xingyaoww can track my progress at all-hands.dev

[openhands-ai]

I've created a new GitHub issue to track the feature request for relaxing tool matching during conversation resume.

Issue Created: #1533 - Allow adding new tools when resuming conversations without failing

Summary:
The issue captures the suggestion from @ixchio's comment on #1451:

• Problem: Currently, tools must match exactly when resuming a conversation, or the entire resume fails
• Proposed Solution: Only fail if a tool that was actually used in history is missing, but allow adding new tools that weren't present in the original conversation
• Context: Links back to the parent discussion in Proposal: agent as composed of immutable instances #1451 (Proposal: agent as composed of immutable instances)

No code changes were needed for this task - it was simply creating a tracking issue for a future enhancement.

_{View full conversation}

[xingyaoww]

@OpenHands please review this proposal AND OpenHands/OpenHands-CLI's usage of SDK and all the .model_copy() operations. Then list the pros and cons of this proposal for me. Also if you can make changes to the proposal, what do you think would be the best way to do it?

[openhands-ai]

I'm on it! xingyaoww can track my progress at all-hands.dev