logic issue with hierarchical context compression?

[gm-kns]

ToolOrchestra/training/lead_agent/llm_agent/generation_quick3.py

Line 1124 in f9a2a22

if code_attempt_str_len<24000:

My understanding with the following piece of code is:

• we consider adding documents to the context only if the pre-truncation context (as code + attempt) was <24k -- this makes sense
• but if this did not apply, we use the truncated to 16k sequence of (code +attempt) -- shouldn't we truncate code + attempt to 24k instead, to take advantage of more context? Or maybe eventually using 16k of code+ attempt, and adding 8k of documents at the beginning?

With current logic, it looks like long trajectories would not have all of the context they could have?

If that's a cautious design decision, I'd love to learn more about the motivations! Thanks

[hongjin-su]

Yes, it could be better to use 24k context