feat: add CLAUDE_CODE_COMPACT_MODEL env var for compaction side-calls

[40verse]

Summary

• Added getCompactionModel() in compact.ts that defaults to getSmallFastModel() — the cheapest provider-aware model (Haiku for Anthropic, gpt-4o-mini for OpenAI, flash-lite for Gemini)
• CLAUDE_CODE_COMPACT_MODEL env var available as explicit override
• Applied to all compact call sites (full, partial, streaming summary) and session memory extraction
• Registered as SAFE_ENV_VAR in managedEnvConstants.ts

Impact

• user-facing impact: compaction defaults to the cheapest available model instead of Opus, reducing per-compact cost with no user configuration needed
• developer/maintainer impact: uses existing getSmallFastModel() infrastructure — automatically adapts when providers or models change

Testing

• bun run build
• bun run smoke
• focused tests: model resolution is a simple function chain, verified via build

Notes

• provider/model path tested: getSmallFastModel() is provider-aware (Anthropic→Haiku, OpenAI→gpt-4o-mini, Gemini→flash-lite)
• screenshots attached (if UI changed): n/a
• follow-up work or known limitations: compaction quality with smaller models should be monitored; env var override provides escape hatch

[gnanam1990]

Good contribution — using getSmallFastModel() for compaction is a sensible cost reduction and the implementation is clean. A couple of things to fix before merge:

1. Duplicated logic in sessionMemoryCompact.ts

compact.ts defines getCompactionModel() but doesn't export it, so sessionMemoryCompact.ts inlines the same logic:

// sessionMemoryCompact.ts
model: process.env.CLAUDE_CODE_COMPACT_MODEL || getSmallFastModel()

If the priority logic ever changes, it now has to be updated in two places. Please export getCompactionModel() from compact.ts and import it in sessionMemoryCompact.ts so there's a single source of truth.

2. _fallbackModel parameter is unused

getCompactionModel(_fallbackModel?: string) accepts a fallback but never uses it — the _ prefix signals it's intentionally ignored. Either wire it in or remove the parameter to avoid confusion.

3. bun run smoke unchecked

Please run and check this before merge.

Otherwise the approach is solid — reusing getSmallFastModel() infrastructure means it automatically adapts as providers change, which is exactly right.

[40verse]

Good catch! I've updated the PR based on the feedback and ran smoke test

[Vasanthdev2004]

This does not actually pick the cheap compaction model for OpenAI or Gemini users who already have a model configured. getSmallFastModel() returns process.env.OPENAI_MODEL for the OpenAI provider and process.env.GEMINI_MODEL for the Gemini provider before falling back to gpt-4o-mini / gemini-2.0-flash-lite. Direct repro on this head: with CLAUDE_CODE_USE_OPENAI=1 and OPENAI_MODEL=gpt-4.1, getCompactionModel() returns gpt-4.1; with CLAUDE_CODE_USE_GEMINI=1 and GEMINI_MODEL=gemini-2.5-pro-preview-03-25, it returns that same expensive model. So the branch does not deliver the stated cost reduction for OpenAI/Gemini setups; it only changes Anthropic (or envs with no model configured).

[Vasanthdev2004]

Rechecked the latest head 235db0b4b19c64b1710addd664713dd9f7f4a175 against current origin/main.

I still can't approve this because the main feature does not actually work as described for OpenAI and Gemini setups.

Current blocker:

1. getCompactionModel() does not default to a cheaper compaction model for OpenAI or Gemini users who already have a model configured.
   The new helper delegates to getSmallFastModel(), but on the current head that function returns:

   • process.env.OPENAI_MODEL for the OpenAI provider
   • process.env.GEMINI_MODEL for the Gemini provider

   before it ever falls back to gpt-4o-mini / gemini-2.0-flash-lite.

   Direct repro on this head:

   • with CLAUDE_CODE_USE_OPENAI=1 and OPENAI_MODEL=gpt-4.1, getCompactionModel() returns gpt-4.1
   • with CLAUDE_CODE_USE_GEMINI=1 and GEMINI_MODEL=gemini-2.5-pro-preview-03-25, getCompactionModel() returns gemini-2.5-pro-preview-03-25
   • Anthropic does switch to Haiku as intended

   So the branch does not deliver the stated cost reduction for OpenAI/Gemini configurations; it only changes Anthropic (or setups with no provider model configured at all).

Fresh verification on this head:

• direct repros of getCompactionModel() above on OpenAI, Gemini, and Anthropic provider states
• isProviderManagedEnvVar('CLAUDE_CODE_COMPACT_MODEL') -> false (not using this as a blocker, but worth noting for future host-managed consistency)
• bun run build -> success
• bun run smoke -> success

I didn't find a compile/runtime blocker beyond the model-selection issue, but I wouldn't merge this until the default compaction model selection actually becomes cheaper for OpenAI/Gemini instead of reusing the main configured model.

[gnanam1990]

I like the goal here, but I don't think the PR currently delivers the advertised behavior.

From the diff, the actual compaction request path still appears to use the existing main-loop model in the important execution path. The change looks more complete in selection helpers and metadata than in the real compaction call itself.

There is also still a risk that the fallback model resolves to the normal provider env model rather than a genuinely cheaper compaction model.

Please wire the selected compaction model all the way through the execution path and add a focused test proving the compaction side-call actually uses it.

[40verse]

I will revisit and resubmit this evening! I need to improve my testing methods to support more model, welcome any tips

[40verse]

Im still working on this. I hit some conflicts, I accepted them so I could re run tests and replan the approach. The hardcoded model names is a mess. Also trying to figure out why the regression doesnt show up for me in my test until I push. Ill pick it back up tomorrow.

[40verse]

Ive run into problems in testing when the context window of large model opus exceeds capacity of lesser model haiku

[kevincodex1]

please fix conflicts

[Vasanthdev2004]

Review: PR #367 — CLAUDE_CODE_COMPACT_MODEL env var for compaction side-calls (head a7a70eb)

CI green ✅. 9 files, +628/-39. 33 new test cases.

Both previous blockers are now properly fixed:

• ✅ getCompactionModel() is exported and reused everywhere — no more duplicated logic between compact.ts and sessionMemoryCompact.ts. All compaction call sites (compactConversation, partialCompactConversation, streamCompactSummary, trySessionMemoryCompaction) and session-start hooks use getCompactionModel().
• ✅ getSmallFastModel() no longer leaks OPENAI_MODEL / GEMINI_MODEL — the old provider branches that returned process.env.OPENAI_MODEL for OpenAI and process.env.GEMINI_MODEL for Gemini are completely gone. Now falls through to getModelStrings().haiku45, which resolves to the correct cheap tier per provider (Haiku / gpt-4o-mini / flash-lite).
• ✅ New provider-agnostic env vars — CLAUDE_CODE_SMALL_FAST_MODEL and CLAUDE_CODE_DEFAULT_SMALL_MODEL take priority over legacy ANTHROPIC_SMALL_FAST_MODEL / ANTHROPIC_DEFAULT_HAIKU_MODEL.
• ✅ settings.modelTiers.small — persistent settings-based override that survives shell restarts. Excellent addition for Ollama / LM Studio users.
• ✅ Ollama detection — getDefaultHaikuModel() detects isOllamaProvider() and uses cached /api/tags models, falling back to OPENAI_MODEL (callable locally). Never sends hardcoded API model names.
• ✅ All env vars registered in SAFE_ENV_VARS and PROVIDER_MANAGED_ENV_VARS as appropriate.
• ✅ Analytics — compactionModel and tokenCompressionRatio fields added to compaction telemetry. Useful for measuring whether small models produce worse summaries.
• ✅ 33 tests covering priority chains, provider defaults, settings integration, Ollama paths, and main-loop-model leak prevention.

Great rework, @40verse — the refactored getSmallFastModel() and getDefaultHaikuModel() are much cleaner than the old provider-branch-based approach.

---

🟡 Non-blocking suggestions

1. getCompactionModel() called multiple times per compaction

getCompactionModel() is called ~4 times per compaction (hooks, API call, analytics). Each call resolves the env var and calls getSmallFastModel(), which calls getInitialSettings(). Not a real performance concern (compaction is rare), but consider caching the resolved value once at the top of each compaction function if you want to be precise.

2. No dedicated getCompactionModel() test

The 33 tests cover getSmallFastModel() thoroughly, which effectively tests the compaction path. But CLAUDE_CODE_COMPACT_MODEL env var override is only tested implicitly (through getSmallFastModel). A single test in compact.ts verifying that CLAUDE_CODE_COMPACT_MODEL takes priority over getSmallFastModel() would close the gap.

3. maxOutputTokensOverride still uses mainLoopModel

In streamCompactSummary, line 1346: getMaxOutputTokensForModel(context.options.mainLoopModel). This caps the compaction output at the main model's limit, which is safe (summary must fit in main model's context). But if the compaction model has a smaller output limit (e.g., gpt-4o-mini at 16K vs Opus at 32K), Math.min already handles it correctly. No action needed — just noting the subtlety for awareness.

4. modelTiers schema only has small tier

Currently modelTiers only defines small. If there's a future medium / large tier, the schema is extensible. Consider documenting the roadmap or just leaving it open-ended (current approach is fine).

---

✅ All blockers resolved

• ✅ No duplicated logic — getCompactionModel() exported and reused
• ✅ OpenAI/Gemini no longer leak expensive main-loop model
• ✅ Provider-agnostic env vars with legacy fallback
• ✅ Settings-based override for persistence
• ✅ Ollama auto-detection
• ✅ CI green, 33 tests
• ✅ Clean priority chain: env var → legacy env var → settings → provider default

Verdict: Approve-ready ✅

Well-structured refactoring that addresses all previous review concerns. The non-blocking items are follow-up polish, not merge blockers.

[kevincodex1]

sorry another conflicts @40verse kindly fix for one last time

[auriti]

Review: CLAUDE_CODE_COMPACT_MODEL + small model tier refactoring

This PR is significantly larger than the title suggests — it's not just a compaction model env var, it's a full refactoring of the small/fast model selection system across the codebase. That's actually a good thing, but the scope should be reflected in the title and description.

What this PR actually does (9 files, +627/-39):

1. getCompactionModel() in compact.ts — new function using CLAUDE_CODE_COMPACT_MODEL → getSmallFastModel() fallback
2. getSmallFastModel() refactored — new priority chain: CLAUDE_CODE_SMALL_FAST_MODEL → ANTHROPIC_SMALL_FAST_MODEL → settings.modelTiers.small → getModelStrings().haiku45
3. getDefaultHaikuModel() refactored — new priority chain: CLAUDE_CODE_DEFAULT_SMALL_MODEL → ANTHROPIC_DEFAULT_HAIKU_MODEL → settings.modelTiers.small → Ollama detection → getModelStrings().haiku45
4. settings.modelTiers.small — new settings schema field for persistent small model override
5. Provider-agnostic env vars — CLAUDE_CODE_SMALL_FAST_MODEL, CLAUDE_CODE_DEFAULT_SMALL_MODEL registered as safe env vars
6. 452 lines of tests — comprehensive coverage of priority chains, provider defaults, Ollama detection, leak prevention
7. Analytics — compactionModel and tokenCompressionRatio added to compaction telemetry

Positive findings:

• Excellent test coverage — 452 lines covering env var priority, provider defaults, settings override, Ollama path, and leak prevention. This is the best-tested PR I've reviewed on this repo.
• Critical fix: main-loop model no longer leaks into small tier. The old code had return process.env.OPENAI_MODEL || 'gpt-4o-mini' in getSmallFastModel() — if OPENAI_MODEL=gpt-4.1, every compaction used the expensive model. The new code correctly routes through getModelStrings().haiku45.
• settings.modelTiers.small is the right UX for Ollama/LM Studio users — set once in settings.json, no env vars needed, survives shell restarts.
• Backward compatible — legacy ANTHROPIC_SMALL_FAST_MODEL and ANTHROPIC_DEFAULT_HAIKU_MODEL still work.
• Analytics additions — compactionModel and tokenCompressionRatio in telemetry will help detect if smaller models produce worse summaries.

Issues:

1. (Major) getCompactionModel() doesn't actually use CLAUDE_CODE_COMPACT_MODEL for the API call.

The function is defined and used to set the model field in processSessionStartHooks('compact', { model }) and streamCompactSummary({ model }). But looking at streamCompactSummary, the model parameter is passed to context.options.mainLoopModel — I need to verify that the compaction API call actually uses this model parameter and not the main loop model from elsewhere in the context. If the model is overridden in the context but the API client still reads from context.options.mainLoopModel, the env var has no effect.

2. (Minor) getCompactionModel() falls back to getSmallFastModel(), not the main model.

This is actually a behavior change: previously compaction used the main loop model (Opus/Sonnet). Now it defaults to the small/fast model (Haiku/gpt-4o-mini). This could reduce compaction quality. The PR description acknowledges this ("compaction quality with smaller models should be monitored") but it should be more prominent — this is a default behavior change for all users, not just those who set the env var.

3. (Minor) Ollama detection in getDefaultHaikuModel() uses getCachedOllamaModelOptions()[0].

This returns the first model from the Ollama cache, which may not be the smallest/cheapest. If a user has llama3.3:70b and llama3.2:3b installed, the first one returned depends on Ollama's sort order (typically alphabetical or by last-used). Users should set modelTiers.small explicitly for predictable behavior — which the code comments correctly recommend.

Verdict: APPROVE with notes

The refactoring is well-designed and the test coverage is excellent. The main concern is the silent default behavior change (compaction now uses small model instead of main model), which should be documented in release notes.

The CLAUDE_CODE_COMPACT_MODEL env var itself is a clean, minimal addition. The broader refactoring of the small model selection system is a welcome improvement that fixes a real cost leak (main model used for side-calls).

[kevincodex1]

hello @40verse please fix conflicts this is ready to merge now