feat(heartbeat): add triage model for cheap heartbeat pre-screening#786
feat(heartbeat): add triage model for cheap heartbeat pre-screening#786zombopanda wants to merge 8 commits intopaperclipai:masterfrom
Conversation
When an agent has `heartbeatModel` configured in its adapterConfig, timer- triggered heartbeats run a short triage phase first using the cheaper model. The triage model checks for assigned work via the Paperclip API and returns a structured JSON decision: "none" (no work, exit early), "handle" (simple work handled by triage), or "escalate" (complex work, launch full model). This significantly reduces costs for agents that heartbeat frequently but only have work intermittently. Non-timer sources (on_demand, assignment, automation) skip triage entirely and run the full model immediately. Triage failures always default to "escalate" to ensure work is never silently skipped. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Greptile SummaryThis PR introduces a triage pre-screening phase for timer-triggered heartbeats: a cheap model (e.g. Haiku) runs up to 3 turns to check for assigned work and returns a structured
Confidence Score: 2/5
Important Files Changed
Prompt To Fix All With AIThis is a comment left during a code review.
Path: server/src/services/heartbeat.ts
Line: 1533-1541
Comment:
**`workspaceStrategy` inherited from full agent config**
`triageConfig` spreads `resolvedConfig` and explicitly nulls out `bootstrapPrompt` and `instructionsFilePath`, but does **not** clear `workspaceStrategy`. If the agent has a `workspaceStrategy` (e.g. `{ type: "git_worktree", ... }`), every triage run will go through a full git worktree setup and teardown — even though the triage prompt only reads from the Paperclip API and never touches the filesystem. Worktree creation can take several seconds, directly undermining the cost-saving motivation for triage.
Consider explicitly clearing it alongside the other lightweight overrides:
```typescript
const triageConfig = {
...resolvedConfig,
model: heartbeatModel,
maxTurnsPerRun: TRIAGE_MAX_TURNS,
promptTemplate: TRIAGE_PROMPT_TEMPLATE,
bootstrapPrompt: "",
instructionsFilePath: undefined,
workspaceStrategy: undefined, // triage is API-only, no workspace needed
timeoutSec: 120,
};
```
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: server/src/services/heartbeat.ts
Line: 1579-1581
Comment:
**Inconsistent `usageJson` schema between triage-only and triage+escalate runs**
The two paths produce structurally different `usageJson` objects:
- **Triage-only early exit** (`"none"` / `"handle"`): triage tokens are promoted to the *top level* with a `triageOnly: true` marker:
```json
{ "inputTokens": 100, "outputTokens": 50, "costUsd": 0.001, "triageOnly": true }
```
- **Triage + escalate path** (lines ~1750–1754): full-model tokens are top-level and triage is *nested* under `triageUsage`:
```json
{ "inputTokens": 5000, "outputTokens": 2000, "costUsd": 0.05, "triageUsage": { ... } }
```
Any analytics or billing code that reads `usageJson.inputTokens` / `usageJson.costUsd` will silently produce wrong cost summaries depending on which path ran. A consistent structure — e.g., always storing main-model usage at the top level with a nested `triageUsage` key (and `null` main-model usage for triage-only runs) — would let consumers handle both cases uniformly.
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: ui/src/components/AgentConfigForm.tsx
Line: 836
Comment:
**Clearing triage model in overlay sends `undefined` instead of `""`**
```tsx
onChange={(v) => mark("heartbeat", "heartbeatModel", v || undefined)}
```
When the user selects "None (skip triage)", `HeartbeatTriageModelDropdown` calls `onChange("")`. Here `"" || undefined` evaluates to `undefined`, so `mark` records the field as `undefined` — which in the overlay system typically means "inherit from base config" rather than "explicitly unset". If the base agent config already has a `heartbeatModel` set, clearing it in an overlay would silently re-inherit the base value instead of disabling triage.
Compare with the direct-edit path (line 807) where `onChange={(v) => set!({ heartbeatModel: v })}` correctly stores `""`.
```suggestion
onChange={(v) => mark("heartbeat", "heartbeatModel", v === "" ? "" : v)}
```
How can I resolve this? If you propose a fix, please make it concise.Last reviewed commit: bd47f9f |
The previous commit only added heartbeatModel documentation to claude-local. This adds the same heartbeat triage section to the remaining 6 adapters: codex-local, cursor-local, gemini-local, openclaw-gateway, opencode-local, and pi-local. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- CRITICAL: pass TRIAGE_SYSTEM_PROMPT via promptTemplate (not dead context.systemPrompt that no adapter reads) - Replace fragile regex parser with brace-balanced JSON extractor that handles } inside reason strings - Fix parser to skip past non-matching JSON objects instead of re-scanning their contents - Remove dead TriageResult.usage field (actual usage flows through separate triageUsage local) - Preserve sessionIdAfter on triage early-exit path - Remove unused _paperclipTriageMode context key - Add tests: braces in reason, multiple JSON objects in stdout Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
@greptile-apps, fixed |
…config, sort model list
- Use `${runId}_triage` to prevent per-run state collisions between triage and full-model adapter calls
- Null out bootstrapPrompt and instructionsFilePath in triage config to avoid inheriting full agent startup instructions and reading CLAUDE.md on every timer heartbeat
- Sort triage model dropdown: lighter models (Haiku) first, show warning hint when expensive model selected
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
@greptile-apps, fixed |
- On triage "handle" early-exit, call updateRuntimeState and upsertTaskSession so next heartbeat sees the triage run instead of stale session data - Prefix triage log output with [triage] to distinguish from full-model output in stdoutExcerpt/stderrExcerpt Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
@greptile-apps, fixed again |
Check run status before overwriting with "succeeded" — if the run was cancelled by the control plane during triage execution, respect the cancellation instead of silently overwriting it. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
@greptile-apps, fixed again |
…e timeout - Rename constant to match actual usage (promptTemplate, not system prompt) - Add timeoutSec: 120 to triage config — triage should complete in seconds, not inherit the full-model timeout of tens of minutes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
@greptile-apps, fixed again |
- Clear workspaceStrategy in triageConfig so triage runs skip unnecessary git worktree setup/teardown - Make triage-only usageJson consistent with escalate path: always nest triage tokens under triageUsage key with null main-model fields Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
@greptile-apps, fixed again — stripped workspaceStrategy from triage config and unified usageJson schema across both paths. |
|
Nice! The schema is now consistent:
And |
Summary
heartbeatModelfield to agent adapter config, allowing a cheaper model (e.g. Haiku) to pre-screen timer-triggered heartbeats before launching the full modelheartbeatModelis set, runs a short triage phase (max 3 turns) that checks for assigned work via Paperclip API and returns a structured decision:none(no work),handle(simple work done), orescalate(launch full model)Motivation
Agents that heartbeat frequently (e.g. every 5 minutes) but only have intermittent work waste significant tokens/cost on the full model just to discover there's nothing to do. A cheap triage model can check for work in 3 turns at a fraction of the cost, only escalating to the full model when real work exists.
Changes
packages/adapter-utils/src/types.tsheartbeatModeltoCreateConfigValuesserver/src/services/heartbeat.tsTRIAGE_SYSTEM_PROMPT,parseTriageOutput(), triage phase inexecuteRun(), triage usage mergingpackages/adapters/claude-local/src/index.tsheartbeatModelinagentConfigurationDocui/src/components/AgentConfigForm.tsxHeartbeatTriageModelDropdowncomponent, shown when heartbeat is enabledui/src/components/agent-config-defaults.tsheartbeatModel: ""defaultserver/src/__tests__/heartbeat-triage.test.tsUsage
Test plan
parseTriageOutputcorrectly parses all action types from stdoutheartbeatModelis not setusageJson🤖 Generated with Claude Code