Skip to content

fix: prevent false usage limit detection and upload failure logs on auto-restart#1291

Merged
konard merged 5 commits intomainfrom
issue-1290-8dde138463b7
Feb 14, 2026
Merged

fix: prevent false usage limit detection and upload failure logs on auto-restart#1291
konard merged 5 commits intomainfrom
issue-1290-8dde138463b7

Conversation

@konard
Copy link
Contributor

@konard konard commented Feb 14, 2026

Summary

Fixes #1290 - Two root causes found and fixed for the missing auto-restart session report when using --tool agent.

Problem

When an auto-restart iteration fails with --tool agent:

  1. No log was uploaded to the PR - the last visible comment was just the "Auto-restart 1/3" notification
  2. The error was falsely categorized as a "Usage Limit" error when it was actually an AI_JSONParseError from a corrupted kimi-k2.5 API stream

Root Cause Analysis

Problem 1 (false error categorization): Two bugs combined:

  1. The 'resets' pattern in isUsageLimitError() was too broad - a simple substring match that caught ordinary English words in agent code output (e.g., C# game code comments like "loads a shell and resets")
  2. The agent fallback pattern matching ran even after the agent had recovered from errors and completed successfully (exitCode=0, session.idle seen), finding stale "type":"error" events in the output

Problem 2 (missing log upload): Per-iteration log uploads only happened on toolResult.success, and the final log upload was skipped due to logsAlreadyUploaded being set by verifyResults() before auto-restart.

Evidence from the original log

[08:34:12.293Z] ⚠️  Error event detected via fallback pattern match: Tool execution aborted
[08:34:12.301Z] ⏳ Usage Limit Reached!    ← FALSE POSITIVE
[08:34:12.301Z] Your Agent usage limit has been reached.

The "resets" word matched in C# code output: "loads a shell and resets _dragStartPosition".

Solution

Changes in src/usage-limit.lib.mjs:

  • Changed 'resets' pattern from substring match to regex: /resets\s+(?:(?:at\s+)?[0-9]|(?:Jan|Feb|...))/i
  • Matches "resets 5am", "resets Jan 15, 8am" but NOT "loads a shell and resets"
  • Supports both string and RegExp patterns in the patterns array

Changes in src/agent.lib.mjs:

  • Skip fallback pattern matching when exitCode === 0 && agentCompletedSuccessfully
  • Prevents stale error events from triggering false error detection after agent recovery

Changes in src/solve.watch.lib.mjs (from previous commit):

  • Upload failure logs for failed auto-restart iterations when --attach-logs is enabled
  • Track autoRestartIterationsRan and lastIterationLogUploaded flags

Changes in src/solve.mjs (from previous commit):

  • Update final log upload condition to check autoRestartRanButNotUploaded

Test plan

  • Lint passes (npm run lint)
  • All existing tests pass (npm test) - 67 usage limit tests, 23 agent error detection tests
  • New tests for false positive "resets" pattern (6 false positive + 6 true positive cases)
  • New tests for fallback pattern matching skip logic (3 tests)
  • Manual test: Run with --tool agent --attach-logs and trigger auto-restart scenario

Case Study

Full analysis documented in docs/case-studies/issue-1290/:

  • README.md: Root cause analysis with detailed chain of events
  • log-analysis.md: Complete timeline and error analysis of the original log file

Generated with Claude Code

Adding CLAUDE.md with task information for AI processing.
This file will be removed when the task is complete.

Issue: #1290
@konard konard self-assigned this Feb 14, 2026
Fixes #1290 - When using --tool agent with --attach-logs, auto-restart
sessions now report their completion with logs when they fail (e.g., due
to usage limit or other errors).

Changes:
- solve.watch.lib.mjs: Add log upload for failed auto-restart iterations
  - Upload failure logs with "⚠️ Auto-restart X/Y Failure Log" title
  - Include error info and usage limit details
  - Track whether iterations ran and logs were uploaded
- solve.mjs: Update final log upload condition
  - Upload logs if auto-restart ran but last iteration wasn't uploaded
  - Ensures final logs are always attached when --attach-logs is enabled

This ensures consistent behavior between --tool claude and --tool agent
for auto-restart session reporting.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@konard konard changed the title [WIP] There is no report on auto-restart session finish for --tool agent fix: upload failure logs when auto-restart iteration fails Feb 14, 2026
@konard konard marked this pull request as ready for review February 14, 2026 10:21
@konard
Copy link
Contributor Author

konard commented Feb 14, 2026

🤖 Solution Draft Log

This log file contains the complete execution trace of the AI solution draft process.

💰 Cost estimation:


Now working session is ended, feel free to review and add any feedback on the solution draft.

@konard
Copy link
Contributor Author

konard commented Feb 14, 2026

Also I noticed, that we have second problem:

[2026-02-14T08:34:12.210Z] [INFO] {
[2026-02-14T08:34:12.211Z] [INFO]   "type": "log",
[2026-02-14T08:34:12.211Z] [INFO]   "level": "error",
[2026-02-14T08:34:12.211Z] [INFO]   "timestamp": "2026-02-14T08:34:12.210Z",
[2026-02-14T08:34:12.212Z] [INFO]   "service": "session.prompt",
[2026-02-14T08:34:12.212Z] [INFO]   "error": {
[2026-02-14T08:34:12.212Z] [INFO]     "error": {
[2026-02-14T08:34:12.212Z] [INFO]       "name": "AI_JSONParseError",
[2026-02-14T08:34:12.212Z] [INFO]       "cause": {},
[2026-02-14T08:34:12.212Z] [INFO]       "text": "{\"id\":\"chatcmpl-jQugNdata:{\"id\":\"chatcmpl-iU6vkr3fItZ0Y4rTCmIyAnXO\",\"object\":\"chat.completion.chunk\",\"created\":1771058051,\"model\":\"kimi-k2.5\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"finish_reason\":null}],\"system_fingerprint\":\"fpv0_f7e5c49a\"}"
[2026-02-14T08:34:12.213Z] [INFO]     }
[2026-02-14T08:34:12.213Z] [INFO]   },
[2026-02-14T08:34:12.213Z] [INFO]   "message": "stream error"
[2026-02-14T08:34:12.213Z] [INFO] }
[2026-02-14T08:34:12.213Z] [INFO] {
[2026-02-14T08:34:12.213Z] [INFO]   "type": "log",
[2026-02-14T08:34:12.214Z] [INFO]   "level": "error",
[2026-02-14T08:34:12.214Z] [INFO]   "timestamp": "2026-02-14T08:34:12.211Z",
[2026-02-14T08:34:12.214Z] [INFO]   "service": "session.processor",
[2026-02-14T08:34:12.214Z] [INFO]   "error": {
[2026-02-14T08:34:12.214Z] [INFO]     "name": "AI_JSONParseError",
[2026-02-14T08:34:12.214Z] [INFO]     "cause": {},
[2026-02-14T08:34:12.215Z] [INFO]     "text": "{\"id\":\"chatcmpl-jQugNdata:{\"id\":\"chatcmpl-iU6vkr3fItZ0Y4rTCmIyAnXO\",\"object\":\"chat.completion.chunk\",\"created\":1771058051,\"model\":\"kimi-k2.5\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"finish_reason\":null}],\"system_fingerprint\":\"fpv0_f7e5c49a\"}"
[2026-02-14T08:34:12.215Z] [INFO]   },
[2026-02-14T08:34:12.215Z] [INFO]   "message": "process"
[2026-02-14T08:34:12.215Z] [INFO] }
[2026-02-14T08:34:12.215Z] [INFO] {
[2026-02-14T08:34:12.215Z] [INFO]   "type": "session.error",
[2026-02-14T08:34:12.215Z] [INFO]   "level": "info",
[2026-02-14T08:34:12.216Z] [INFO]   "timestamp": "2026-02-14T08:34:12.211Z",
[2026-02-14T08:34:12.216Z] [INFO]   "service": "bus",
[2026-02-14T08:34:12.216Z] [INFO]   "message": "publishing"
[2026-02-14T08:34:12.216Z] [INFO] }
[2026-02-14T08:34:12.216Z] [INFO] {
[2026-02-14T08:34:12.217Z] [INFO]   "type": "error",
[2026-02-14T08:34:12.217Z] [INFO]   "timestamp": 1771058052211,
[2026-02-14T08:34:12.217Z] [INFO]   "sessionID": "ses_3a4b73b0effeFXKMNNCv1Lm3b2",
[2026-02-14T08:34:12.217Z] [INFO]   "error": {
[2026-02-14T08:34:12.217Z] [INFO]     "name": "UnknownError",
[2026-02-14T08:34:12.217Z] [INFO]     "data": {
[2026-02-14T08:34:12.217Z] [INFO]       "message": "AI_JSONParseError: JSON parsing failed: Text: {\"id\":\"chatcmpl-jQugNdata:{\"id\":\"chatcmpl-iU6vkr3fItZ0Y4rTCmIyAnXO\",\"object\":\"chat.completion.chunk\",\"created\":1771058051,\"model\":\"kimi-k2.5\",\"choices\":[{\"index\":0,\"delta\":{\"role\":\"assistant\",\"content\":\"\"},\"finish_reason\":null}],\"system_fingerprint\":\"fpv0_f7e5c49a\"}.\nError message: JSON Parse error: Expected '}'"
[2026-02-14T08:34:12.217Z] [INFO]     }
[2026-02-14T08:34:12.218Z] [INFO]   }
[2026-02-14T08:34:12.218Z] [INFO] }
[2026-02-14T08:34:12.218Z] [INFO] {
[2026-02-14T08:34:12.218Z] [INFO]   "type": "message.part.updated",
[2026-02-14T08:34:12.218Z] [INFO]   "level": "info",
[2026-02-14T08:34:12.218Z] [INFO]   "timestamp": "2026-02-14T08:34:12.214Z",
[2026-02-14T08:34:12.218Z] [INFO]   "service": "bus",
[2026-02-14T08:34:12.218Z] [INFO]   "message": "publishing"
[2026-02-14T08:34:12.219Z] [INFO] }
[2026-02-14T08:34:12.219Z] [INFO] {
[2026-02-14T08:34:12.219Z] [INFO]   "type": "tool_use",
[2026-02-14T08:34:12.219Z] [INFO]   "timestamp": 1771058052214,
[2026-02-14T08:34:12.219Z] [INFO]   "sessionID": "ses_3a4b73b0effeFXKMNNCv1Lm3b2",
[2026-02-14T08:34:12.219Z] [INFO]   "part": {
[2026-02-14T08:34:12.220Z] [INFO]     "id": "prt_c5b4925d30010l5F6BrMXeE7d6",
[2026-02-14T08:34:12.220Z] [INFO]     "sessionID": "ses_3a4b73b0effeFXKMNNCv1Lm3b2",
[2026-02-14T08:34:12.220Z] [INFO]     "messageID": "msg_c5b490d94001JytADaAvPm4n2k",
[2026-02-14T08:34:12.220Z] [INFO]     "type": "tool",
[2026-02-14T08:34:12.220Z] [INFO]     "callID": "tool_FxSEARcHF9cviw2c0abG9qQE",
[2026-02-14T08:34:12.220Z] [INFO]     "tool": "read",
[2026-02-14T08:34:12.220Z] [INFO]     "state": {
[2026-02-14T08:34:12.220Z] [INFO]       "status": "error",
[2026-02-14T08:34:12.220Z] [INFO]       "input": {},
[2026-02-14T08:34:12.221Z] [INFO]       "error": "Tool execution aborted",
[2026-02-14T08:34:12.221Z] [INFO]       "time": {
[2026-02-14T08:34:12.221Z] [INFO]         "start": 1771058052213,
[2026-02-14T08:34:12.221Z] [INFO]         "end": 1771058052213
[2026-02-14T08:34:12.221Z] [INFO]       }
[2026-02-14T08:34:12.221Z] [INFO]     }
[2026-02-14T08:34:12.221Z] [INFO]   }
[2026-02-14T08:34:12.221Z] [INFO] }

This error was falsely identified as rate limit error, but it is not that. It is just the regular error and fail, we need to ensure we catch the last error, and correctly recognize its category.

Ensure all changes are correct, consistent, validated, tested and fully meet all discussed requirements (check issue description and all comments in issue and in pull request). Ensure all CI/CD checks pass.

@konard konard marked this pull request as draft February 14, 2026 10:44
@konard
Copy link
Contributor Author

konard commented Feb 14, 2026

🤖 AI Work Session Started

Starting automated work session at 2026-02-14T10:44:46.827Z

The PR has been converted to draft mode while work is in progress.

This comment marks the beginning of an AI work session. Please wait for the session to finish, and provide your feedback.

…"resets"

Two root causes found via case study analysis of issue #1290:

1. The "resets" pattern in isUsageLimitError() was too broad - a simple
   substring match that matched ordinary English words in agent code output
   (e.g., "loads a shell and resets"). Changed to a regex requiring
   time-like content after "resets" (digit or month name).

2. Agent fallback pattern matching ran even after the agent had recovered
   from errors and completed successfully (exitCode=0, session.idle seen).
   Added guard to skip fallback when agent completed successfully.

These two bugs combined caused an AI_JSONParseError from kimi-k2.5 to be
falsely categorized as a "UsageLimit" error - the fallback found a stale
"type":"error" in the output, then detectUsageLimit scanned the full output
and matched "resets" in C# game code comments.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@konard konard changed the title fix: upload failure logs when auto-restart iteration fails fix: prevent false usage limit detection and upload failure logs on auto-restart Feb 14, 2026
@konard konard marked this pull request as ready for review February 14, 2026 11:03
@konard
Copy link
Contributor Author

konard commented Feb 14, 2026

🤖 Solution Draft Log

This log file contains the complete execution trace of the AI solution draft process.

💰 Cost estimation:

  • Public pricing estimate: $10.590334
  • Calculated by Anthropic: $7.973643 USD
  • Difference: $-2.616691 (-24.71%)
    📎 Log file uploaded as Gist (1979KB)
    🔗 View complete solution draft log

Now working session is ended, feel free to review and add any feedback on the solution draft.

@konard
Copy link
Contributor Author

konard commented Feb 14, 2026

🔄 Auto-restart 1/3

Detected uncommitted changes from previous run. Starting new session to review and commit them.

Uncommitted files:

?? docs/case-studies/issue-1290/original-log.txt

Auto-restart will stop after changes are committed or after 2 more iterations. Please wait until working session will end and give your feedback.

Add the complete solve session log (3MB) that demonstrates the bug where
AI_JSONParseError from kimi-k2.5 was falsely categorized as a UsageLimit
error. This log is referenced by the case study README and log-analysis
documents.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@konard
Copy link
Contributor Author

konard commented Feb 14, 2026

🔄 Auto-restart 1/3 Log

This log file contains the complete execution trace of the AI solution draft process.

💰 Cost estimation:

  • Public pricing estimate: $3.823399
  • Calculated by Anthropic: $2.110535 USD
  • Difference: $-1.712864 (-44.80%)
    📎 Log file uploaded as Gist (2902KB)
    🔗 View complete solution draft log

Now working session is ended, feel free to review and add any feedback on the solution draft.

@konard konard merged commit 4b63bf9 into main Feb 14, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

There is no report on auto-restart session finish for --tool agent

1 participant