Skip to content

🐛 fix: extract JSON from turn_end in PiAgent json mode NDJSON stream#85

Open
SamuelLHuber wants to merge 1 commit intoevmts:mainfrom
SamuelLHuber:fix/pi-agent-json-mode-ndjson
Open

🐛 fix: extract JSON from turn_end in PiAgent json mode NDJSON stream#85
SamuelLHuber wants to merge 1 commit intoevmts:mainfrom
SamuelLHuber:fix/pi-agent-json-mode-ndjson

Conversation

@SamuelLHuber
Copy link
Contributor

@SamuelLHuber SamuelLHuber commented Feb 15, 2026

Problem

The usage of Dynamic Task Discovery with PiAgent is not working due to Pi's output not being parsed correctly into a task list.

When PiAgent uses mode: "json", pi outputs NDJSON (newline-delimited JSON) where:

  • First object: Session metadata {"type":"session",...}
  • Middle objects: Thinking tokens, progress updates
  • Last object: turn_end containing the actual response

Previously, the code tried to parse the entire stdout as a single JSON object, which failed. The first line (session metadata) would be returned instead of the actual response.

Solution

Added extractTextFromPiNdjson() which:

  1. Scans the NDJSON stream backwards
  2. Finds turn_end or agent_end events
  3. Extracts text from the assistant message content
  4. Falls back to generic extraction for simple cases

Changes

  • src/agents/cli.ts: Add extractTextFromPiNdjson() function, update PiAgent.generate()
  • tests/pi-support.test.ts: Add 3 tests for NDJSON extraction

Testing

  • ✅ Unit tests pass (8/8 in pi-support.test.ts)
  • ✅ Verified with real pi --mode json
  • ✅ TypeScript compiles cleanly

Example

Before:

text: {"type":"session","version":3,"id":"...",...}
output: { type: "session", ... }  // Wrong!

After:

text: {"v":1,"tickets":[...],"batchComplete":true}
output: { v: 1, tickets: [...], batchComplete: true }  // Correct!

When PiAgent uses mode: 'json', pi outputs NDJSON with session metadata as
the first object. The actual response is in the turn_end event's message.content.

Previously, tryParseJson() would fail on the entire stdout since it's not a
single JSON object. The first line (session metadata) would be parsed,
resulting in wrong output.

This adds extractTextFromPiNdjson() which scans the NDJSON stream backwards
to find turn_end or agent_end events and extracts the assistant message content.
@SamuelLHuber
Copy link
Contributor Author

CI Status Note

The CI failures are pre-existing issues on the main branch, not related to this PR:

Failure Cause
MergeQueue tests MDX preload configuration issue
HTTP Server tests Server connection issues in test environment

These same 12 tests fail on main branch (verified locally). This PR adds 3 new passing tests for the PiAgent json mode fix.

Verification

  • ✅ TypeScript compiles cleanly
  • ✅ 8/8 pi-support tests pass locally
  • ✅ Tested with real pi --mode json
  • ✅ No merge conflicts with main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant