Bundle keyframes into event_detail; recover stalled compiler agent#59
Merged
softpudding merged 5 commits intomainfrom Apr 14, 2026
Merged
Bundle keyframes into event_detail; recover stalled compiler agent#59softpudding merged 5 commits intomainfrom
softpudding merged 5 commits intomainfrom
Conversation
Recording: capture drag-and-drop operations via HTML5 drag events and pointer-distance heuristic for non-HTML5 DnD libraries; detect slider elements in change handler and emit set_slider events with value/min/max; suppress noisy range input events and post-drag click events. Server: add drag_and_drop and set_slider step builders and normalization handlers in workflow_compiler; update trace viewer to display DnD source/target and slider value metadata. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…led agent Recording keyframes now match the runtime capture quality (1920x1080 jpeg) and include an after-action keyframe for click/change/submit/ drag_and_drop/set_slider events, giving the compiler agent both before and after page states for every action. The compiler's TraceViewerTool drops the separate keyframe command; event_detail now returns an image_urls list (before + after when present) along with the full event JSON, and the visualizer + SSE pipeline surface every URL to the frontend. The compiler prompt is bumped via the agent-sdk rev to describe the new shape. When the agent stops without calling ask_user or submit_workflow (e.g. it replies in prose to a vague user answer), _collect_result keeps the session alive and returns status="stalled" with the last agent message. The Compile Routine UI reuses the question panel to prompt the user for a follow-up so /compile/answer can resume the loop, instead of leaving the session stranded. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
TraceViewerTooldrops the separatekeyframecommand —event_detailreturns animage_urlslist (before + after) inline with the event JSON. Prompt and agent-sdk rev updated to match.ask_userorsubmit_workflow._collect_resultnow returnsstatus="stalled"with the agent's last message, and the Compile Routine UI reuses the question panel to let the user send a follow-up that resumes the loop via/compile/answer..claude/,CLAUDE.local.md).Test plan
bun testinextension/(191 pass)uv run pytest server/tests/unit/test_compiler_agent_finalize.py server/tests/unit/test_base_classes.py(24 pass)event_detailshows both before/after keyframes inlineask_userquestion vaguely, confirm the stalled-agent follow-up panel appears and/compile/answerresumes the loop🤖 Generated with Claude Code