Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ npm run dev # Development with hot-reload
- **TypeScript + ESM** - Modern Node.js with ES modules
- **MCP SDK** - Uses `@modelcontextprotocol/sdk` for MCP protocol
- **PTY** - Uses `node-pty` for pseudo-terminal management
- **File Server** - HTTP file server runs on port 7498 regardless of MCP transport mode (stdio or HTTP). Tool results include `download_url` which is always valid.
- **File Server** - HTTP file server runs on port 7498 regardless of MCP transport mode (stdio or HTTP). Tool results include format-specific download URLs (`download_png_url`, `download_svg_url`, `download_gif_url`) which are always valid.

## MCP Tools

Expand Down
17 changes: 9 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,7 @@ Some configuration can also be provided by the LLM, simply prompt for it:
| [`shell_start`](#shell_start) | Start a new PTY session |
| [`shell_send`](#shell_send) | Send input to a session |
| [`shell_read`](#shell_read) | Read the terminal buffer |
| [`shell_screenshot`](#shell_screenshot) | Capture terminal as PNG |
| [`shell_screenshot`](#shell_screenshot) | Capture terminal as PNG and SVG |
| [`shell_record_start`](#shell_record_start) | Start recording for GIF export |
| [`shell_record_stop`](#shell_record_stop) | Stop recording and save GIF |
| [`shell_stop`](#shell_stop) | Stop a PTY session |
Expand Down Expand Up @@ -286,7 +286,7 @@ drwxr-xr-x 10 user staff 320 Dec 18 09:00 ..

### **shell_screenshot**

Capture terminal as PNG. Also saves SVG, ANSI, and plain text versions. Pass `name` without extension (`.png` is added automatically). Optionally add a macOS-style window border (off by default):
Capture terminal as PNG and SVG. Also saves ANSI and plain text versions. Pass `name` without extension (`.png` is added automatically). Optionally add a macOS-style window border (off by default):

```json
{
Expand All @@ -296,13 +296,14 @@ Capture terminal as PNG. Also saves SVG, ANSI, and plain text versions. Pass `na
}
```

The response contains a `download_url` for curl to save the file locally:
The response contains `download_png_url` and `download_svg_url` for curl to save the files locally:

```json
{
"filename": "my-screenshot.png",
"download_url": "http://localhost:7498/files/mcp-.../screenshots/my-screenshot.png",
"hint": "Use curl -o <filename> <download_url> to save the file"
"download_png_url": "http://localhost:7498/files/mcp-.../screenshots/my-screenshot.png",
"download_svg_url": "http://localhost:7498/files/mcp-.../screenshots/my-screenshot.svg",
"hint": "Use curl -o <filename> <url> to save the file"
}
```

Expand Down Expand Up @@ -339,13 +340,13 @@ Stop recording and render frames to GIF:
}
```

The response contains a `download_url` for curl to save the file locally:
The response contains a `download_gif_url` for curl to save the file locally:

```json
{
"filename": "my-recording.gif",
"download_url": "http://localhost:7498/files/mcp-.../recordings/my-recording.gif",
"hint": "Use curl -o <filename> <download_url> to save the file",
"download_gif_url": "http://localhost:7498/files/mcp-.../recordings/my-recording.gif",
"hint": "Use curl -o <filename> <url> to save the file",
"frame_count": 42,
"duration_ms": 4200
}
Expand Down
6 changes: 3 additions & 3 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,13 +35,13 @@

## File Download Flow

Screenshots and recordings return a `download_url` instead of base64 data:
Screenshots and recordings return format-specific download URLs instead of base64 data:

```
1. LLM calls shell_screenshot() or shell_record_stop()
2. Server saves file to temp directory
3. Server returns: { filename, download_url, ... }
4. LLM uses curl to download: curl -o file.png <download_url>
3. Server returns: { filename, download_png_url, download_svg_url, ... } or { filename, download_gif_url, ... }
4. LLM uses curl to download: curl -o file.png <download_png_url>
```

This avoids token overflow from large base64 payloads.
Expand Down
Binary file added docs/examples/vim-close-bordered.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
23 changes: 17 additions & 6 deletions evaluations/run.ts
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ ${prompt}`,
}
}
} else if (message.type === "user") {
// Capture tool results to extract download_url
// Capture tool results to extract download URLs
for (const block of message.message.content) {
if (block.type === "tool_result") {
const content = typeof block.content === "string"
Expand All @@ -102,11 +102,22 @@ ${prompt}`,
} catch {
continue; // Not JSON, skip
}
if (parsed.download_url && parsed.filename) {
const dest = path.join(scenarioPath, parsed.filename);
console.log(` Downloading: ${parsed.download_url} → ${parsed.filename}`);
await downloadArtifact(parsed.download_url, dest);
artifacts.push({ filename: parsed.filename, localPath: dest });
const downloadUrls: { url: string; filename: string }[] = [];
if (parsed.download_png_url && parsed.filename) {
downloadUrls.push({ url: parsed.download_png_url, filename: parsed.filename });
}
if (parsed.download_svg_url && parsed.filename) {
const svgFilename = parsed.filename.replace(/\.png$/i, ".svg");
downloadUrls.push({ url: parsed.download_svg_url, filename: svgFilename });
}
if (parsed.download_gif_url && parsed.filename) {
downloadUrls.push({ url: parsed.download_gif_url, filename: parsed.filename });
}
for (const { url, filename } of downloadUrls) {
const dest = path.join(scenarioPath, filename);
console.log(` Downloading: ${url} → ${filename}`);
await downloadArtifact(url, dest);
artifacts.push({ filename, localPath: dest });
console.log(` ✓ Artifact saved: ${dest}`);
}
}
Expand Down
4 changes: 2 additions & 2 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ Tips:

server.tool(
"shell_screenshot",
"Capture terminal screenshot as PNG. Optionally add a macOS-style window border with border: { style: \"macos\", title: \"...\" } (off by default). Returns a download_url - use curl to save the file locally (e.g., curl -o screenshot.png <url>)",
"Capture terminal screenshot as PNG and SVG. Optionally add a macOS-style window border with border: { style: \"macos\", title: \"...\" } (off by default). Returns download_png_url and download_svg_url - use curl to save (e.g., curl -o screenshot.png <download_png_url>)",
shellScreenshotSchema,
async (params) => shellScreenshot(params, toolContext)
);
Expand All @@ -223,7 +223,7 @@ Tips:

server.tool(
"shell_record_stop",
"Stop recording and save GIF. Returns a download_url - use curl to save the file locally (e.g., curl -o recording.gif <url>)",
"Stop recording and save GIF. Returns download_gif_url - use curl to save the file locally (e.g., curl -o recording.gif <download_gif_url>)",
shellRecordStopSchema,
async (params) => shellRecordStop(params, toolContext)
);
Expand Down
2 changes: 1 addition & 1 deletion src/lib/buffer-to-svg.ts
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ export function bufferToSvg(
}

// Render cursor if visible
if (terminal.modes.showCursor) {
if ((terminal.modes as { showCursor?: boolean }).showCursor) {
const cursorX = buffer.cursorX;
const cursorY = buffer.cursorY;

Expand Down
4 changes: 2 additions & 2 deletions src/tools/shell-record-stop.ts
Original file line number Diff line number Diff line change
Expand Up @@ -47,10 +47,10 @@ export async function shellRecordStop(
const downloadUrl = context.getDownloadUrl(context.getMcpSessionId(), session_id, "recordings", filename);
const output = {
filename,
download_url: downloadUrl,
download_gif_url: downloadUrl,
frame_count: result.frameCount,
duration_ms: durationMs,
hint: "Use curl -o <filename> <download_url> to save the file"
hint: "Use curl -o <filename> <url> to save the file"
};
context.logToolCall("shell_record_stop", { session_id, name }, output);

Expand Down
11 changes: 9 additions & 2 deletions src/tools/shell-screenshot.ts
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,15 @@ export async function shellScreenshot(

context.log(`[shellwright] Screenshot saved: ${screenshotDir}/${baseName}.{png,svg,ansi,txt}`);

const downloadUrl = context.getDownloadUrl(context.getMcpSessionId(), session_id, "screenshots", filename);
const output = { filename, download_url: downloadUrl, hint: "Use curl -o <filename> <download_url> to save the file" };
const mcpSessionId = context.getMcpSessionId();
const pngUrl = context.getDownloadUrl(mcpSessionId, session_id, "screenshots", `${baseName}.png`);
const svgUrl = context.getDownloadUrl(mcpSessionId, session_id, "screenshots", `${baseName}.svg`);
const output = {
filename,
download_png_url: pngUrl,
download_svg_url: svgUrl,
hint: "Use curl -o <filename> <url> to save the file",
};
context.logToolCall("shell_screenshot", { session_id, name }, output);

return {
Expand Down