Skip to content

Commit d5f58a8

Browse files
committed
Add persistent Chrome via CDP — browser survives session restarts
Three-layer browser resilience: Layer 1 — Persistent Chrome via CDP: Chrome runs as a background process with remote debugging (port 9222). Playwright MCP connects via Chrome DevTools Protocol instead of launching its own browser. Chrome survives Claude Code restarts, login sessions persist, browser never dies with the session. New: bin/chrome-debug.sh (start/stop/status/restart/url) Updated: config/playwright-config.json uses cdpEndpoint Updated: install.sh auto-starts Chrome CDP during installation Layer 2 — Auto-retry on context errors: When a page/context closes, the agent tries browser_close + retry once before falling back. Recovers from tab-level crashes without user intervention. Layer 3 — Smart browser avoidance: Agent checks if browser is actually needed before using it. Skips browser for encrypted messaging apps, QR code flows, native apps, and any task with a CLI/API alternative.
1 parent 594a944 commit d5f58a8

File tree

5 files changed

+304
-50
lines changed

5 files changed

+304
-50
lines changed

README.md

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -127,8 +127,17 @@ A PreToolUse hook that runs before every Bash command. It's a shell script — d
127127
### Smart Permissions
128128
All tools auto-approved (same speed as `--dangerously-skip-permissions`), with the Guardian catching dangerous patterns. Safe commands fly through with zero prompts. Dangerous commands are hard-blocked before they execute.
129129

130-
### Browser Stability & Recovery
131-
The Playwright MCP is pre-configured with Chromium stability flags that prevent background throttling, hang detection, and IPC flooding — the most common causes of browser death during long sessions. A persistent browser profile at `~/MCPs/autopilot/browser-profile/` keeps login sessions alive across restarts. If the browser still dies (rare), the agent falls back to CLI tools automatically and only asks you to restart if browser is truly needed (e.g., first-time login to a new service).
130+
### Persistent Browser (Never Dies)
131+
The browser runs as a separate Chrome process via Chrome DevTools Protocol — independent of Claude Code. When your session ends or restarts, the browser stays alive. Login sessions persist. The Playwright MCP just reconnects to it.
132+
133+
```
134+
Chrome (persistent, background) ←── CDP ──→ Playwright MCP ←──→ Claude Code
135+
↑ ↑
136+
Always alive Dies with session
137+
Login sessions persist Reconnects on start
138+
```
139+
140+
Three-layer resilience: (1) persistent Chrome via CDP — browser never dies with the session, (2) auto-retry on context errors — recovers from tab crashes, (3) smart browser avoidance — skips the browser entirely for tasks it can't help with (encrypted apps, CLI-available operations).
132141

133142
### MCP Auto-Discovery
134143
Maintains a whitelist of trusted MCP servers. Whitelisted MCPs install silently when needed. Unknown MCPs: Autopilot explains what it found, why it's useful, and asks once. Approved MCPs are whitelisted forever.
@@ -237,8 +246,9 @@ If something breaks midway, open the log to see exactly what happened and where.
237246
```
238247
~/MCPs/autopilot/
239248
bin/
240-
keychain.sh # macOS Keychain wrapper (get/set/delete/list/has)
249+
keychain.sh # Cross-platform credential store (Keychain/libsecret/Credential Manager)
241250
guardian.sh # PreToolUse safety hook (hard-blocks dangerous commands)
251+
chrome-debug.sh # Persistent Chrome manager (start/stop/status via CDP)
242252
setup-clis.sh # CLI installer (gh, vercel, supabase, wrangler, etc.)
243253
test-guardian.sh # 55-test suite for the guardian
244254
config/
@@ -373,7 +383,7 @@ Edit `~/MCPs/autopilot/config/trusted-mcps.yaml` and add to the `whitelisted` se
373383
- **Payment method setup** — PCI-compliant forms resist automation
374384

375385
### Technical
376-
- **Browser can still crash**Stability flags reduce browser deaths significantly, but can't prevent all crashes. Autopilot falls back to CLI automatically. If you need browser automation after a crash, restart your Claude Code session. Login sessions persist in the browser profile.
386+
- **Chrome CDP needs to be running**The persistent Chrome must be started before using browser features: `~/MCPs/autopilot/bin/chrome-debug.sh start`. The installer does this automatically, but after a system reboot you may need to start it again.
377387
- **Browser UIs change** — Playwright steps in service registry can break when dashboards redesign
378388
- **New MCPs need a restart** — installed MCPs take effect next Claude Code session
379389
- **No automatic rollback** — if a deploy goes wrong, you fix it manually

agent/autopilot.md

Lines changed: 50 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -270,31 +270,63 @@ Do NOT search when:
270270

271271
## Browser Automation Protocol
272272

273-
When using Playwright MCP for service interaction:
273+
### Pre-Browser Check (Layer 3 — avoid unnecessary browser use)
274274

275-
1. **Navigate** to the service dashboard URL
276-
2. **Snapshot** the page (use `browser_snapshot`, NOT screenshots) to understand the current state
277-
3. **Check login status** — look for dashboard elements vs. login form
278-
4. If login needed:
279-
a. Retrieve email/password from keychain
275+
Before opening the browser, ask: **can this task be done without it?**
276+
277+
DO NOT use the browser for:
278+
- **Encrypted/authenticated messaging** (WhatsApp, Slack, Telegram) — data is encrypted, browser automation won't help
279+
- **Native apps or desktop software** — browser can't interact with these
280+
- **QR code login flows** — can't scan QR codes programmatically
281+
- **Tasks where a CLI/API exists** — always prefer CLI over browser
282+
283+
Only use the browser for:
284+
- **Signing up for a new service** (no CLI can do this)
285+
- **Getting API tokens from dashboards** (when no CLI auth flow exists)
286+
- **Service-specific web operations** with no API/CLI equivalent
287+
288+
### Browser Automation Steps
289+
290+
When the browser IS needed:
291+
292+
1. **Check Chrome CDP is running**: `~/MCPs/autopilot/bin/chrome-debug.sh status`. If not running, tell the user: "Run `~/MCPs/autopilot/bin/chrome-debug.sh start` to start the persistent browser."
293+
2. **Navigate** to the service dashboard URL
294+
3. **Snapshot** the page (use `browser_snapshot`, NOT screenshots) to understand the current state
295+
4. **Check login status** — look for dashboard elements vs. login form
296+
5. If login needed:
297+
a. Retrieve email/password from keychain (service-specific or primary)
280298
b. Fill the login form using `browser_fill_form`
281299
c. Click the sign-in button
282300
d. Snapshot again to check result
283-
5. **If 2FA/MFA appears**: STOP IMMEDIATELY. Tell the user exactly what's needed. Do not attempt to bypass.
284-
6. **If CAPTCHA appears**: STOP. Tell the user.
285-
7. **Take it step by step** — snapshot after every significant action to verify it succeeded
286-
8. **Wait for page loads** — use `browser_wait_for` when navigating between pages
287-
9. When done, capture any values needed (API keys, URLs, etc.) and store them in keychain
301+
6. **If 2FA/MFA appears**: STOP IMMEDIATELY. Tell the user exactly what's needed. Do not attempt to bypass.
302+
7. **If CAPTCHA appears**: STOP. Tell the user.
303+
8. **Take it step by step** — snapshot after every significant action to verify it succeeded
304+
9. **Wait for page loads** — use `browser_wait_for` when navigating between pages
305+
10. When done, capture any values needed (API keys, URLs, etc.) and store them in keychain
306+
307+
### Browser Error Recovery (Layer 2 — auto-retry)
288308

289-
### Browser Recovery Protocol
309+
If a browser operation fails with "Target page, context or browser has been closed" or similar:
290310

291-
The Playwright browser instance can die or expire during a session. When this happens:
311+
1. **Try to recover**: call `browser_close` to clean up, then retry the navigation ONCE. Sometimes only the page/context dies, not the whole browser — a close + reopen can recover it.
312+
2. **If retry fails**: DO NOT attempt to fix it further. Never run `kill`, `pkill`, `killall` on Playwright or MCP processes.
313+
3. **Check if CLI can handle the task.** Most operations that use the browser have a CLI equivalent. Check if the required credential is already in keychain (`keychain.sh has {service} {key}`). If yes, switch to CLI and continue.
314+
4. **If CLI works** → switch to CLI, complete the task, include a brief note: "Browser context error, completed via CLI instead."
315+
5. **If browser is truly required** → tell the user: "The browser context has closed. Run `~/MCPs/autopilot/bin/chrome-debug.sh restart` and try again."
316+
317+
### Persistent Chrome Architecture
318+
319+
The browser runs as a separate Chrome process with Chrome DevTools Protocol (CDP) on port 9222. Playwright MCP connects to it rather than launching its own browser.
320+
321+
```
322+
Chrome (persistent, background) ←── CDP ──→ Playwright MCP ←──→ Claude Code
323+
↑ ↑
324+
Survives restarts Dies with session
325+
Login sessions persist Reconnects on start
326+
~/MCPs/autopilot/browser-profile/
327+
```
292328

293-
1. **DO NOT attempt to fix it.** Never run `kill`, `pkill`, `killall`, or any command targeting Playwright or MCP processes. MCP servers are managed by the Claude Code harness — killing them disconnects the MCP entirely and makes things worse.
294-
2. **Check if CLI can handle the task.** Most operations that use the browser have a CLI equivalent. Check if the required credential is already in keychain (`keychain.sh has {service} {key}`). If yes, switch to CLI and continue.
295-
3. **If CLI works** → switch to CLI, complete the task, include a brief note: "Browser session expired, completed via CLI instead."
296-
4. **If browser is truly required** (first-time login to a service with no CLI, no credential in keychain) → tell the user: "The Playwright browser session has expired. Please restart Claude Code (`claude --agent autopilot`) and I'll pick up where I left off. Your credentials and progress are saved."
297-
5. **Never retry browser operations** after the browser is confirmed dead. Immediately fall back.
329+
Managed via `~/MCPs/autopilot/bin/chrome-debug.sh start|stop|status|restart`.
298330

299331
**Key insight:** Once a credential is stored in Keychain, the browser is rarely needed again. The browser's primary job is first-time credential acquisition. Prioritize getting tokens into Keychain early in any workflow so subsequent operations are browser-independent.
300332

bin/chrome-debug.sh

Lines changed: 213 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,213 @@
1+
#!/bin/bash
2+
# chrome-debug.sh — Launch and manage a persistent Chrome instance with CDP
3+
#
4+
# The Playwright MCP connects to this via Chrome DevTools Protocol instead of
5+
# launching its own browser. The browser persists independently of Claude Code —
6+
# it never dies when your session ends.
7+
#
8+
# Usage:
9+
# chrome-debug.sh start # Launch Chrome with CDP (background, survives terminal close)
10+
# chrome-debug.sh stop # Stop the Chrome instance
11+
# chrome-debug.sh status # Check if Chrome CDP is running
12+
# chrome-debug.sh restart # Stop + start
13+
# chrome-debug.sh url # Print the CDP endpoint URL
14+
15+
set -euo pipefail
16+
17+
CDP_PORT="${CDP_PORT:-9222}"
18+
PROFILE_DIR="$HOME/MCPs/autopilot/browser-profile"
19+
PID_FILE="$HOME/MCPs/autopilot/.chrome-debug.pid"
20+
21+
# ─── Detect Chrome Binary ────────────────────────────────────────────────────
22+
23+
find_chrome() {
24+
case "$(uname -s)" in
25+
Darwin)
26+
for bin in \
27+
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" \
28+
"/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary" \
29+
"/Applications/Chromium.app/Contents/MacOS/Chromium" \
30+
"/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge"; do
31+
[ -x "$bin" ] && echo "$bin" && return
32+
done
33+
;;
34+
Linux)
35+
for bin in google-chrome google-chrome-stable chromium-browser chromium; do
36+
command -v "$bin" &>/dev/null && echo "$bin" && return
37+
done
38+
# Check snap
39+
[ -x "/snap/bin/chromium" ] && echo "/snap/bin/chromium" && return
40+
;;
41+
MINGW*|MSYS*|CYGWIN*)
42+
for bin in \
43+
"/c/Program Files/Google/Chrome/Application/chrome.exe" \
44+
"/c/Program Files (x86)/Google/Chrome/Application/chrome.exe" \
45+
"$LOCALAPPDATA/Google/Chrome/Application/chrome.exe"; do
46+
[ -x "$bin" ] && echo "$bin" && return
47+
done
48+
;;
49+
esac
50+
echo ""
51+
}
52+
53+
# ─── CDP Endpoint Detection ──────────────────────────────────────────────────
54+
55+
# macOS Chrome may bind to IPv6 only — try both
56+
get_cdp_url() {
57+
# Try IPv4 first, then IPv6
58+
if curl -s --connect-timeout 2 "http://127.0.0.1:${CDP_PORT}/json/version" > /dev/null 2>&1; then
59+
echo "http://127.0.0.1:${CDP_PORT}"
60+
elif curl -s --connect-timeout 2 "http://[::1]:${CDP_PORT}/json/version" > /dev/null 2>&1; then
61+
echo "http://[::1]:${CDP_PORT}"
62+
else
63+
echo ""
64+
fi
65+
}
66+
67+
is_running() {
68+
local url
69+
url=$(get_cdp_url)
70+
[ -n "$url" ]
71+
}
72+
73+
# ─── Commands ────────────────────────────────────────────────────────────────
74+
75+
cmd_start() {
76+
if is_running; then
77+
echo "Chrome CDP already running on port ${CDP_PORT}"
78+
echo "Endpoint: $(get_cdp_url)"
79+
return 0
80+
fi
81+
82+
local chrome_bin
83+
chrome_bin=$(find_chrome)
84+
85+
if [ -z "$chrome_bin" ]; then
86+
echo "ERROR: No Chrome/Chromium installation found." >&2
87+
echo "Install Google Chrome or Chromium and try again." >&2
88+
exit 1
89+
fi
90+
91+
echo "Starting Chrome with CDP on port ${CDP_PORT}..."
92+
echo "Binary: $chrome_bin"
93+
echo "Profile: $PROFILE_DIR"
94+
95+
mkdir -p "$PROFILE_DIR"
96+
97+
# Launch Chrome in background with CDP enabled
98+
nohup "$chrome_bin" \
99+
--remote-debugging-port="${CDP_PORT}" \
100+
--no-first-run \
101+
--no-default-browser-check \
102+
--user-data-dir="$PROFILE_DIR" \
103+
--disable-background-timer-throttling \
104+
--disable-renderer-backgrounding \
105+
--disable-backgrounding-occluded-windows \
106+
--disable-ipc-flooding-protection \
107+
--disable-hang-monitor \
108+
--disable-back-forward-cache \
109+
--disable-features=CalculateNativeWinOcclusion \
110+
> /dev/null 2>&1 &
111+
112+
local pid=$!
113+
echo "$pid" > "$PID_FILE"
114+
115+
# Wait for CDP to become available (up to 10 seconds)
116+
local attempts=0
117+
while [ $attempts -lt 20 ]; do
118+
if is_running; then
119+
echo "Chrome CDP ready"
120+
echo "Endpoint: $(get_cdp_url)"
121+
echo "PID: $pid"
122+
return 0
123+
fi
124+
sleep 0.5
125+
((attempts++))
126+
done
127+
128+
echo "ERROR: Chrome started but CDP not responding on port ${CDP_PORT}" >&2
129+
echo "Check if another Chrome instance is using the debugging port" >&2
130+
exit 1
131+
}
132+
133+
cmd_stop() {
134+
if [ -f "$PID_FILE" ]; then
135+
local pid
136+
pid=$(cat "$PID_FILE")
137+
if kill -0 "$pid" 2>/dev/null; then
138+
kill "$pid" 2>/dev/null
139+
echo "Stopped Chrome CDP (PID: $pid)"
140+
else
141+
echo "PID $pid not running (stale PID file)"
142+
fi
143+
rm -f "$PID_FILE"
144+
else
145+
# Try to find and stop any Chrome with our debug port
146+
local pids
147+
pids=$(lsof -ti ":${CDP_PORT}" 2>/dev/null || true)
148+
if [ -n "$pids" ]; then
149+
echo "$pids" | xargs kill 2>/dev/null
150+
echo "Stopped Chrome CDP process(es) on port ${CDP_PORT}"
151+
else
152+
echo "No Chrome CDP instance found on port ${CDP_PORT}"
153+
fi
154+
fi
155+
}
156+
157+
cmd_status() {
158+
if is_running; then
159+
local url
160+
url=$(get_cdp_url)
161+
echo "Chrome CDP is running"
162+
echo "Endpoint: $url"
163+
# Show browser version
164+
local version
165+
version=$(curl -s "${url}/json/version" 2>/dev/null | jq -r '.Browser // "unknown"' 2>/dev/null)
166+
echo "Browser: $version"
167+
# Show open tabs
168+
local tabs
169+
tabs=$(curl -s "${url}/json/list" 2>/dev/null | jq 'length' 2>/dev/null || echo "?")
170+
echo "Open tabs: $tabs"
171+
if [ -f "$PID_FILE" ]; then
172+
echo "PID: $(cat "$PID_FILE")"
173+
fi
174+
else
175+
echo "Chrome CDP is NOT running"
176+
echo "Start it: ~/MCPs/autopilot/bin/chrome-debug.sh start"
177+
return 1
178+
fi
179+
}
180+
181+
cmd_url() {
182+
local url
183+
url=$(get_cdp_url)
184+
if [ -n "$url" ]; then
185+
echo "$url"
186+
else
187+
echo "ERROR: Chrome CDP not running on port ${CDP_PORT}" >&2
188+
exit 1
189+
fi
190+
}
191+
192+
cmd_restart() {
193+
cmd_stop
194+
sleep 1
195+
cmd_start
196+
}
197+
198+
# ─── Main ────────────────────────────────────────────────────────────────────
199+
200+
case "${1:-status}" in
201+
start) cmd_start ;;
202+
stop) cmd_stop ;;
203+
status) cmd_status ;;
204+
restart) cmd_restart ;;
205+
url) cmd_url ;;
206+
*)
207+
echo "Usage: chrome-debug.sh {start|stop|status|restart|url}"
208+
echo ""
209+
echo "Manages a persistent Chrome instance with Chrome DevTools Protocol."
210+
echo "Playwright MCP connects to this instead of launching its own browser."
211+
exit 2
212+
;;
213+
esac

config/playwright-config.json

Lines changed: 1 addition & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,6 @@
11
{
22
"browser": {
3-
"browserName": "chromium",
4-
"launchOptions": {
5-
"channel": "chrome",
6-
"headless": false,
7-
"args": [
8-
"--disable-background-timer-throttling",
9-
"--disable-renderer-backgrounding",
10-
"--disable-backgrounding-occluded-windows",
11-
"--disable-ipc-flooding-protection",
12-
"--disable-hang-monitor",
13-
"--disable-back-forward-cache",
14-
"--disable-features=CalculateNativeWinOcclusion"
15-
]
16-
},
17-
"contextOptions": {
18-
"viewport": { "width": 1280, "height": 720 }
19-
}
3+
"cdpEndpoint": "http://127.0.0.1:9222"
204
},
215
"timeouts": {
226
"action": 10000,

0 commit comments

Comments
 (0)