Skip to content

feat: add browser tool powered by playwright-go#187

Open
strelov1 wants to merge 2 commits intosipeed:mainfrom
strelov1:feat/browser-playwright
Open

feat: add browser tool powered by playwright-go#187
strelov1 wants to merge 2 commits intosipeed:mainfrom
strelov1:feat/browser-playwright

Conversation

@strelov1
Copy link

@strelov1 strelov1 commented Feb 15, 2026

Summary

  • Add browser automation tool using playwright-go
  • Support two connection protocols:
    • CDP (ConnectOverCDP) for Chromium/Browserless
    • Playwright Wire Protocol (Firefox.Connect) for Firefox/Camoufox
  • 13 browser actions: navigate, click, type, screenshot, get_text, evaluate, wait, scroll, hover, select, pdf, cookies, close
  • Add BrowserConfig to config with protocol, URLs, token, stealth, timeouts
  • Unit tests (10 tests) + integration test scaffolding for both protocols

Configuration

"tools": {
  "browser": {
    "enabled": true,
    "protocol": "cdp",
    "cdp_url": "ws://localhost:3000",
    "ws_url": "",
    "token": "YOUR_TOKEN",
    "stealth": true,
    "launch_timeout": 120000,
    "action_timeout": 30000
  }
}

New dependency

  • github.com/playwright-community/playwright-go v0.5200.1

Test plan

  • go build ./pkg/... — compiles without errors
  • go test ./pkg/... — all existing tests pass
  • 10 unit tests for browser tool pass
  • Integration test with Browserless (CDP): BROWSER_TEST_CDP_URL=ws://... go test ./pkg/tools/ -run TestBrowserTool_Integration_CDP -v
  • Integration test with Camoufox (Playwright): BROWSER_TEST_PW_URL=ws://... go test ./pkg/tools/ -run TestBrowserTool_Integration_Playwright -v

🤖 Generated with Claude Code

Replace chromedp with playwright-go to support two connection protocols:
- CDP (Chromium/Browserless) via ConnectOverCDP
- Playwright Wire Protocol (Firefox/Camoufox) via Firefox.Connect

13 actions: navigate, click, type, screenshot, get_text, evaluate,
wait, scroll, hover, select, pdf, cookies, close.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@strelov1 strelov1 force-pushed the feat/browser-playwright branch from 394356b to 7142079 Compare February 15, 2026 03:53
@diffray-bot
Copy link

Changes Summary

This PR adds browser automation capabilities to the picoclaw agent using playwright-go. It introduces a new browser tool supporting both CDP (Chrome DevTools Protocol) for Chromium/Browserless and Playwright Wire Protocol for Firefox/Camoufox, with 13 browser actions including navigation, screenshots, form interaction, and content extraction.

Type: feature

Components Affected: tools, config, agent, dependencies

Files Changed
File Summary Change Impact
pkg/tools/browser.go New browser automation tool with 674 lines implementing 13 actions (navigate, click, type, screenshot, get_text, evaluate, wait, scroll, hover, select, pdf, cookies, close) supporting both CDP and Playwright protocols 🔴
pkg/tools/browser_test.go Comprehensive test suite with 10 unit tests and integration test scaffolding for both CDP and Playwright protocols 🟡
pkg/config/config.go Added BrowserConfig struct with 8 fields (enabled, protocol, cdp_url, ws_url, token, stealth, launch_timeout, action_timeout) and integrated into ToolsConfig ✏️ 🟡
pkg/agent/loop.go Added 18-line conditional registration of browser tool in createToolRegistry function based on configuration ✏️ 🟢
config/config.example.json Added browser configuration section with default values (disabled by default, CDP protocol, localhost:3000, 120s/30s timeouts) ✏️ 🟢
go.mod Added playwright-community/playwright-go v0.5200.1 as direct dependency and several transitive dependencies ✏️ 🟡
go.sum Updated checksums for new dependencies including playwright-go and its transitive dependencies ✏️ 🟢
Architecture Impact
  • New Patterns: Protocol abstraction pattern (CDP vs Playwright), Connection pooling/reuse pattern, Lazy connection initialization
  • Dependencies: added: playwright-community/playwright-go v0.5200.1, added: deckarep/golang-set/v2 v2.7.0, added: go-jose/go-jose/v3 v3.0.4, added: go-stack/stack v1.8.1, added: mitchellh/go-ps v1.0.0
  • Coupling: Browser tool is loosely coupled via configuration; conditionally registered only when enabled and URL configured

Risk Areas: Connection management: browser connections are maintained as stateful resources with mutex protection but no explicit timeout/cleanup beyond close action, Resource leaks: if browser.Close() or pw.Stop() fail, resources may leak; errors are collected but connection marked as closed anyway, Security: JavaScript evaluation via 'evaluate' action allows arbitrary code execution in browser context, Cookie manipulation: delete action uses clear-and-restore pattern which could fail partially leaving cookies in inconsistent state, Text truncation: get_text action truncates at 50000 chars which could lose important data without clear indication to LLM beyond [truncated] note, Concurrent access: mutex protects connection state but individual page operations are not synchronized, potential race conditions if tool called concurrently

Suggestions
  • Consider adding explicit connection pooling or lifecycle management to prevent resource exhaustion from abandoned connections
  • Add connection health checks or heartbeat to detect stale browser connections
  • Consider making the 50000 char truncation limit configurable
  • Add rate limiting or safeguards around the 'evaluate' action to prevent abuse
  • Consider adding connection timeout/idle timeout configuration to automatically close unused browser instances
  • Cookie delete operation should handle partial failure more gracefully (currently ignores error from AddCookies)
  • Consider adding metrics/logging for connection pool usage and browser action latencies

Full review in progress... | Powered by diffray

@diffray-bot
Copy link

Review Summary

Free public review - Want AI code reviews on your PRs? Check out diffray.ai

Validated 81 issues: 23 kept, 58 filtered

Issues Found: 23

💬 See 23 individual line comment(s) for details.

📊 16 unique issue type(s) across 23 location(s)

📋 Full issue list (click to expand)

🔴 CRITICAL - Race condition: page accessed without mutex protection

Agent: bugs

Category: bug

File: pkg/tools/browser.go:297-674

Description: Action methods (doNavigate, doClick, etc.) access t.page without mutex after ensureConnected() returns. If doClose() runs concurrently, t.page becomes nil causing panic.

Suggestion: Add t.mu.Lock()/defer t.mu.Unlock() at start of each action method, or hold mutex during entire Execute() call.

Confidence: 90%

Rule: go_concurrent_data_race_prevention


🔴 CRITICAL - Integration tests depend on external browser services

Agent: testing

Category: testing

File: pkg/tools/browser_test.go:143-174

Description: Integration tests require real external browser services via environment variables. These will skip in CI/CD without access to these services. Unit tests for action handlers would improve coverage.

Suggestion: Create mocks for playwright-go Browser, Page, and Playwright interfaces. Use dependency injection to inject mocked playwright instances for unit testing action handlers.

Confidence: 85%

Rule: testing_unit_external_dependencies_go


🟠 HIGH - Ignored error when re-adding cookies in delete operation (2 occurrences)

Agent: bugs

Category: bug

📍 View all locations
File Description Suggestion Confidence
pkg/tools/browser.go:625-627 After clearing cookies and filtering, AddCookies(toReAdd) return value is ignored. If it fails, user... Check the error: if len(toReAdd) > 0 { if err := browserCtx.AddCookies(toReAdd); err != nil { return... 95%
pkg/tools/browser.go:304-321 After ensureConnected(), t.page is accessed without nil check. Due to race with doClose(), t.page co... Add nil check: if t.page == nil { return &ToolResult{ForLLM: "Error: browser connection lost"} } 88%

Rule: go_nil_dereference_check


🟠 HIGH - Resource cleanup missing defer for playwright.Stop() on Connect error

Agent: go

Category: bug

File: pkg/tools/browser.go:224-226

Description: When Firefox.Connect fails, pw.Stop() is called manually. If panic occurs between Run() and error check, playwright process leaks.

Suggestion: Use defer pattern with cleanup flag: success := false; defer func() { if !success { pw.Stop() } }(); then set success = true before return nil.

Confidence: 75%

Rule: go_use_defer_to_release_resources


🟠 HIGH - Slice access without bounds check

Agent: go

Category: bug

File: pkg/tools/browser.go:242-244

Description: The code accesses contexts[0].Pages()[0] without storing the Pages() result. While it checks len(contexts[0].Pages()) > 0, calling Pages() again could return a different slice due to race conditions, causing a panic if the second call returns an empty slice.

Suggestion: Store the result of contexts[0].Pages() in a variable before checking length and accessing index 0:
pages := contexts[0].Pages()
if len(contexts) > 0 && len(pages) > 0 {
page = pages[0]
} else {
page, err = browser.NewPage()
// ... handle error
}

Confidence: 85%

Rule: bug_slice_bounds_go


🟠 HIGH - Unbounded screenshot data allocation (3 occurrences)

Agent: performance

Category: performance

📍 View all locations
File Description Suggestion Confidence
pkg/tools/browser.go:359-396 The doScreenshot function loads entire screenshots into memory with no size limit. Full-page screens... Add a maximum screenshot size limit (e.g., 10MB for raw data). Check dimensions before capture using... 75%
pkg/tools/browser.go:399-420 The doGetText function extracts the entire page text into memory before truncating at 50,000 charact... Use page.Evaluate with JavaScript to truncate at the source: `page.Evaluate("document.querySelector(... 80%
pkg/tools/browser.go:524-540 The doPDF function generates PDFs with no size limits using t.page.PDF() with default options. Large... Add maximum PDF size limit (e.g., 50MB). Use Playwright's PagePdfOptions to set PrintBackground: fal... 75%

Rule: perf_missing_resource_limits


🟠 HIGH - Arbitrary JavaScript Execution via LLM Output

Agent: security

Category: security

File: pkg/tools/browser.go:422-434

Description: The doEvaluate function accepts an 'expression' parameter and executes it directly via page.Evaluate() without any validation. This allows arbitrary JS execution in the browser context if LLM output is manipulated.

Suggestion: Consider implementing a whitelist of allowed JavaScript patterns, restricting to read-only operations, or adding a warning about the security implications in documentation.

Confidence: 85%

Rule: sec_llm_output_not_validated


🟠 HIGH - Server-Side Request Forgery (SSRF) via Unvalidated URL Navigation (2 occurrences)

Agent: security

Category: security

📍 View all locations
File Description Suggestion Confidence
pkg/tools/browser.go:297-321 The doNavigate function accepts a URL parameter and navigates to it without validation. An attacker ... Implement URL validation: allowlist schemes (http/https only), block private IP ranges (127.0.0.0/8,... 85%
pkg/tools/browser.go:267-268 Token is concatenated directly into URL parameters without proper URL encoding. Special characters i... Use url.QueryEscape() or url.Values to properly encode URL parameters: url.Values{"token": []string{... 70%

Rule: security_missing_input_validation


🟠 HIGH - Missing unit test coverage for error paths in action handlers (2 occurrences)

Agent: testing

Category: testing

📍 View all locations
File Description Suggestion Confidence
pkg/tools/browser_test.go:1-309 Only 3 out of 13 action handlers have unit tests. Missing tests for: doClick, doType, doEvaluate, do... Add unit tests for each error path using a mock page. Test missing required parameters and error res... 80%
pkg/tools/browser_test.go:92-108 TestBrowserTool_Unit_BuildCdpURL only tests token and stealth parameters. Missing: launch timeout pa... Add test cases for: launchTimeout in URL, URL with existing '?' (verify '&' separator), URL without ... 70%

Rule: test_missing_edge_case_coverage


🟡 MEDIUM - Unchecked error return from os.MkdirAll

Agent: bugs

Category: bug

File: pkg/agent/loop.go:109

Description: Error from os.MkdirAll(workspace, 0755) is not checked. If directory creation fails, subsequent file operations will fail.

Suggestion: Check the error: if err := os.MkdirAll(workspace, 0755); err != nil { return nil, fmt.Errorf("failed to create workspace: %w", err) }

Confidence: 85%

Rule: go_unchecked_error_return


🟡 MEDIUM - BrowserConfig has overlapping fields (CdpURL and WsURL)

Agent: architecture

Category: quality

File: pkg/config/config.go:213-222

Description: Both CdpURL and WsURL exist with undocumented fallback logic. WsURL takes precedence over CdpURL (seen in loop.go:91-94) but this is not documented.

Suggestion: Document precedence in struct comments or use single URL field with protocol determining interpretation.

Confidence: 70%

Rule: arch_config_validation_after_state_change


🟡 MEDIUM - JSON marshalling error ignored in doEvaluate (2 occurrences)

Agent: go

Category: quality

📍 View all locations
File Description Suggestion Confidence
pkg/tools/browser.go:433 json.Marshal error is assigned to blank identifier. While marshalling primitive types rarely fails, ... Handle the error: if err != nil, log it or return an error message to the user indicating the result... 65%
pkg/agent/loop.go:510 json.Marshal error is ignored when serializing tool call arguments. This is in the critical path whe... Handle the error: if marshalling fails, log an error and skip this tool call or return an error to p... 70%

Rule: go_add_error_handling_for_json_marshalling


🟡 MEDIUM - URL parameter concatenation lacks proper encoding

Agent: quality

Category: quality

File: pkg/tools/browser.go:263-286

Description: The buildCdpURL function manually constructs URL query parameters by string concatenation without proper URL encoding. The token and launch JSON parameters are not URL-encoded, which could cause issues with special characters.

Suggestion: Use url.Values from the net/url package to properly encode query parameters instead of manual string concatenation.

Confidence: 80%

Rule: go_network_timeout_and_addressing


🟡 MEDIUM - Sensitive Cookie Data Exposure via LLM Context (2 occurrences)

Agent: security

Category: security

📍 View all locations
File Description Suggestion Confidence
pkg/tools/browser.go:542-557 The doCookies 'get' action returns all browser cookies (including auth tokens and session IDs) direc... Consider redacting cookie values by default, showing only names and metadata. Provide a separate act... 75%
pkg/tools/browser.go:262-287 buildCdpURL places the auth token in query parameters. URLs with tokens may be logged by browsers, s... If CDP protocol requires token in URL, ensure URLs are never logged. Add URL sanitization in logging... 70%

Rule: sec_log_sensitive_data_redaction


🔵 LOW - Custom string contains function reimplements stdlib

Agent: quality

Category: quality

File: pkg/tools/browser_test.go:125-136

Description: Test file implements custom contains/containsSubstring functions instead of using strings.Contains from standard library.

Suggestion: Replace with strings.Contains: import "strings" and use strings.Contains(s, substr)

Confidence: 90%

Rule: go_custom_reimplements_stdlib


🔵 LOW - Error ignored when getting user home directory

Agent: go

Category: quality

File: pkg/config/config.go:446

Description: os.UserHomeDir() error is ignored in expandHome. If getting the home directory fails, an empty string is used which could lead to incorrect path resolution.

Suggestion: Check the error and either return the original path unchanged, log a warning, or document that empty home is acceptable. Consider: home, err := os.UserHomeDir(); if err != nil { return path }

Confidence: 65%

Rule: go_handle_errors_not_ignore


🔇 3 low-severity issue(s) not posted (min: medium)

Issues below medium severity are saved but not posted as comments.
View all issues in the full review details.

📝 10 additional issue(s) shown in summary only (max: 10 inline)

To reduce noise, only 10 inline comments are posted.
All issues are listed above in the full issue list.

🔗 View full review details


Review ID: 333f4447-a656-40c2-9205-935c8a1613cd
Rate it 👍 or 👎 to improve future reviews | Powered by diffray

- Handle ignored error from AddCookies in cookie delete operation
- Check os.MkdirAll error in workspace directory creation
- Store Pages() result in variable to avoid redundant calls
- Document CdpURL/WsURL field precedence in BrowserConfig

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@Leeaandrob
Copy link
Collaborator

@Zepan This PR addresses roadmap issue #293 (Autonomous Browser Operations — priority: high) using playwright-go. It's a comprehensive implementation at +1055 lines.

Consideration: PR #318 also targets the same roadmap item but uses a CLI subprocess approach (agent-browser), which is lighter weight and more aligned with PicoClaw's pattern of external tool integration (Codex CLI, Claude CLI providers). Playwright-go would add significant binary size and CGO complexity.

Recommendation: Compare with #318 before deciding. For a project targeting $10 hardware with <10MB RAM, the subprocess approach may be more appropriate than embedding playwright-go.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants