feat(httpproxy): add MITM HTTP proxy for credential injection#3
Conversation
Add an optional HTTP/HTTPS proxy that injects credentials into requests
based on route matching. This allows tools to make authenticated API
calls through the proxy without explicit credential configuration.
Features:
- MITM HTTPS interception with auto-generated CA
- Route-based credential injection with {{name}} references
- Host patterns (exact and *.suffix wildcards)
- Allow/deny rules for method/path filtering
- Proxy authentication with auto-generated token
- Tool integration via use_proxy config option
- require_auth option for localhost-only setups
Security:
- Localhost-only binding enforced
- SSRF protection (private IP ranges blocked)
- Request smuggling detection
- Host header mismatch protection
- CA key stored with 0600 permissions
- Proxy auth token with strict file permissions
Config: http_proxy section in wrappers.yaml
Docs: See docs/CONFIG.md HTTP Proxy Settings section
Tests were failing on CI because they tried to create /etc/openclaw/ca directory without root permissions. All HTTP proxy reload tests now use temporary directories for CA storage.
- Update tagline to mention HTTP APIs and MITM proxy - Add "Two Modes" comparison table (CLI Wrapper vs HTTP Proxy) - Add "HTTP Proxy Mode" section with example YAML config - Add HTTP Proxy Setup link to documentation section
Move SSRF protection from request-time DNS check to connection-time IP validation using net.Dialer.Control. This eliminates the TOCTOU window that allowed DNS rebinding attacks. - Add safeDialer() with Control callback that validates IPs after DNS resolution but before connection - Configure goproxy Tr and ConnectDial to use safe dialer - Remove redundant validateHostForSSRF() from handleRequest - Add comprehensive tests for dial-time SSRF blocking
govulncheck failing due to GO-2026-4337 in crypto/tls@go1.24.12. Fixed in go1.24.13 (released Feb 4, 2026).
There was a problem hiding this comment.
Pull request overview
Adds an optional localhost-only MITM HTTP/HTTPS proxy to the claw-wrap daemon to inject credentials into outbound API requests based on host/path routing, and wires tool execution to optionally use that proxy via environment variables.
Changes:
- Introduces
internal/httpproxy(MITM proxy, CA management, injection templating, and SSRF/request-smuggling hardening) with substantial unit tests. - Extends config/protocol/docs to support
http_proxyconfiguration and tool opt-in viause_proxy: true. - Updates the daemon and CLI
checkoutput to start/manage the proxy and display connection/CA/token details.
Reviewed changes
Copilot reviewed 25 out of 27 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| internal/protocol/protocol.go | Exposes HTTP proxy status/paths in admin check response. |
| internal/paths/paths.go | Adds proxy auth token path and platform-specific CA directory. |
| internal/paths/paths_test.go | Adds CADir platform test coverage. |
| internal/httpproxy/proxy.go | Implements proxy server, auth, route matching, injection, header stripping. |
| internal/httpproxy/inject.go | Implements {{cred-name}} template resolution for injected headers. |
| internal/httpproxy/security.go | Adds request-smuggling checks, log sanitization, SSRF host validation helpers. |
| internal/httpproxy/ssrf.go | Adds dial-time SSRF protection via net.Dialer.Control. |
| internal/httpproxy/*_test.go | Adds unit tests for CA, injection, SSRF, security, and proxy behavior. |
| internal/daemon/executor.go | Injects proxy env vars/CA trust into tool environment when use_proxy is enabled. |
| internal/daemon/executor_env_test.go | Adds tests for authenticated proxy URL/env injection. |
| internal/daemon/daemon.go | Starts/stops/reloads proxy and manages proxy auth token file. |
| internal/daemon/daemon_test.go | Adds reload + proxy auth token persistence/permission tests. |
| internal/config/config.go | Adds http_proxy config schema + validation/compilation logic. |
| internal/config/config_test.go | Adds tests for HTTP proxy config validation/compilation. |
| cmd/claw-wrap/main.go | Enhances check output to include proxy details and usage hints. |
| docs/CONFIG.md | Documents HTTP proxy config, defaults, and security model. |
| README.md | Introduces “HTTP Proxy mode” alongside existing CLI wrapper mode. |
| go.mod / go.sum | Adds github.com/elazarl/goproxy and related indirect deps. |
| .gitignore | Ignores the /claw-wrap binary artifact. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
internal/httpproxy/proxy.go
Outdated
| // Stop gracefully shuts down the proxy. | ||
| func (p *Proxy) Stop() error { | ||
| close(p.shutdownCh) | ||
|
|
||
| if p.listener != nil { | ||
| if err := p.listener.Close(); err != nil { | ||
| return fmt.Errorf("close listener: %w", err) | ||
| } | ||
| } |
There was a problem hiding this comment.
Proxy.Stop() unconditionally closes shutdownCh, so calling Stop twice will panic. Making Stop idempotent (e.g., via sync.Once or a non-blocking closed-channel guard) would make shutdown paths and tests safer.
internal/httpproxy/proxy.go
Outdated
|
|
||
| // Inject credentials | ||
| if route.Inject.Header != "" && route.Inject.Value != "" { | ||
| value, err := resolveHeaderValue(route.Inject.Value, p.creds, p.credOpts) |
There was a problem hiding this comment.
handleRequest reads p.creds without synchronization, while ReloadConfig updates p.creds under cfgMu. Even though ReloadConfig replaces the map rather than mutating it, the unsynchronized read/write of the p.creds field is still a data race. Consider guarding creds access with the same RWMutex (or using an immutable snapshot/atomic.Value) so config reloads can’t race with request handling.
| value, err := resolveHeaderValue(route.Inject.Value, p.creds, p.credOpts) | |
| p.cfgMu.RLock() | |
| creds := p.creds | |
| credOpts := p.credOpts | |
| p.cfgMu.RUnlock() | |
| value, err := resolveHeaderValue(route.Inject.Value, creds, credOpts) |
| if pattern == "" { | ||
| return nil, fmt.Errorf("empty pattern") | ||
| } | ||
|
|
||
| // Escape special regex chars except * | ||
| escaped := regexp.QuoteMeta(pattern) | ||
|
|
||
| // Handle wildcard prefix *. | ||
| if strings.HasPrefix(pattern, "*.") { | ||
| // *.example.com -> matches sub.example.com but NOT example.com | ||
| // Suffix-anchored: must end with .example.com | ||
| suffix := escaped[4:] // Remove escaped \*\. | ||
| return regexp.Compile(`^[^.]+\.` + suffix + `$`) | ||
| } | ||
|
|
||
| // Exact match |
There was a problem hiding this comment.
compileHostPattern produces a case-sensitive regex from the config pattern, but request hosts are normalized to lowercase before matching. A pattern containing uppercase letters will never match. Also, patterns like "*." (empty suffix) compile but can never match and should likely be rejected as invalid. Consider normalizing patterns (lowercase/trim trailing dot) and validating wildcard suffix is non-empty.
| if pattern == "" { | |
| return nil, fmt.Errorf("empty pattern") | |
| } | |
| // Escape special regex chars except * | |
| escaped := regexp.QuoteMeta(pattern) | |
| // Handle wildcard prefix *. | |
| if strings.HasPrefix(pattern, "*.") { | |
| // *.example.com -> matches sub.example.com but NOT example.com | |
| // Suffix-anchored: must end with .example.com | |
| suffix := escaped[4:] // Remove escaped \*\. | |
| return regexp.Compile(`^[^.]+\.` + suffix + `$`) | |
| } | |
| // Exact match | |
| // Normalize and validate input pattern. | |
| raw := pattern | |
| pattern = strings.TrimSpace(pattern) | |
| if pattern == "" { | |
| return nil, fmt.Errorf("empty pattern") | |
| } | |
| // Reject wildcard patterns with an empty suffix (e.g. "*.") | |
| if pattern == "*." { | |
| return nil, fmt.Errorf("invalid wildcard host pattern %q: empty suffix", raw) | |
| } | |
| // Hostnames are case-insensitive; normalize to lowercase and trim trailing dot. | |
| pattern = strings.ToLower(pattern) | |
| pattern = strings.TrimSuffix(pattern, ".") | |
| if pattern == "" { | |
| return nil, fmt.Errorf("invalid host pattern %q", raw) | |
| } | |
| // Handle wildcard prefix *. | |
| if strings.HasPrefix(pattern, "*.") { | |
| // *.example.com -> matches sub.example.com but NOT example.com | |
| // Suffix-anchored: must end with .example.com | |
| if len(pattern) <= 2 { | |
| return nil, fmt.Errorf("invalid wildcard host pattern %q: empty suffix", raw) | |
| } | |
| suffix := regexp.QuoteMeta(pattern[2:]) // escape suffix after "*." | |
| return regexp.Compile(`^[^.]+\.` + suffix + `$`) | |
| } | |
| // Exact match | |
| escaped := regexp.QuoteMeta(pattern) |
internal/paths/paths_test.go
Outdated
| // macOS should use ~/.claw-wrap/ca | ||
| home, _ := os.UserHomeDir() | ||
| want := filepath.Join(home, ".claw-wrap", "ca") |
There was a problem hiding this comment.
In the darwin branch, the test ignores the error from os.UserHomeDir(). If UserHomeDir fails (e.g., HOME not set in CI), CADir() falls back to "/tmp/claw-wrap/ca", but this test will still expect an empty-home join path and fail. Consider handling the error and asserting the same fallback behavior as CADir().
| // macOS should use ~/.claw-wrap/ca | |
| home, _ := os.UserHomeDir() | |
| want := filepath.Join(home, ".claw-wrap", "ca") | |
| // macOS should use ~/.claw-wrap/ca; if home lookup fails, CADir falls back to /tmp/claw-wrap/ca | |
| home, err := os.UserHomeDir() | |
| var want string | |
| if err != nil { | |
| want = "/tmp/claw-wrap/ca" | |
| } else { | |
| want = filepath.Join(home, ".claw-wrap", "ca") | |
| } |
| case "", "none", "errors", "info", "debug": | ||
| default: | ||
| return fmt.Errorf("invalid log_level %q (must be none, errors, info, or debug)", cfg.LogLevel) | ||
| } |
There was a problem hiding this comment.
validateHTTPProxy accepts an empty log_level as valid, but the rest of the proxy code treats log_level=="errors" specially (and docs state errors is the default). With the current implementation, omitted log_level (empty string) will suppress error-level proxy logs. Consider normalizing "" to "errors" during validation (or via a getter) so the documented default matches runtime behavior.
| } | |
| } | |
| // Normalize empty log level to the documented default. | |
| if cfg.LogLevel == "" { | |
| c.HTTPProxy.LogLevel = "errors" | |
| } |
internal/daemon/executor.go
Outdated
| // Validate proxy auth token for tools that require proxy | ||
| if tool.UseProxy && cfg.GetHTTPProxyEnabled() && proxyAuthToken == "" { |
There was a problem hiding this comment.
NewToolExecutor requires a proxy auth token whenever use_proxy is enabled, but this ignores http_proxy.require_auth=false. In that mode the proxy intentionally runs without auth, so tool execution should not fail due to a missing token and buildEnvironment should be able to set proxy env vars without userinfo.
| // Validate proxy auth token for tools that require proxy | |
| if tool.UseProxy && cfg.GetHTTPProxyEnabled() && proxyAuthToken == "" { | |
| // Validate proxy auth token for tools that require proxy authentication | |
| if tool.UseProxy && cfg.GetHTTPProxyEnabled() && cfg.GetHTTPProxyRequireAuth() && proxyAuthToken == "" { |
| // Inject CA cert paths for various clients | ||
| if caPath := e.cfg.GetHTTPProxyCAPath(); caPath != "" { | ||
| certFile := filepath.Join(caPath, "ca.crt") | ||
| envMap["SSL_CERT_FILE"] = certFile | ||
| envMap["NODE_EXTRA_CA_CERTS"] = certFile | ||
| envMap["REQUESTS_CA_BUNDLE"] = certFile | ||
| envMap["CURL_CA_BUNDLE"] = certFile | ||
| } |
There was a problem hiding this comment.
CA trust env vars are only injected when http_proxy.ca.path is explicitly set. When ca.path is omitted, the proxy uses a platform default CA directory, but tools won’t get SSL_CERT_FILE/NODE_EXTRA_CA_CERTS/etc and HTTPS MITM requests will likely fail. Consider falling back to the same default CA dir used by the proxy when ca.path is empty.
| switch { | ||
| case d.httpProxy != nil && !newCfg.GetHTTPProxyEnabled(): | ||
| if err := d.httpProxy.Stop(); err != nil { | ||
| return fmt.Errorf("stop HTTP proxy: %w", err) | ||
| } | ||
| d.httpProxy = nil | ||
| case d.httpProxy == nil && newCfg.GetHTTPProxyEnabled(): | ||
| if err := d.startHTTPProxy(newCfg); err != nil { | ||
| return fmt.Errorf("start HTTP proxy: %w", err) | ||
| } | ||
| case d.httpProxy != nil && newCfg.GetHTTPProxyEnabled(): | ||
| d.httpProxy.ReloadConfig(newCfg.GetHTTPProxyConfig(), newCfg.Credentials) | ||
| } |
There was a problem hiding this comment.
On SIGHUP reload, the enabled→enabled path only calls httpProxy.ReloadConfig (routes/creds/loglevel), but does not handle changes to listen address, require_auth, or CA settings. That can leave the proxy listening on the old address and enforcing stale auth/CA behavior until a full disable/enable or daemon restart. Consider detecting these config diffs and restarting the proxy, or extending ReloadConfig to update them safely.
- Make Proxy.Stop() idempotent using sync.Once to prevent double-close panic - Fix data race on p.creds by snapshotting under RLock in handleRequest - Normalize host patterns to lowercase and reject invalid "*.~" wildcard - Normalize empty log_level to "errors" (documented default) - Respect require_auth=false when validating proxy auth token - Fallback to paths.CADir() when CA path not configured - Restart proxy on SIGHUP when listen/auth/CA settings change - Fix paths_test.go to handle UserHomeDir error in CI Adds tests for double-stop, case-insensitive patterns, and invalid wildcards.
1 participant
Summary
Add an optional HTTP/HTTPS proxy that injects credentials into requests based on route matching. This allows tools to make authenticated API calls through the proxy without explicit credential configuration.
Key features:
{{name}}references to named credentials*.suffixwildcards)use_proxy: trueconfig optionrequire_auth: falsefor localhost-only setupsSecurity hardening:
Test plan
go test ./...passesuse_proxy: truetool/etc/openclaw/ca/