Add ApiRequestContext to Selenium

[mayank-at-sauce]

The problem this would solve:

Selenium currently has no way to make HTTP API calls that share authentication/cookie state with the browser session. This is a significant gap compared to Playwright's APIRequestContext,
which is one of the most-used features in modern test automation.

Common real-world scenarios that require this:

Fast test setup — Log in via API (POST /auth/login) to get auth cookies, then use those cookies in the browser to skip slow UI login flows
API + UI hybrid testing — Verify a UI action triggers the correct backend state (e.g., click "Delete" button, then call GET /api/items to confirm deletion)
Test data seeding — Create test fixtures via API before browser tests, sharing the same authenticated session
Backend assertion — After a browser interaction, assert API responses without opening new pages
Today, users must manually extract cookies from driver.get_cookies(), construct a separate requests.Session, and handle cookie format conversion. The reverse (API cookies → browser)
requires iterating and calling driver.add_cookie() per cookie, with domain/path matching issues. There is no built-in support for this.

Third-party libraries (https://github.com/tryolabs/requestium, https://github.com/cryzed/Selenium-Requests) exist but have limitations — they still require a running browser for every
request, lack isolated cookie contexts, and have no auth state persistence to disk.

How I expect the feature would work:

A new APIRequestContext class accessible from the WebDriver instance:

1. Make API calls that share cookies with the browser
   response = driver.request.get("https://api.example.com/users")
   assert response.status == 200
   data = response.json()

2. Log in via API, cookies automatically available in browser
   driver.request.post("https://example.com/api/login", data={
   "username": "testuser",
   "password": "testpass"
   })

Browser now has the auth cookies — no UI login needed
driver.get("https://example.com/dashboard")

3. Create an isolated request context (separate cookie jar)
   api = driver.request.new_context(base_url="https://api.example.com/")
   api.post("/seed-data", data={"item": "test"})
   api.dispose()

4. Save/load auth state to disk for reuse across tests
   driver.request.storage_state(path="auth.json")

In another test:
api = driver.request.new_context(storage_state="auth.json")

Common kwargs for HTTP methods: data, form, headers, params, timeout, max_redirects, fail_on_status_code

Key behaviors:

Bidirectional cookie sync — Cookies set by API responses are automatically available in the browser (via driver.add_cookie()), and browser cookies are automatically sent with API
requests (via driver.get_cookies())
Isolated contexts — new_context() creates a separate cookie jar that does not affect the browser session
Auth state persistence — storage_state() serializes cookies to a JSON file; new_context(storage_state="file.json") restores them
No browser required for API calls — HTTP requests use Python's standard urllib or http.client (no external dependencies), the browser only needs to be running if you want cookie sync
Have you considered any alternatives or workarounds?
Current workarounds and their limitations:

Manual cookie transfer with requests library:
import requests
session = requests.Session()
for cookie in driver.get_cookies():
session.cookies.set(cookie['name'], cookie['value'], domain=cookie.get('domain'))
response = session.get("https://api.example.com/data")
Problems: Verbose, error-prone (domain/path/secure flag handling), no auto-sync back to browser, requires external dependency.

requestium library (github.com/tryolabs/requestium):
Provides transfer_session_cookies_to_driver() and transfer_driver_cookies_to_session(). But: requires explicit manual transfer calls (no auto-sync), no isolated contexts, no auth state
persistence, no storage_state to disk.

selenium-requests library (github.com/cryzed/Selenium-Requests):
Auto-syncs cookies but requires a running WebDriver for every HTTP request (opens hidden tabs), last updated Feb 2024 (stale), and has open bugs with modern browsers.

JavaScript execution (driver.execute_script with fetch()):
Can make API calls within the browser, but: responses are hard to extract to Python, no isolated contexts, can't run before page navigation, subject to CORS restrictions.

None of these provide a clean, built-in, first-class API that matches the simplicity and reliability of Playwright's APIRequestContext.

[CLAassistant]

All committers have signed the CLA.

[qodo-code-review]

Review Summary by Qodo

Add APIRequestContext for HTTP requests with browser cookie sync

✨ Enhancement

Walkthroughs

Description

• Add APIRequestContext for HTTP requests with automatic browser cookie synchronization
• Implement bidirectional cookie sync between browser and API requests
• Support isolated contexts via new_context() with separate cookie jars
• Add storage_state() for auth state persistence to/from JSON files
• Support all HTTP methods (GET, POST, PUT, PATCH, DELETE, HEAD) plus fetch()
• Implement RFC 6265 compliant cookie matching with domain, path, secure, and expiry validation
• Add comprehensive unit and integration tests with local HTTP server

File Changes

1. py/selenium/webdriver/common/api_request_context.py ✨ Enhancement +656/-0

New APIRequestContext module with cookie sync

• New 656-line module implementing APIRequestContext, APIResponse, and APIRequestFailure classes
• Implements _cookie_matches() for RFC 6265 domain/path/secure/expiry matching with host-only cookie
 support
• Implements _parse_set_cookie() for parsing Set-Cookie headers into cookie dicts
• Base class _BaseRequestContext with shared HTTP logic (URL resolution, headers, body prep, query
 params)
• APIRequestContext for browser-synced requests with driver integration and storage_state() support
• _IsolatedAPIRequestContext for isolated cookie jar with no browser sync
• Uses urllib3 for HTTP client, no BiDi/WebSocket dependency

py/selenium/webdriver/common/api_request_context.py

---

2. py/selenium/webdriver/remote/webdriver.py ✨ Enhancement +23/-0

Add request property and lifecycle management

• Import APIRequestContext from api_request_context module
• Add _request instance variable initialized to None in __init__
• Add request property that lazily initializes and returns APIRequestContext bound to driver
• Call _request.dispose() in quit() method before stopping client

py/selenium/webdriver/remote/webdriver.py

---

3. py/test/selenium/webdriver/common/api_request_context_tests.py 🧪 Tests +225/-0

Integration tests for APIRequestContext

• New 225-line integration test suite for APIRequestContext
• Tests cover request initialization, HTTP methods (GET, HEAD, POST), response parsing
• Tests verify browser cookie sync (sent with requests, synced from responses)
• Tests for isolated contexts, storage_state export/import, fail_on_status_code behavior
• Tests use real webserver fixture with pages.url() helper

py/test/selenium/webdriver/common/api_request_context_tests.py

---

View more (2)

4. py/test/unit/selenium/webdriver/common/api_request_context_tests.py 🧪 Tests +1395/-0

Comprehensive unit tests with local HTTP server

• New 1395-line comprehensive unit test suite covering all APIRequestContext components
• Tests _cookie_matches() with RFC 6265 domain/path/secure/expiry edge cases (host-only cookies, dot
 domains, IP addresses)
• Tests _parse_set_cookie() header parsing including Max-Age, Expires, SameSite, HttpOnly, Secure
• Tests APIResponse methods (json(), text(), body(), dispose(), ok property)
• Tests APIRequestFailure exception behavior and message formatting
• Tests _BaseRequestContext URL resolution, header merging, body preparation, query params
• Tests _IsolatedAPIRequestContext cookie jar CRUD, deduplication, storage_state roundtrip
• Tests APIRequestContext with mocked driver for cookie sync, storage_state file I/O, new_context()
• End-to-end tests with local HTTP server covering all HTTP methods, redirects, cookie lifecycle
• Tests error handling (FileNotFoundError, ValueError for invalid JSON, OSError for unwritable
 paths)

py/test/unit/selenium/webdriver/common/api_request_context_tests.py

---

5. py/test/selenium/webdriver/common/webserver.py 🧪 Tests +48/-1

Add test endpoints for API request testing

• Add urllib.parse import for query string parsing
• Add /echo_headers endpoint that echoes request headers as plain text
• Add /echo_json endpoint that returns {"status": "ok"} as JSON
• Add /set_cookie endpoint that parses name/value query params and sets Set-Cookie header
• Add do_HEAD() method handler for HEAD requests
• Modify do_POST() to add /echo_body endpoint that echoes request body
• Add content-length safety check with .get() fallback

py/test/selenium/webdriver/common/webserver.py

---

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (2) 📘 Rule violations (2) 📎 Requirement gaps (0)

1. resp.reason AttributeError ☑ 🐞 Bug ⛯ Reliability

Description

_BaseRequestContext._build_response unconditionally reads resp.reason, which will raise
AttributeError under urllib3 2.x and prevents any APIRequestContext request from succeeding. The
intended fallback to http.client.responses never runs because the attribute access fails first.

Code

py/selenium/webdriver/common/api_request_context.py[R402-405]

+        resp_headers = {k.lower(): v for k, v in resp.headers.items()}
+        # urllib3 2.x removed resp.reason; fall back to stdlib phrase lookup
+        status_text = resp.reason or http_status_phrases.get(resp.status, "")
+        return APIResponse(

Evidence

The code comments that urllib3 2.x removed resp.reason but still accesses it. This repo pins
urllib3 to 2.x (>=2.6.3,<3.0), so this codepath is exercised with urllib3 2.x at runtime.

py/selenium/webdriver/common/api_request_context.py[400-408]
py/pyproject.toml[28-35]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`_build_response()` reads `resp.reason` even though the code comment notes urllib3 2.x removed it; with this repo’s pinned urllib3 2.x, this can raise `AttributeError` and break every request.
### Issue Context
The code already has a correct fallback mechanism (`http.client.responses`) but it’s unreachable if attribute access fails.
### Fix Focus Areas
- py/selenium/webdriver/common/api_request_context.py[400-405]
### Suggested change (sketch)

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Redirect limit ignored ☑ 🐞 Bug ✓ Correctness

Description

max_redirects is documented/stored as an integer but is passed to urllib3 as
redirect=max_redirects, so non-zero values act as a truthy flag rather than enforcing the
requested redirect limit. This silently ignores the max_redirects configuration and may follow
more redirects than the caller specified.

Code

py/selenium/webdriver/common/api_request_context.py[R387-398]

+        timeout = kwargs.get("timeout", self._timeout)
+        max_redirects = kwargs.get("max_redirects", self._max_redirects)
+
+        return self._pool.request(
+            method,
+            url,
+            headers=headers,
+            body=body,
+            timeout=timeout,
+            redirect=max_redirects,
+            preload_content=True,
+        )

Evidence

The constructor and public method docs treat max_redirects as an integer limit, but the request
call wires it into a parameter named redirect, with no other code applying the integer as a limit
(e.g., via retries/redirect policy). Therefore the configured limit is not enforced.

py/selenium/webdriver/common/api_request_context.py[233-246]
py/selenium/webdriver/common/api_request_context.py[248-254]
py/selenium/webdriver/common/api_request_context.py[383-398]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`max_redirects` is exposed as an integer limit, but `_execute_request()` passes it as `redirect=max_redirects`, which does not enforce the numeric limit.
### Issue Context
The API surface (constructor + per-call kwargs) promises a redirect *count* cap. The implementation currently only toggles redirect behavior based on truthiness.
### Fix Focus Areas
- py/selenium/webdriver/common/api_request_context.py[383-398]
### Suggested change (sketch)
- Introduce a redirect policy using urllib3’s retry configuration so the integer is respected.
- Keep behavior:
- `max_redirects <= 0` -> do not follow redirects
- `max_redirects > 0` -> follow redirects up to that count
- Add/adjust a unit test to assert the limit is enforced (e.g., a redirect chain longer than the configured max).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. delete_cookie errors swallowed silently 📘 Rule violation ⛯ Reliability

Description

The new cookie-sync logic catches all exceptions from delete_cookie() and ignores them without any
logging, which can hide real failures and make troubleshooting difficult. This reduces
resiliency/diagnosability of error paths.

Code

py/selenium/webdriver/common/api_request_context.py[R586-590]

+                try:
+                    self._driver.delete_cookie(cookie["name"])
+                except Exception:
+                    pass
+                continue

Evidence

PR Compliance ID 15 requires intentional, resilient error handling and logging; the added code uses
an overly broad catch and silently discards exceptions during cleanup, making failures
non-actionable.

py/selenium/webdriver/common/api_request_context.py[586-590]
Best Practice: Learned patterns

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`APIRequestContext._handle_response_cookies()` silently swallows all exceptions from `self._driver.delete_cookie(...)`, which can hide legitimate failures and makes troubleshooting difficult.
## Issue Context
This occurs when processing expired cookies from `Set-Cookie` headers. The compliance requirement asks for resilient error handling with intentional catch scope and actionable logging.
## Fix Focus Areas
- py/selenium/webdriver/common/api_request_context.py[586-590]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

4. Redundant inline comments added 📘 Rule violation ⛯ Reliability

Description

Several new inline comments restate obvious behavior (what the code does) rather than explaining
rationale, increasing noise and reducing maintainability. This conflicts with the project's
comment-style requirement.

Code

py/selenium/webdriver/common/api_request_context.py[R434-442]

+        # Apply cookies
+        matching_cookies = self._get_cookies_for_request(url)
+        if matching_cookies:
+            cookie_header = "; ".join(f"{c['name']}={c['value']}" for c in matching_cookies)
+            if "Cookie" in headers:
+                headers["Cookie"] = headers["Cookie"] + "; " + cookie_header
+            else:
+                headers["Cookie"] = cookie_header
+

Evidence

PR Compliance ID 10 requires comments to explain why, not what; the added comment labels directly
mirror the immediately-following code and do not provide rationale or constraints.

AGENTS.md
py/selenium/webdriver/common/api_request_context.py[434-442]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Some newly added comments narrate the code rather than explaining rationale.
## Issue Context
The compliance guideline asks for comments that explain intent/constraints/tradeoffs, relying on code structure and naming for the "what".
## Fix Focus Areas
- py/selenium/webdriver/common/api_request_context.py[434-442]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

ⓘ The new review experience is currently in Beta. Learn more