feat: E2E cloud testing infrastructure for autonomous agents#2
Open
johnsonfamily1234 wants to merge 11 commits intomainfrom
Open
feat: E2E cloud testing infrastructure for autonomous agents#2johnsonfamily1234 wants to merge 11 commits intomainfrom
johnsonfamily1234 wants to merge 11 commits intomainfrom
Conversation
Create a reusable Playwright E2E testing workflow that deploys to Vercel preview, waits for readiness, and runs tests with Browserbase cloud execution support and local fallback. Key design decisions: - Vercel project/team IDs use repository variables (vars.*) instead of hardcoded values, so any project can configure its own - Test directory is configurable via E2E_TEST_DIR env var with tests/e2e default, overridable via workflow_dispatch input - 3-job structure: deploy-preview -> wait-for-deployment -> e2e-tests - Artifacts: JSON results, HTML report, failure screenshots/traces - PR comment with pass/fail summary and failure details - Concurrency group prevents duplicate runs per branch - Validation step fails fast if required vars are missing Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Create knowledge/testing/templates/ with portable fixtures for any project: - browserbase.ts: CDP fixture with session recording and local fallback - playwright.config.ts: Browserbase auto-detection, CI settings, reporters - package.json: minimal deps with test/agent/debug npm scripts - process-results.js: transforms Playwright JSON into agent-results.json - README.md: usage guide with quick start and customization points Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Port three testing knowledge docs from my-hq, replacing hardcoded project IDs, team IDs, and repo references with placeholder variables. All C:/my-hq paths replaced with relative paths, installer/-prefixed paths updated to tests/e2e, and hq-installer-specific content generalized for any Vercel-deployed project. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add ready-to-use Playwright testing templates covering the three most common application types: Next.js web apps, REST API endpoints, and CLI tools with browser-based OAuth flows. Each template includes setup, playwright.config.ts example, common patterns, assertions, and cleanup. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Introduces the test-plan skill for the qa-tester worker - a structured interview and analysis process that ensures E2E tests are grounded in real user behavior rather than arbitrary UI coverage. Key capabilities: - Structured interview protocol covering critical journeys, revenue paths, fragile flows, edge cases, and minimum viable user journeys - Automated analysis fallback when humans are unavailable (crawl UI structure, analyze source code, infer priorities) - Hybrid mode combining automated analysis with validation interviews - Machine-parseable JSON output consumed by write-test and run-tests - Coverage matrix revealing blind spots across auth, navigation, forms, payments, API, mobile, and accessibility - Clear critical-path vs coverage test classification - Template mapping to knowledge/testing/templates/ patterns Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add comprehensive CI E2E Verification section to ralph-loop-pattern.md that requires E2E workflow pass before task completion. Includes: - Full push/trigger/poll/parse workflow with gh CLI commands - agent-results.json download and failure parsing - 15-minute timeout with BLOCKED status handling - Failure handling with checkpoint logging - Emergency skip process with audit requirements - Quick reference commands for all CI operations Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
e2eTestsfield, Ralph loop blocks task completion without green CI, branch protection docs included/metrics --testsfor coverage tracking, agent-friendly result parser with jq queriesArchitecture
13 user stories, 31 files, all knowledge/worker/workflow additions — no existing code modified.
Test plan
pathsmatching app code/run qa-tester test-planagainst an existing project to validate discovery flow/run frontend-dev e2e-testing writeto generate a test from a test planvalidate-prd.ps1catches PRDs missinge2eTestsfield🤖 Generated with Claude Code