feat(e2e): add Playwright smoke tests for Heroku review apps#275
Open
feat(e2e): add Playwright smoke tests for Heroku review apps#275
Conversation
Adds test infrastructure to run E2E tests against deployed Heroku review apps, verifying code-server loads with the Debrief extension: - test-heroku-smoke.spec.ts: 5 smoke tests (HTTP check, workbench, activity bar, sample files, evidence screenshot) with generous timeouts for Heroku cold-starts - heroku-e2e.sh: convenience script with cold-start wait loop - heroku-e2e.yml: GitHub Actions workflow (manual trigger with URL) https://claude.ai/code/session_015qorofjGTWZ9LuimGaYGTE
… run - .dockerignore: whitelist apps/vscode/*.vsix so the code-server Dockerfile can COPY the pre-built extension - e2e.yml: skip test-heroku-* specs (they require a remote Heroku URL) https://claude.ai/code/session_015qorofjGTWZ9LuimGaYGTE
--ignore-pattern is not a valid Playwright flag; use --grep-invert to filter out tests with "Heroku" in their describe block name. https://claude.ai/code/session_015qorofjGTWZ9LuimGaYGTE
The codercom/code-server base image may set HOME to a non-root directory, causing uv installed to ~/.local/bin to be unfindable. Use UV_INSTALL_DIR to place it in /usr/local/bin instead. https://claude.ai/code/session_015qorofjGTWZ9LuimGaYGTE
…lectors Three fixes for E2E CI failures: 1. docker/code-server/Dockerfile: install the Debrief VSIX as the `coder` user, not root. code-server stores extensions under ~/.local/share/code-server/extensions — installing as root puts them in /root/... which the coder user cannot see. 2. Dockerfile.preview: apply the same UV_INSTALL_DIR fix so uv is reliably on PATH regardless of $HOME. 3. Smoke tests: add broader CSS selectors for the Debrief activity-bar icon. VS Code ≥1.93 moved activity-bar items into the sidebar header, changing the DOM hierarchy. The new selectors cover .action-item and [role="tab"] patterns in addition to the original .activitybar and .composite.bar selectors. https://claude.ai/code/session_015qorofjGTWZ9LuimGaYGTE
Root cause: the Docker code-server image had no user settings, so VS Code showed the workspace trust dialog on startup. The test tried to dismiss it within 5s, but in CI cold-start the dialog renders later. Without dismissal, extensions cannot activate and the Debrief activity bar icon never appears. Fix: pre-seed code-server User settings in the Dockerfile with workspace trust disabled and welcome tab suppressed — matching what global-setup.ts writes for locally-started code-server. Also adds diagnostic output to S02 on failure: captures a screenshot and dumps all role="tab" elements with their IDs and aria-labels. If the selectors still miss, the CI artifact will show exactly what the DOM contains. https://claude.ai/code/session_015qorofjGTWZ9LuimGaYGTE
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds test infrastructure to run E2E tests against deployed Heroku
review apps, verifying code-server loads with the Debrief extension:
activity bar, sample files, evidence screenshot) with generous
timeouts for Heroku cold-starts
https://claude.ai/code/session_015qorofjGTWZ9LuimGaYGTE