diff --git a/docs/product/snapshots/index.mdx b/docs/product/snapshots/index.mdx new file mode 100644 index 00000000000000..25128784fc1348 --- /dev/null +++ b/docs/product/snapshots/index.mdx @@ -0,0 +1,33 @@ +--- +title: Snapshots +sidebar_order: 138 +description: Detect visual changes by comparing PNG snapshots against a baseline on every pull request. +new: true +--- + + + +Snapshots catches visual regressions before they reach users. Your CI pipeline generates PNG screenshots, uploads them to Sentry, and Sentry compares each image against a baseline from your main branch using pixel-level diffing. + +When a snapshot changes, Sentry posts a failing GitHub Check on your pull request. You review the diff in Sentry's comparison viewer and approve or reject the changes. + +TODO: some sort of image + +## How It Works + +1. **Generate snapshots** — Your CI job produces PNG screenshots however you like: Playwright, Paparazzi, or any tool that outputs PNGs. +2. **Upload to Sentry** — `sentry-cli` uploads the snapshot directory along with Git metadata so Sentry can associate each upload with a commit. +3. **Sentry diffs against the baseline** — Sentry compares uploaded snapshots against snapshots corresponding to the upload's `base_sha`. With Sentry, you do not have to manage "Golden" images and everything is determined via git metadata. +4. **Results post to your PR** — Snapshot results post to your PR\*. You can configure settings to control status check and comment behavior. +5. **Review and approve** — You can view snapshot results on the Sentry UI. If the Snapshot "failed" you can approve it from either the PR or the Sentry UI. + +\*Note: Snapshots EA only supports Github + +## Prerequisites + +- **Early Adopter access** — Enable the Early Adopter toggle in your [organization settings](/organization/early-adopter-features/). +- **Auth token** — A Sentry auth token with `project:write` scope (personal token) or `org:ci` scope (org-level token). +- **sentry-cli >= 3.3.4** — The `build snapshots` command requires version 3.3.4 or later. +- **GitHub integration** — Snapshots can be used with any VCS provider, but only Github is supported for status check integration and PR comments. Install the [Sentry GitHub App](/organization/integrations/source-code-mgmt/github/) and grant it access to your repository for PR integration. + + diff --git a/docs/product/snapshots/reviewing-snapshots/index.mdx b/docs/product/snapshots/reviewing-snapshots/index.mdx new file mode 100644 index 00000000000000..94e7323523562e --- /dev/null +++ b/docs/product/snapshots/reviewing-snapshots/index.mdx @@ -0,0 +1,44 @@ +--- +title: Reviewing Snapshots +sidebar_order: 20 +description: Review snapshot diffs on your pull request and approve changes. +--- + + + +## Status Checks + +After you upload snapshots from a pull request branch, a **Snapshot Testing** GitHub Check appears on the pull request. If no snapshots changed, the check passes. If any snapshots were added, removed, or modified, the check fails and requires approval. + +IMAGE: GitHub Check showing Snapshot Testing results on a pull request + +## Comparison Viewer + +Click **Details** on the GitHub Check to open Sentry's comparison viewer. The viewer has two panels: + +- **Sidebar** — Lists all snapshots grouped by status: modified, added, removed, renamed, and unchanged. Click a snapshot to view it. +- **Main panel** — Displays the selected snapshot's diff with three viewing modes: + - **Split** — Side-by-side view of base and current branch with a diff overlay highlighting changed pixels. + - **Wipe** — Drag a slider across the image to compare base and current branch. + - **Onion** — Overlay both images with an adjustable opacity slider. + +IMAGE: Sentry Snapshot comparison viewer showing a modified image with split diff + +## Approve Changes + +There are two ways to approve snapshot changes: + +- Click the **Approve** button on the GitHub Check Run. +- Click **Approve** in the Sentry comparison viewer. + +Approval resolves the failing check. If new changes are pushed to the PR, the check resets and requires approval again. + +## Settings + +Configure snapshot behavior in **Project Settings > Mobile Builds > Snapshots**. + +| Setting | Default | Description | +| ------------------------- | :-----: | ----------------------------------------------------- | +| Status Checks Enabled | On | Post GitHub Check Runs for snapshot comparisons. | +| Fail on Added Snapshots | Off | Require approval when new snapshots are added. | +| Fail on Removed Snapshots | On | Require approval when existing snapshots are removed. | diff --git a/docs/product/snapshots/uploading-snapshots/index.mdx b/docs/product/snapshots/uploading-snapshots/index.mdx new file mode 100644 index 00000000000000..ccd16763f87529 --- /dev/null +++ b/docs/product/snapshots/uploading-snapshots/index.mdx @@ -0,0 +1,159 @@ +--- +title: Uploading Snapshots +sidebar_order: 10 +description: Structure your snapshot directory and upload from CI with sentry-cli. +--- + + + +Snapshots can be used with any frontend screenshot framework. We have examples in different platform documentaiton pages (TODO: EXAMPLES). This page covers the general upload structure and metadata schema. + +## Upload command + +You upload snapshots with the `sentry-cli build snapshots` command, which takes a directory of images and optional JSON metadata files. + +```bash +sentry-cli build snapshots \ + --auth-token "$SENTRY_SNAPSHOTS_AUTH_TOKEN" \ + --app-id \ + --project \ + --head-sha \ + --vcs-provider github \ + --head-repo-name +``` + +You can reference our [example workflow](#upload-with-ci) and [full flag reference](#sentry-cli-build-snapshots-reference) below. + +### Upload Structure + +Each snapshot consists of two files: a `.png` image and an optional `.json` metadata file with the same base name and path. You can organize snapshots into subdirectories, but is not required and will not affect the UX. + +``` +snapshots/ +├── homepage.png +├── homepage.json +├── settings/ +│ ├── profile.png +│ ├── profile.json +│ ├── billing.png +│ └── billing.json +``` + +Filenames are the identity key for each snapshot. Keep them stable across runs so Sentry can accurately diff head against base. + +### JSON Metadata + +You can optionally include a JSON file to add metadata to each image: + +```json +{ + "display_name": "Billing Page", + "group": "Settings" +} +``` + +| Field | Type | Description | +| -------------- | ------ | ---------------------------------------------------------- | +| `display_name` | string | Human-readable label shown in the comparison viewer. | +| `group` | string | Groups related snapshots in the comparison viewer sidebar. | + +TODO: IMAGE OF DISPLAY NAME AND GROUP + +## Upload With CI + +To see the workflow Sentry uses on its own codebase, view the [frontend snapshots workflow](https://github.com/getsentry/sentry/blob/master/.github/workflows/frontend-snapshots.yml). + +Below is an example GitHub Actions workflow you can use as a starting point. Replace the placeholder values with your own snapshot generation command, app ID, and Sentry project slug. + +```yml {filename:.github/workflows/snapshots.yml} +name: Snapshots + +on: + push: + branches: [main] + pull_request: + +env: + SNAPSHOT_OUTPUT_DIR: .artifacts/snapshots + +jobs: + snapshots: + runs-on: ubuntu-24.04 + timeout-minutes: 20 + steps: + - uses: actions/checkout@v4 + with: + # Use PR head SHA, not the merge commit — avoids phantom diffs + ref: ${{ github.event.pull_request.head.sha || github.sha }} + + # ------------------------------------------------------- + # YOUR SNAPSHOT GENERATION STEP(S) HERE + # ------------------------------------------------------- + # Generate PNG + JSON pairs into $SNAPSHOT_OUTPUT_DIR. + # Examples: + # - Playwright screenshots + # - iOS/Android screenshots + # - Laravel Dusk screenshots + # - Any tool that produces PNGs + # ------------------------------------------------------- + - name: Generate snapshots + run: + + - name: Install sentry-cli + if: ${{ !cancelled() }} + run: curl -sL https://sentry.io/get-cli/ | SENTRY_CLI_VERSION=3.3.4 sh + + - name: Upload snapshots + if: ${{ !cancelled() }} + env: + SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }} + run: | + ARGS=( + --auth-token "$SENTRY_AUTH_TOKEN" + build snapshots "${{ env.SNAPSHOT_OUTPUT_DIR }}" + --app-id + --project + --head-sha "${{ github.event.pull_request.head.sha || github.sha }}" + --vcs-provider github + --head-repo-name "${{ github.repository }}" + ) + + if [[ "${{ github.event_name }}" == "pull_request" ]]; then + ARGS+=( + --base-sha "${{ github.event.pull_request.base.sha }}" + --base-repo-name "${{ github.repository }}" + --head-ref "${{ github.head_ref }}" + --base-ref "${{ github.base_ref }}" + --pr-number "${{ github.event.number }}" + ) + fi + + sentry-cli "${ARGS[@]}" +``` + +## `sentry-cli build snapshots` Reference + +```bash +sentry-cli build snapshots [OPTIONS] --app-id +``` + +| Flag | Description | +| ------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `` | Directory containing `.png` images (and optional `.json`). | +| `--app-id ` | Identifier for the app or bundle (e.g., `web-frontend`, `ios-app`). Must be consistent across uploads — Sentry uses it to match head and base snapshots. Max 255 characters. | +| `--auth-token ` | Sentry auth token with `project:write` or `org:ci` scope. Can also be set via `SENTRY_AUTH_TOKEN`. | +| `-o`, `--org ` | Sentry organization slug. Can also be set via `SENTRY_ORG`. | +| `-p`, `--project ` | Sentry project slug. Can also be set via `SENTRY_PROJECT`. | +| `--head-sha ` | 40-character commit SHA of the snapshot build. Auto-detected in CI. | +| `--base-sha ` | Base branch commit SHA for comparison (PR only). Auto-detected in CI. | +| `--vcs-provider ` | VCS provider, e.g., `github`. Auto-detected from git remote. | +| `--head-repo-name ` | Repository in `owner/repo` format. Auto-detected from git remote. | +| `--base-repo-name ` | Base repository in `owner/repo` format (PR only, for forks). Auto-detected in CI. | +| `--head-ref ` | Head branch name. Auto-detected in CI. | +| `--base-ref ` | Base branch name (PR only). Auto-detected in CI. | +| `--pr-number ` | Pull request number (PR only). Auto-detected in GitHub Actions. | +| `--diff-threshold ` | Float between 0.0 and 1.0. Sentry only reports images as changed if the percentage of changed pixels exceeds this value. For example, `0.01` ignores changes under 1%. | +| `--no-git-metadata` | Skip automatic git metadata detection. | +| `--force-git-metadata` | Force git metadata collection even outside CI. | +| `--log-level ` | Logging verbosity: `debug`, `info`, `warn`, or `error`. | +| `--url ` | Sentry instance URL. Defaults to `https://sentry.io`. |