TST-25: Add opt-in headed manual-audit Playwright pack

[Chris0Jeky]

Summary

Closes #369

• Add tests/e2e/manual-audit.spec.ts covering the core Home -> Inbox/Capture -> Review -> Board loop with numbered screenshots at each milestone, plus advanced checks (command palette, capture hotkey, board/filter lifecycle) and an opt-in live LLM provider probe
• Update test:e2e:audit:headed npm script to use the new dedicated audit spec
• Add docs/testing/MANUAL_AUDIT_PACK.md documenting when to use vs stakeholder demo recorder vs default smoke

Design decisions

• Headed mode with 250ms slow motion for operator visibility
• Screenshots captured at every meaningful step (18 total across all tests)
• Live LLM probe gated behind TASKDECK_RUN_LIVE_LLM_TESTS=1 env var
• NOT added to CI — local/operator-focused only
• Uses existing registerAndAttachSession auth pattern — no demo seed required

Test plan

• Run npm run test:e2e:audit:headed locally with backend running
• Verify headed browser opens with visible slow motion
• Verify screenshots appear in test output directory
• Verify live LLM tests skip by default (no env var)
• Verify TASKDECK_RUN_LIVE_LLM_TESTS=1 npm run test:e2e:audit:headed runs LLM probe (requires configured provider)
• Verify existing npm run test:e2e is unaffected

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

[Chris0Jeky]

Self-review findings

Selectors verified against existing specs

All test selectors in manual-audit.spec.ts are copied from or match patterns in the existing capture-loop.spec.ts and smoke.spec.ts — specifically:

• [data-testid="inbox-item"] (capture-loop.spec.ts:31)
• .td-inbox-detail__actions button (capture-loop.spec.ts:35)
• [data-card-id] (smoke.spec.ts:129)
• #proposal-{id} (capture-loop.spec.ts:54)
• getByRole('button', { name: 'Approve for board' }) (capture-loop.spec.ts:55)
• getByRole('dialog', { name: 'Command palette' }) (smoke.spec.ts:189)
• getByRole('dialog', { name: 'Capture item' }) (smoke.spec.ts:209)
• .td-message / .td-message-role / .td-message-content (live-llm.spec.ts:32-34)

Playwright config change is non-breaking

Only added testIgnore: ['**/manual-audit.spec.ts'] to exclude the audit pack from the default playwright test run. No other config changes. Existing tests unaffected.

npm script works correctly

playwright test tests/e2e/manual-audit.spec.ts --headed --reporter=line explicitly targets the file, which bypasses testIgnore per Playwright docs.

Screenshots at useful points

18 screenshots covering: Home landing, Inbox with capture, Inbox detail pre-triage, Review proposal, Review approved/applied, Board with card, Card provenance, Command palette, Capture modal, Filter panel, LLM states.

Live-provider opt-in properly gated

Uses parseTrueishEnv(process.env.TASKDECK_RUN_LIVE_LLM_TESTS) — same pattern as live-llm.spec.ts. Skipped by default.

No issues found

All selectors realistic, config non-breaking, CI excluded, screenshots useful.

[Chris0Jeky]

Fresh Adversarial Review

Critical Issues

1. testIgnore blocks the npm script -- the audit pack cannot run

The PR adds testIgnore: ['**/manual-audit.spec.ts'] to playwright.config.ts and claims the npm run test:e2e:audit:headed script "explicitly targets the file, bypassing testIgnore." This claim is wrong. Playwright's testIgnore applies during test collection even when a file is passed as a CLI argument. I confirmed by checking out the branch and running:

npx playwright test tests/e2e/manual-audit.spec.ts --list
# -> Error: No tests found. Total: 0 tests in 0 files

This means npm run test:e2e:audit:headed will always fail with "No tests found." The entire audit pack is unrunnable as shipped.

Fix applied: Removed testIgnore from playwright.config.ts. Instead, added TASKDECK_RUN_AUDIT env var gate (via test.skip) to each test.describe block, matching the existing pattern used by stakeholder-demo.spec.ts and live-llm.spec.ts. The npm script sets the env var automatically via a cross-platform node -e wrapper.

Minor Issues

2. CRLF line-ending pollution in playwright.config.ts

The diff shows ~40 lines changed in playwright.config.ts, but the only semantic change was adding testIgnore. All other changes are LF->CRLF line-ending conversions on lines 2-18 and 31-36. On main, line 1 has CRLF but lines 2+ have LF. The branch converted many LF lines to CRLF, adding noise to the diff and making the line-ending state inconsistent.

Fix applied: Reverted playwright.config.ts to match main exactly (since testIgnore is no longer needed).

3. trace: 'retain-on-failure' in test.use() is redundant

The global playwright.config.ts already sets trace: 'retain-on-failure' at line 57. The test.use({ trace: 'retain-on-failure' }) in the spec just restates the default. Not harmful, but misleading -- the doc claims it's a "test-level override" when it's actually a no-op.

4. Doc inaccuracy in MANUAL_AUDIT_PACK.md

The "CI Exclusion" section stated the file is excluded via testIgnore which "bypasses" on explicit targeting. This was both incorrect about the mechanism and wrong about the behavior. Updated to reflect the env var gate approach.

Observations

• All data-testid selectors, CSS class selectors, ARIA roles, placeholder texts, and button names used in the test do exist in the corresponding Vue components. No phantom selectors.
• The core loop test faithfully mirrors the proven pattern from capture-loop.spec.ts with screenshots added at each step. Good reuse of registerAndAttachSession, createBoardWithColumn, createCaptureItem, waitForProposalCreated, and waitForCardWithTitle helpers.
• The parseTrueishEnv import from ../../scripts/demo-shared.mjs is the established pattern for env var gating (used by stakeholder-demo.spec.ts and live-llm.spec.ts).
• Screenshot naming is logical and sequential (01-18).
• The LLM probe section correctly double-gates behind both TASKDECK_RUN_AUDIT and TASKDECK_RUN_LIVE_LLM_TESTS.
• No test state leakage -- each test uses registerAndAttachSession with fresh users, and board/capture data is unique per run via timestamp seeds.

Verdict

One critical bug (the pack literally cannot run), one cosmetic diff-noise issue, and a minor doc inaccuracy. All three are fixed in the follow-up commit pushed to this branch.