refactor: GA ubuntu-slim

[ainsleyclark]

Added detailed research document analyzing the feasibility and cost
savings of migrating WebKit workflow jobs to GitHub Actions ubuntu-slim
runners.

Key findings:

• ubuntu-slim runners are ~50% cheaper but have 15-minute timeout
• Identified 8 jobs suitable for migration (Phase 1)
• Estimated savings: $19-23/month per private repository
• Organization-wide potential: $2,000-14,000/year

Recommendations:

• Proceed with Phase 1 migration for low-risk jobs
• Do NOT migrate build/test jobs due to timeout limitations
• Requires thorough testing before rollout

Document includes:

• Technical specifications of ubuntu-slim runners
• Detailed job-by-job compatibility analysis
• Cost projections and ROI calculations
• Phased migration strategy with testing plan
• Risk assessment and rollback procedures

[claude]

Review summary

• Overall score: 8/10
• Critical issues: 0
• Warnings: 2
• Suggestions: 2
• Recommendation: ⚠️ Approve with comments

This PR implements a sensible cost optimisation by migrating lightweight GitHub Actions jobs to ubuntu-slim runners. The changes are straightforward and well-scoped to jobs with minimal runtime requirements. However, there are important considerations around testing, monitoring, and documentation that should be addressed.

Critical issues 🔴

None

Warnings 🟡

1. Missing validation of ubuntu-slim runner availability (release.yaml.tmpl:394, 421)

The notify-success and notify-failure jobs use ubuntu-slim runners but run conditionally based on deployment job success/failure. If ubuntu-slim runners are unavailable or queued, notification failures could mask the actual deployment status.

Impact: Users might not receive critical deployment notifications if slim runners are unavailable.

Recommendation: Consider keeping notification jobs on ubuntu-latest to ensure maximum reliability for critical alerts, or add fallback logic.

Example fallback pattern

notify-success:
  runs-on: ubuntu-latest  # Keep critical notifications on standard runners
  # ... rest of job

2. No testing strategy mentioned in PR description

The PR description mentions "Requires thorough testing before rollout" and includes detailed implementation and testing plans in the commit messages, but there's no evidence of actual testing performed before this PR was opened.

Impact: Without real-world validation, edge cases or compatibility issues with ubuntu-slim runners might only surface in production.

Recommendation:

• Test these changes in a staging environment or branch first
• Document test results in the PR description
• Consider a phased rollout (e.g., migrate 2-3 jobs first, monitor for a week, then migrate the rest)

Suggestions 🟢

1. Consider documenting the 15-minute timeout limitation

The commit messages mention that ubuntu-slim runners have a 15-minute timeout, but this isn't documented in the templates or in user-facing documentation. Future maintainers might not be aware of this constraint when adding new jobs.

Suggestion: Add a comment in the template files explaining the runner choice:

# Uses ubuntu-slim for cost savings (~50% cheaper)
# Note: ubuntu-slim has 15-minute timeout - only use for jobs completing in <5 minutes
jobs:
  setup-webkit:
    runs-on: ubuntu-slim

2. Job timing validation could prevent future regressions

Since the safety margin is based on jobs completing in under 1 minute with a 14+ minute buffer, adding workflow timing assertions could prevent accidental migration of longer-running jobs.

Suggestion: Consider adding a post-deployment monitoring step or workflow that alerts if any ubuntu-slim job exceeds a threshold (e.g., 5 minutes).

---

Scoring guide (1-10):

• 9-10: Production ready, excellent quality
• 7-8: Minor improvements needed, safe to merge
• 5-6: Notable issues but functional
• 3-4: Significant problems requiring fixes
• 1-2: Critical issues, do not merge

Score justification: The code changes are minimal, focused, and technically sound. The selection of jobs to migrate is well-reasoned based on actual runtime data. However, the lack of documented testing and potential reliability concerns for notification jobs prevent a higher score. This is a solid improvement that would benefit from addressing the monitoring and documentation gaps.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.47%. Comparing base (7f6b060) to head (213a2a0).
⚠️ Report is 432 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #355      +/-   ##
==========================================
+ Coverage   64.59%   69.47%   +4.87%     
==========================================
  Files         154      184      +30     
  Lines        6064     7279    +1215     
==========================================
+ Hits         3917     5057    +1140     
+ Misses       2064     2025      -39     
- Partials       83      197     +114     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.