Skip to content

Conversation

@hlbmtc
Copy link
Contributor

@hlbmtc hlbmtc commented Jan 21, 2026

Summary by CodeRabbit

  • Chores
    • Updated PR preview deployment domain configuration with enhanced SSL certificate management for preview environments.

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Jan 21, 2026

📝 Walkthrough

Walkthrough

The PR updates the CI/CD preview workflow to replace the default Fly.io domain with a Cloudflare-managed preview domain, adds certificate registration via flyctl, and updates the corresponding URL configuration used by the deployment pipeline.

Changes

Cohort / File(s) Summary
PR Preview Workflow Configuration
.github/workflows/pr_preview.yml
Replaces default Fly.io domain (APP_NAME.fly.dev) with Cloudflare preview domain (APP_NAME-preview.mtcl.cc); adds flyctl certs add step for certificate registration (tolerant of pre-existing certs); updates PUBLIC_APP_URL to reference new APP_DOMAIN

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly related PRs

Suggested reviewers

  • ncarazon
  • elisescu

Poem

🐰 A hop through the clouds, from fly to the sky,
Where Cloudflare certs dance and domains fly by,
Preview previews now glow with a DNS so new,
With a bound and a flurry, the workflow's complete—whoohoo!

🚥 Pre-merge checks | ✅ 2
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: configuring Fly.io with a custom mtcl.cc domain for PR previews instead of the default Fly.io domain.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Comment @coderabbitai help to get the list of available commands and usage tips.

@hlbmtc hlbmtc marked this pull request as ready for review January 21, 2026 19:58
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In @.github/workflows/pr_preview.yml:
- Around line 292-299: The workflow currently swallows errors by using `flyctl
certs add "${APP_DOMAIN}" --app "${APP_NAME}" || true`; instead, query Fly for
existing certs using `flyctl certs list --json` and `jq` to check if APP_DOMAIN
already has a cert, skip adding if it exists, otherwise run `flyctl certs add
"${APP_DOMAIN}" --app "${APP_NAME}"` and allow its non-zero exit code to fail
the job; replace the `|| true` pattern with this JSON-based existence check and
proper exit propagation so auth/DNS/rate-limit errors are not ignored.
📜 Review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1e1e996 and 0fb1e10.

📒 Files selected for processing (1)
  • .github/workflows/pr_preview.yml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: Build Docker Image
  • GitHub Check: integration-tests
  • GitHub Check: Backend Checks

✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.

Comment on lines 292 to +299
# Define app URLs (used for secrets and outputs)
APP_DOMAIN="${APP_NAME}.fly.dev"
# Custom domain via Cloudflare Worker: {fly_app}-preview.mtcl.cc
APP_DOMAIN="${APP_NAME}-preview.mtcl.cc"
PUBLIC_APP_URL="https://${APP_DOMAIN}"
# Register custom domain with Fly.io (Cloudflare handles SSL)
flyctl certs add "${APP_DOMAIN}" --app "${APP_NAME}" || true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🌐 Web query:

flyctl certs add exit code behavior when certificate already exists

💡 Result:

Short answer: flyctl certs add fails with an error (prints "Error: Hostname already exists on app") and returns a non‑zero exit status rather than silently succeeding — you must remove or re-use the existing certificate (or target the correct app) before adding the same hostname. [1][2]

Sources:

  • Fly docs: certs add (usage/info). [1]
  • Fly community threads showing the "Hostname already exists on app" error when re-adding an existing cert. [2]

🌐 Web query:

flyctl certs add documentation exit codes

💡 Result:

Short answer: Fly's official docs for "fly certs add" do not document any CLI exit codes. The command page shows usage and flags but no exit-status information. [1]

If you need exit-code behaviour for scripting, check the flyctl source on GitHub (superfly/flyctl) or run the specific flyctl version and test cases (e.g., missing args, auth failure) to observe returned statuses — the docs don't list them. [2]

Sources:

  • flyctl certs add documentation. [1]
  • superfly/flyctl GitHub repository (source for CLI behavior). [2]

🌐 Web query:

flyctl certs list output format grep

💡 Result:

By default fly certs list prints a human-readable table. For reliable scripting/grepping use JSON output (-j) and jq; example:

  • Human-readable (quick grep):
    fly certs list -a myapp | grep example.com

  • Reliable JSON parsing:
    fly certs list -a myapp -j | jq -r '.[] | .hostname'

Notes:

  • The CLI supports -j/--json for machine-readable output. [1]
  • The API/GraphQL responses show certificates under app.certificates.nodes (useful when using the API). [2]

Sources: Fly docs — fly certs list; Fly docs — custom domain / certificates API. [1][2]


Don't swallow Fly cert registration failures.

flyctl certs add ... || true ignores real failures (auth/DNS/rate limits), potentially leaving a preview URL that deploys but never serves HTTPS. However, the suggested grep-based check is unreliable—Fly's documentation recommends JSON output with jq for scripting, not text parsing.

Use this approach instead:

🔧 Suggested fix
           # Register custom domain with Fly.io (Cloudflare handles SSL)
-          flyctl certs add "${APP_DOMAIN}" --app "${APP_NAME}" || true
+          # Check if cert already exists using JSON output (more reliable than grepping text)
+          if ! flyctl certs list --app "${APP_NAME}" -j | jq -e ".[] | select(.hostname == \"${APP_DOMAIN}\")" > /dev/null 2>&1; then
+            flyctl certs add "${APP_DOMAIN}" --app "${APP_NAME}"
+          fi

This uses JSON parsing instead of text grepping for robustness, and fails on genuine errors while safely skipping re-runs where the cert already exists.

🤖 Prompt for AI Agents
In @.github/workflows/pr_preview.yml around lines 292 - 299, The workflow
currently swallows errors by using `flyctl certs add "${APP_DOMAIN}" --app
"${APP_NAME}" || true`; instead, query Fly for existing certs using `flyctl
certs list --json` and `jq` to check if APP_DOMAIN already has a cert, skip
adding if it exists, otherwise run `flyctl certs add "${APP_DOMAIN}" --app
"${APP_NAME}"` and allow its non-zero exit code to fail the job; replace the `||
true` pattern with this JSON-based existence check and proper exit propagation
so auth/DNS/rate-limit errors are not ignored.

@github-actions
Copy link
Contributor

🚀 Preview Environment

Your preview environment is ready!

Resource Details
🌐 Preview URL https://metaculus-pr-4134-feat-preview-environment-custo-preview.mtcl.cc
📦 Docker Image ghcr.io/metaculus/metaculus:feat-preview-environment-custo-9c9562b
🗄️ PostgreSQL NeonDB branch preview/pr-4134-feat-preview-environment-custo
Redis Fly Redis mtc-redis-pr-4134-feat-preview-environment-custo

Details

  • Commit: 9c9562b1f0c9f455dec631be57f85fe91ccb3ff0
  • Branch: feat/preview-environment-custom-domain
  • Fly App: metaculus-pr-4134-feat-preview-environment-custo

ℹ️ Preview Environment Info

Isolation:

  • PostgreSQL and Redis are fully isolated from production
  • Each PR gets its own database branch and Redis instance
  • Changes pushed to this PR will trigger a new deployment

Limitations:

  • Background workers and cron jobs are not deployed in preview environments
  • If you need to test background jobs, use Heroku staging environments

Cleanup:

  • This preview will be automatically destroyed when the PR is closed

@hlbmtc hlbmtc marked this pull request as draft January 22, 2026 11:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants