Refactor docker healthchecks from Redis to RabbitMQ and add worker protections

[Copilot]

Summary

Ports healthcheck and worker protection features from PR #1024 to use RabbitMQ (actual broker) instead of Redis, while maintaining debugpy configuration from main.

List of Changes

• Added RabbitMQ healthcheck scripts using kombu.Connection for broker connectivity testing
• Installed procps in Dockerfiles for process monitoring via pgrep
• Added worker protection flags: --max-tasks-per-child=50, --max-memory-per-child=4000000
• Configured Docker healthchecks for celeryworker, celerybeat, and flower services
• Added autoheal service for automatic container restart on health failure
• Fixed shell injection vulnerability by accessing CELERY_BROKER_URL within Python context
• Replaced f-strings with .format() for Python < 3.6 compatibility

Related Issues

Relates to #1024

Detailed Description

PR #1024 implemented healthchecks using redis-cli to test Redis connectivity, but this system uses RabbitMQ as the Celery broker. This PR adapts those changes:

Healthcheck Implementation:

# Uses kombu to test actual RabbitMQ connection
from kombu import Connection
broker_url = os.environ.get('CELERY_BROKER_URL', 'amqp://...')
conn = Connection(broker_url)
conn.ensure_connection(max_retries=1, timeout=3)

Debugpy Integration:
Maintained main's DEBUGGER env var approach (port 5679) rather than PR #1024's CELERY_DEBUG (port 5678).

Security Fix:
Original PR passed CELERY_BROKER_URL to Python via shell variable expansion. Now accessed directly from os.environ within Python to prevent injection attacks.

Autoheal Configuration:
Monitors container health every 10s, restarts unhealthy containers after 3 consecutive failures (90s total).

How to Test the Changes

Verify healthcheck:

docker compose build celeryworker
docker compose up -d celeryworker
# Wait 60s for start_period, check status
docker ps | grep celeryworker  # Should show "healthy"

Test unhealthy detection:

docker exec <container> pkill -STOP -f "celery.*worker"
# Wait 90s for 3 retries
docker ps | grep celeryworker  # Should show "unhealthy"
# With autoheal, container restarts automatically

Test debugpy:

DEBUGGER=1 docker compose up celeryworker
# Attach debugger to localhost:5679

Screenshots

N/A - Infrastructure change

Deployment Notes

Docker Compose deployments:
Autoheal monitors containers with autoheal=true label. No configuration changes required beyond deploying updated compose files.

Kubernetes/Swarm (future):
Healthcheck scripts are orchestration-agnostic. Use as liveness probes directly.

Tuning parameters:

• --max-tasks-per-child and --max-memory-per-child in worker start scripts
• Healthcheck intervals/timeouts in docker-compose files

Checklist

• I have tested these changes appropriately.
• I have added and/or modified relevant tests.
• I updated relevant documentation or comments.
• I have verified that this PR follows the project's coding standards.
• Any dependent changes have already been merged to main.

Original prompt

bring this branch up-to-date with main (use the debugpy approach in main) and then refactor the healthchecks to be for our new broker, rabbitmq instead of redis
#1024

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[coderabbitai]

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[netlify]

✅ Deploy Preview for antenna-preview canceled.

Name	Link
🔨 Latest commit	64deb28
🔍 Latest deploy log	https://app.netlify.com/projects/antenna-preview/deploys/6916ca666ceb520008eed451