celery-healthcheck http service

Celery-Healthcheck HTTP Service.md

For a context where health checks are only available via http, e.g. render.com:

• deploy celery as a web service
• add healthchecks that trigger / url
  • celery-healthcheck.py will handle the request
  • perform a command that equals to celery inspect ping -d celery@$HOSTNAME
  • Returns 200 or 500 depending on a result

TODO: Contribute it to celery, e.g. something like running celery with: celery --app app worker --loglevel INFO --concurrency 4 --with-http-healthcheck 8080

celery-healthcheck.py

"""
Simple HTTP server that returns 200 OK if Celery is up and running.
"""

import logging
import os
from http.server import BaseHTTPRequestHandler, HTTPServer

from celery import Celery

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

HOSTNAME = f"celery@{os.environ.get('HOSTNAME')}"

app = Celery("healthcheck")

app.conf.update(
    broker_url=os.environ.get("CELERY_BROKER_URL"),
    result_backend=os.environ.get("CELERY_BROKER_URL"),
)


class HealthcheckHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        pong = app.control.inspect().ping(destination=[HOSTNAME])
        if pong:
            logger.info(f"ping: {pong}")
            self.send_response(200)
        else:
            logger.error("celery ping failed")
            self.send_response(500)
        self.end_headers()


def run(server_class=HTTPServer, handler_class=HealthcheckHandler, port=8080) -> None:
    server_address = ("", port)
    httpd = server_class(server_address, handler_class)
    logger.info(f"Starting {HOSTNAME} healthcheck httpd server on port {port}")
    httpd.serve_forever()


if __name__ == "__main__":
    run()


# Run it with: `python celery-healthcheck.py`

deploy-celery.sh

# Start celery-healthcheck http service in background
python celery-healthcheck.py &
# Start celery worker as usually
celery --app app worker --loglevel INFO  --concurrency 4