Dash Background Callbacks on Render - how to set up celery background workers

Dash Python
2

I think I have got it working, below I have put my steps in case anyone else struggles with this in future. I can’t be sure that all of these settings (eg my specific Python version, and all of the requirements lines I listed) are absolutely necessary but they are what I have and it seems to be working:

  1. Preparing your Dash app and Github repository: Add the background worker lines to your Dash app, as shown in “Example 1” here. Background Callbacks | Dash for Python Documentation | Plotly
    I also added some additional lines to my requirements.txt to make sure the dependencies get installed at Render in the steps below. I added:
    dash[celery]==2.11.1
    celery==5.3.1
    diskcache==5.6.1
    redis==4.6.0
    gunicorn==20.1.0
    Upload this app to a Github repository (it can be public or private).
  2. Set up a new Redis service on Render, I believe I left all options at default other than “maxmemory policy” which I changed to “noeviction” as it was recommended for queue services
  3. Set up a new Background Worker service on Render, with the below settings. I believe this is the process that executes the code that is within the Background Callback:
    Repository: the URL of your dash app on github from step 1
    Build Command : pip install --upgrade pip setuptools wheel && pip install -r requirements.txt
    Start Command: celery -A app:celery_app worker --loglevel=INFO --concurrency=2
    Environment variable: PYTHON_VERSION = 3.11.4
    Environment variable: REDIS_URL = Use the “internal redis url” found within the info tab of the Redis service you set up in step 2
  4. Set up your a new Web Service on render, with the below settings. I believe this is the process that executes the code that is not in the Background Callback:
    Repository: the URL of your dash app on github from step 1
    Build Command : pip install --upgrade pip setuptools wheel && pip install -r requirements.txt
    Start Command: gunicorn app:server
    Environment variable: PYTHON_VERSION = 3.11.4
    Environment variable: REDIS_URL = Use the “internal redis url” found within the info tab of the Redis service you set up in step 2

One of the things that was causing me errors initially was my use of Flask Limiter as a rate limiter, it seems to stop my app from working when I change my standard dash callback to a Background Callback, so I had to disable my Flask Limiter lines. I will make a new topic about this as it would be good to be able to set some kind of rate limit so that the app/queue can’t be overloaded. Edit - here Flask Rate Limiter does not seem compatible with Background Callbacks - any other options?

Many thanks again to Nicholas Kluge for sharing how he got the web service working in the first place :slight_smile:

3 Likes