My dash app uses background callbacks with Celery and Redis for long-running requests. It is deployed in a k8s cluster.
Versions
- Python: 3.10
- Dash: 2.8
- Plotly: 5.13.0
- Flask: 2.2.3
- gunicorn: 20.1.0
Issue
I’m getting 204s intermittently when triggering the background callback. Sometimes the callback and caching work. Sometimes the dash/_dash-update-component endpoint just returns 204s, and nothing on the frontend gets updated.
Below is the gunicorn log
The value in Redis shows that the corresponding task is not registered. However, I can see that the task is listed in the output of Celery command.
basic code structure
app.py
redis_conn_str = "..."
app_celery = Celery(__name__, broker=redis_conn_str, backend=redis_conn_str)
background_callback_manager = CeleryManager(app_celery, expire=1200)
app_server = Flask(__name__)
app = Dash(
__name__,
use_pages=True,
suppress_callback_exceptions=True,
server=app_server,
background_callback_manager=background_callback_manager,
)
run_server.sh
nohup celery -A app.app_celery worker --loglevel=INFO -c 2 | tee /dev/null &
gunicorn --bind=0.0.0.0:8000 --workers=2 --worker-class=gthread --threads=20 app:app_server
callbacks.py
...
register_page(__name__, path="/dashboard")
layout = html.Div(...)
@callback(
Output("output1", "data"),
Output("output2", "children"),
[
Input("button", "n_clicks"),
State("date-picker-range", "start_date"),
State("date-picker-range", "end_date"),
State("start1", "value"),
State("start1", "value"),
State("start1", "value"),
... # 12 states in total
],
prevent_initial_call=True,
background=True,
cache_args_to_ignore=[0],
interval=9000,
)
def run_sql_query_callback(...):
...
return dcc.send_data_frame(res.to_csv, "result.csv"), None
I’ve tried everything I can to debug, but it doesn’t work.
Much appreciated!