Overview
I’m observing that even though a background callback has its results cached from the initial call and subsequent calls to the callback with the same input do return the cached results, the subsequent calls are still executing the background callback. The callback’s execution is cut short because the celery task is being terminated. This is an issue because my background callback is issuing the long running part of the code (which executes an API query on an external server) before it gets terminated. That query is left hanging and contributing to external server load.
Setup
Version:
- dash: 3.0.4
- celery: 5.5.3
- gunicorn: 22.0.0
- redis: 5.2.1
- redis-server:7.4.2
I’m running three services with docker compose (extraneous code removed)
version: "3.8"
services:
dash:
image: dash-base:latest
command: gunicorn -b 0.0.0.0 --worker-class gthread --workers 1 --thread 1 --reload app:server --capture-output --enable-stdio-inheritance
...
celery:
image: dash-base:latest
command: celery -A app.celery_app worker --loglevel=debug
...
redis:
image: localhost:5000/redis:7.4.2
command: redis-server
...
In the dash service app.py:
from celery import Celery
from dash import Dash, CeleryManager
redisport = os.environ.get("REDIS_PORT", 6379)
redishost = f"redis://redis:{redisport}"
celery_app = Celery(
__name__,
broker=redishost,
result_backend=redishost
)
celery_app.config_from_object("celeryconfig")
background_callback_manager = CeleryManager(
celery_app,
cache_by=[lambda: launch_uid],
expire=60*60*24
)
app = Dash(
__name__,
use_pages=True,
background_callback_manager=background_callback_manager
)
server = app.server
This is a multi-page app. in pages/test/test.py:
from components.pages.basepage import PageConfig
import dash
from dash import dcc, callback, html, Input, Output, State
import time
import dash_bootstrap_components as dbc
config = PageConfig(
module = __name__,
title = "Test Page",
path = "/test",
description = "Test Page"
)
dash.register_page(**config.asdict())
def layout():
return html.Div(
[
html.H4("test callback 1"),
dcc.Input(id="input-1"),
html.P(
"The first 4 times the button is clicked, it's slow. After that, cached values are used"
),
html.Div([html.P(id="paragraph1_id", children=["Button not clicked"])]),
dbc.Button(id="button1_id", children="Run Job!"),
dbc.Button(id="cancel_button1_id", children="Cancel Running Job!"),
]
@callback(
output=(Output("paragraph1_id", "children")),
inputs=dict(button=Input("button1_id", "n_clicks")),
state=dict(value=State("input-1", "value")),
background=True,
running=[(Output("test-cache", "data"), True, False)],
cancel=Input("cancel_button1_id", "n_clicks"),
cache_args_to_ignore=["button"],
config_prevent_initial_callbacks=True
)
def update_clicks(button, value):
print("in update_clicks", flush=True)
time.sleep(5.0)
print("slept for 5 seconds. updating", flush=True)
return [f"Result for {value}"]
I see the background callback registered on startup:
[tasks]
.background_callback_3c649ccd457cc22322a078d6d33b934075c53ae1dcad0dbdedd862aa9e7ed8c1
Enter “foo” into the “input-1” input. Click button to trigger the callback for first time:
[2025-08-26 19:47:22,196: INFO/MainProcess] Task background_callback_3c649ccd457cc22322a078d6d33b934075c53ae1dcad0dbdedd862aa9e7ed8c1[29fc8a42-75ee-4905-b4da-294eec49d10f] received
first and second print statements:
[2025-08-26 19:33:12,764: WARNING/ForkPoolWorker-1] in update_clicks
[2025-08-26 19:33:17,666: WARNING/ForkPoolWorker-1] slept for 5 seconds. updating
task success:
[2025-08-26 19:47:27,213: INFO/ForkPoolWorker-2] Task background_callback_3c649ccd457cc22322a078d6d33b934075c53ae1dcad0dbdedd862aa9e7ed8c1[29fc8a42-75ee-4905-b4da-294eec49d10f] succeeded in 5.014073019032367s: None
task key in redis:
get celery-task-meta-29fc8a42-75ee-4905-b4da-294eec49d10f
"{\"status\": \"SUCCESS\", \"result\": null, \"traceback\": null, \"children\": [], \"date_done\": \"2025-08-26T19:47:27.211646+00:00\", \"task_id\": \"29fc8a42-75ee-4905-b4da-294eec49d10f\"}"
result in redis:
get 1209be6a2a9bbe3d7fa91986de99884393a168453e6749ea76ff5c5f3f3dcc6a
"[\"Result for foo\"]"
Subsequent click, same exact input:
[2025-08-26 19:49:07,826: INFO/MainProcess] Task background_callback_3c649ccd457cc22322a078d6d33b934075c53ae1dcad0dbdedd862aa9e7ed8c1[52cf2527-34f2-42e6-b06a-757a9e930a83] received
Callback function body is executed (first print statement):
[2025-08-26 19:49:07,828: WARNING/ForkPoolWorker-2] in update_clicks
Revoke received for this task:
[2025-08-26 19:49:09,009: DEBUG/MainProcess] pidbox received method revoke(task_id='52cf2527-34f2-42e6-b06a-757a9e930a83', terminate=True, signal='SIGTERM') [reply_to:None ticket:None]
Terminating before the callback finishes:
[2025-08-26 19:49:09,009: INFO/MainProcess] Terminating 52cf2527-34f2-42e6-b06a-757a9e930a83 (15)
Another revoke:
[2025-08-26 19:49:09,010: DEBUG/MainProcess] pidbox received method revoke(task_id='52cf2527-34f2-42e6-b06a-757a9e930a83', terminate=True, signal='SIGTERM') [reply_to:None ticket:None]
redis task entry:
get celery-task-meta-52cf2527-34f2-42e6-b06a-757a9e930a83
"{\"status\": \"REVOKED\", \"result\": {\"exc_type\": \"TaskRevokedError\", \"exc_message\": [\"terminated\"], \"exc_module\": \"celery.exceptions\"}, \"traceback\": null, \"children\": [], \"date_done\": \"2025-08-26T19:49:09.009885+00:00\", \"task_id\": \"52cf2527-34f2-42e6-b06a-757a9e930a83\"}"
No new redis result entry was created. No errors on page.
This is a simple example with no consequences, but this happening with my main code has a significant impact.
According to this page
If the hashed key exists, then the function is not called and the cached result is returned. If not, the function is called and the result is stored in the dictionary using the associated key.
Why does this seemingly execute the function before finding the cached results?
Thanks in advance. I can provide more information