Prevent re-entrant callbacks

Dash Python
1

Hi,

How do I detect and prevent re-entrant callbacks? Suppose I have a 1 second interval, but my callback takes 10 seconds. As far as I can tell, Dash keeps hammering away, and the thread count grows without bound.

If I could detect re-entrancy, it would be easy to raise PreventUpdate to skip any long-running task.

My best idea so far was to use a hidden div with child=0 and then a callback that updates it to some unique hash value (but does nothing if value is already non-zero). So now, other callbacks can use the value in that hidden div to identify the client, and some other application cache (per-process or using Redis, for example) could be used to track callback re-entrancy.

Because this idea seems fairly convoluted, and it seems Dash must be tracking user session anyway to make the callbacks work, I thought I’d ask if there is a better way.

4 Likes
2

As per usual I found part of the answer almost immediately after posting: example 4 on https://dash.plot.ly/sharing-data-between-callbacks The core idea is use a function as the app.layout so a custom layout can be served each time, and embed a random string in that layout to server as a session id.

However, it turn out aborting later callbacks (because they are re-entrant) breaks things. The example below run, and prints what you would expect (about every 10th call is not re-entrant) but the app never updates the text.

Any clever ideas on how to achieve this?

import threading
import time
import uuid

import dash
from dash.dependencies import Input, Output
import dash_html_components as html
import dash_core_components as dcc

app = dash.Dash(__name__)

@app.server.errorhandler(dash.exceptions.PreventUpdate)
def _handle_error(error):
    """Replace the default handler with one that does not print anything"""
    return '', 204

def serve_layout():
    session_id = str(uuid.uuid4())
    return html.Div([
    dcc.Interval(
        id='interval',
        interval=1*1000,
        n_intervals=0
    ),
    html.P(
        id='text',
        children='Loading...'
    ),
    html.Div(children=session_id, id='session-id', style={'display': 'none'})
])

app.layout = serve_layout

lock = threading.Lock()
d = set()
def acquire(id):
    with lock:
        if id in d:
            return False
        else:
            d.add(id)
            return True

def release(id):
    with lock:
        d.remove(id)

@app.callback(
    Output('text', 'children'),
    [Input('interval', 'n_intervals'), Input('session-id', 'children')]
)
def update(n, session_id):
    if acquire(session_id):
        try:
            print(f'{threading.current_thread()} {n} Not re-entrant! {session_id}')
            time.sleep(10.)
            print(f'---------------------- returning {n}')
            return n
        finally:
            release(session_id)
    else:
        print(f'{threading.current_thread()} {n} Re-entrant! {session_id}')
        raise dash.exceptions.PreventUpdate


if __name__ == '__main__':
    app.run_server(debug=False)
3

I’m not sure I follow what you’re asking… Can you start with outcome you’re looking for, what you’ve done so far to get there and what problems you’re experiencing.

This sends up a bunch of red flags for me. Given the type of solution your looking for and that 10 sec call back. I’d bet you’re probably using callbacks when you shouldn’t.

If you were to get rid of the callbacks could you still solve this problem? If not, why can’t you?

4

My use case is: My back-end database updates about once a minute (but at unpredictable times). I would like clients to react to this change within about 1 second. Processing and sending this data to clients (sending about 250k data points) was taking a bit longer than a second.

I’d love to get rid of the callback, but as far as I can tell the client must poll for changes - there is no way for the server to initiate data exchange with all clients.

You are right about the red flags, and the work around is very easy: make sure the callbacks finish fast enough - or slow the client polling down slightly. The documentation has plenty of ideas about how to do that - in my case raising PreventUpdate when the data hasn’t changed is more than enough to make my app work (I discovered this feature as part of debugging my problems!) - so on average callbacks finish in milliseconds and once a minute a callback takes slightly longer. I could also down-sample the data, or poll every two seconds, or perhaps cache the callback result (assuming its not the sending of the data that is the problem).

So my question is more academic than practical (and more of a suggestion or feature request rather than needing any immediate help!). First, can we detect re-entrancy in callbacks (perhaps so we can log a warning that our app has become too slow)? The answer is yes, but I can’t figure out a simple solution. Second, if we have re-entrancy, is there a quick solution such as aborting all later calls until the first call finishes? I can’t find a solution to this - my example above comes close but the client doesn’t update.

I think it would be a useful recipe (or feature, if it needs work to support it). In my experience re-entrancy is at best unexpected, and at worst a bug, so giving the user a few tools to detect and prevent might save some grief.

5

That sounds like a real time analytics platform with a client that is connected using a socket protocol. Not really what I’d use Dash for but if you got the time to hack away you can probably get something working.

6

JITCache is designed specifically for this purpose. Re-entrant callbacks can be blocked and the computation only run once.