What can I do to run callbacks in parallel using plotly dash?

Dash Python
1

I have a dashboard built on plotly dash. It is very similar to this dashboard–

import datetime

import dash
from dash import dcc, html
import plotly
from dash.dependencies import Input, Output

# pip install pyorbital
from pyorbital.orbital import Orbital
satellite = Orbital('TERRA')

external_stylesheets = ['https://codepen.io/chriddyp/pen/bWLwgP.css']

app = dash.Dash(__name__, external_stylesheets=external_stylesheets)
app.layout = html.Div(
    html.Div([
        html.H4('TERRA Satellite Live Feed'),
        html.Div(id='live-update-text'),
        dcc.Graph(id='live-update-graph'),
        dcc.Interval(
            id='interval-component',
            interval=1*1000, # in milliseconds
            n_intervals=0
        )
    ])
)


@app.callback(Output('live-update-text', 'children'),
              Input('interval-component', 'n_intervals'))
def update_metrics(n):
    lon, lat, alt = satellite.get_lonlatalt(datetime.datetime.now())
    style = {'padding': '5px', 'fontSize': '16px'}
    return [
        html.Span('Longitude: {0:.2f}'.format(lon), style=style),
        html.Span('Latitude: {0:.2f}'.format(lat), style=style),
        html.Span('Altitude: {0:0.2f}'.format(alt), style=style)
    ]


# Multiple components can update everytime interval gets fired.
@app.callback(Output('live-update-graph', 'figure'),
              Input('interval-component', 'n_intervals'))
def update_graph_live(n):
    satellite = Orbital('TERRA')
    data = {
        'time': [],
        'Latitude': [],
        'Longitude': [],
        'Altitude': []
    }

    # Collect some data
    for i in range(180):
        time = datetime.datetime.now() - datetime.timedelta(seconds=i*20)
        lon, lat, alt = satellite.get_lonlatalt(
            time
        )
        data['Longitude'].append(lon)
        data['Latitude'].append(lat)
        data['Altitude'].append(alt)
        data['time'].append(time)

    # Create the graph with subplots
    fig = plotly.tools.make_subplots(rows=2, cols=1, vertical_spacing=0.2)
    fig['layout']['margin'] = {
        'l': 30, 'r': 10, 'b': 30, 't': 10
    }
    fig['layout']['legend'] = {'x': 0, 'y': 1, 'xanchor': 'left'}

    fig.append_trace({
        'x': data['time'],
        'y': data['Altitude'],
        'name': 'Altitude',
        'mode': 'lines+markers',
        'type': 'scatter'
    }, 1, 1)
    fig.append_trace({
        'x': data['Longitude'],
        'y': data['Latitude'],
        'text': data['time'],
        'name': 'Longitude vs Latitude',
        'mode': 'lines+markers',
        'type': 'scatter'
    }, 2, 1)

    return fig


if __name__ == '__main__':
    app.run_server(debug=True)

The difference is that, in my dashboard, there are about twenty different time series and every one of those time series gets updated every two seconds. Right now, my dashboard runs sequentially and without any parallelization, but it ends up being pretty slow both because there are a lot of time series to update and there is a lot of data.

I am on a windows machine and I set up waitress using this command: waitress.serve(app.server, threads = 16)

What do I need to change about my script to take advantage of parallelization? I was thinking about making a different callback for every time series and having them all update the figure independently of one another.

2

This would be the way to go about it, unfortunately, running things asynchronously with Flask → Dash is pretty hard. But that would be the only way to update all the graphs at the same time without multiple calls.

3

What about waitress?

1 Like
4

Waitress can cater to multiple requests at the same time, but it won’t split up an individual request into multiple processes.

5

Can I set a time-out on the queries? I think the default behavior is that if the Interval triggers and the callback is still running, then it skips the running callback and starts the new one. What if I want the running callback to be blocking?

6

I don’t know if you can set a timeout. What you can do is adjust your interval or setup background callbacks that will disable the next update until it’s been fulfilled.

7

What do you mean by " setup background callbacks that will disable the next update until it’s been fulfilled."?

8

Basically, you set a state that you then test in your next call to make sure it isn’t true.

9

Can one callback have multiple outputs at different points in the run? I’m wondering if I can set a state at the beginning of the run that says “working” and then set it to “done” at the end of the run and add some logic that says "if state is ‘working’, then do nothing til some specified timeout period or til it changes to done’

10

Yes. With background callbacks.

Go to the bit with the progress bar.

11

Wow that’s really great. If I’m reading this right, then this means I can do little updates over the course of a single callback. That also helps me avoid a race condition that I was worrked about because I can house all the updates in a single callback and then just through them sequentially and send back the output as each one finishes.

Will this prevent new triggers from over-riding the callback? For example, if an update interval hit before the progress bar was done

12

Depends on how you set it up.

I don’t know about incremental updates to static properties, but maybe it can.