Optimal way to use hover data to select and plot another dataset

I have some callback that uses times from hoverData of a timeseries graph to select times from some other multi-dimensional array and update another graph. I was wondering if there would be some recommendations on how to optimise performance under this scenario when running from a remote server.

This works great when I run the app locally. Running on a remote server however causes the update of the second graph to be slow due to the round-trip.

One thing I was playing with was trying to save the multidimensional array in the browser within a hidden div as explained here. The challenge I am facing is that the multi-dimensional array I need accessing data from is relatively large (>30M) so deserializing it from the hidden div within the hover callback also becomes slow for this use case.

One thing I was thinking was to try and save data from each time in this array within its own hidden div so I could access / deserialize only what I really needed within the callback (I tried to exemplify this below). The idea would be something similar to Example 2 here, but trying to avoid deserializing the full data within the callback. I’m not sure however how I would set the (dynamic) ids within the hover callback though since the Input id in the callback need to be defined upfront…

Any suggestions would be really appreciated!

Thanks

app.layout = html.Div(children=[
    html.Div(
        id="hidden-div",
        style={"display": "none"},
    ),
    dcc.Interval(
        id="interval-component",
        interval=3600 * 1000,
        n_intervals=0,
    ),
])


@app.callback(
    Output('hidden-div', 'children'),
    [Input('interval-component', 'n_intervals')]
)
def set_tmp_data(value):
    df = load_my_data()
    children = []
    for time, data in df.iterrows():
        children.append(
            html.Div(id=str(time), children=data.to_json(), style={"display": "none"})
        )
    return children

Hi @ocerafa did you take a look at clientside callbacks (https://dash.plot.ly/performance and 📣 Dash 0.41.0 released). Indeed, at the moment if your callback is written in Python I’m not sure it’s deserializing which takes most of the time. Indeed, the Python app running on the server first needs to retrieve the contents of the Div from the client, which would involve transferring the data over the network. You can probably eliminate this with a clientside callback written in Javascript (I’m a bit outside of my comfort zone here, I hope what I wrote is correct).

Thanks a lot for your response @Emmanuelle,

I was just looking at client side callbacks and it looks like it may be a nice way to solve it - though this is outside my comfort zone as well so I’m struggling a bit to find out how to have available / how to access all the data I need with the client side callbacks.

You are definitely right in terms of the network traffic, but in this case there is also (de)serialization costs as well (unless I’m doing something very wrong which is also possible!) But just testing manually deserializing my array, it takes around 0.5 - 1 sec which for this use case is too slow…

The way I managed to get this approach the most responsive was by creating upfront images from all the plots that need updating when hovering on the timeseries, and just changing the Img.src in the hover callback. This works nicely locally (with a python callback) but I’m not sure either how to have these images available in the client side when running it from the remote server…

Thanks,