Reducing loading time for image update

I have a quite simple application (on the outside :)) which is supposed to show pictures created with matplotlib: believe me, I would be happy to use plotly instead but I can’t do these plots in plotly.

There is a quite simple callback which takes values from a slider and two dropdowns, calls the matplotlib function which produces a plot, saves it on the server and passes the filename to dash which then takes the picture and shows it as base64-encoding. As far as I know this is the easiest way to embed pictures in a Dash application. Here is a minimal snippet from the app

app = dash.Dash(__name__)
server = app.server

app.layout = html.Div([
    html.Div([dcc.Dropdown(id='chart-dropdown',
                 options=dropdown_options, value='gph_t_850',
                 style=dict(width='40%', verticalAlign="middle")
                 ),
    dcc.Dropdown(id='proj-dropdown',
                 options=projection_options, value='euratl',
                 style=dict(width='40%', verticalAlign="middle")
                 )],style={'float': 'left', 'width': '100%'}),
    html.P('Forecast lead time'), html.Div(dcc.Slider(id='slider-step', min=0, max=120, value=0,
                               marks={i: i for i in steps}, step=None,
                              tooltip={'always_visible': True}))], style={'float': 'left', 'width': '100%'})]),
    html.Div(html.Img(id="image_plt"), style={'textAlign': 'center'})
])

@app.callback(
    Output('image_plt', 'src'),
    [Input('chart-dropdown', 'value'), Input('slider-step', 'value'), Input('proj-dropdown', 'value')])
def update_figure(chart, f_step, projection):
  filename = '/tmp/' + projection + '_' + chart + '_%s_%03d.png' % (run_string, f_step)

  if os.path.exists(filename):
    out = b64_image(filename)
  else:
    filename_fig = plot_vars(f_step, projection, load_all=False)
    assert filename_fig == filename, "Mismatching filename strings! From plot_vars:%s , defined:%s" % (filename_fig, filename)
    out = b64_image(filename_fig)

  return out

plot_vars is the matplotlib function that creates the plot and save it as png in a temp folder.

As you can see If the image already exists on the server the cached version of the picture is shown without calling the matplotlib function. In this case the duration of _dash-update-component is solely related to the base64 encoding, I believe. Unfortunately just loading the picture and showing it (as an effect of moving the slider) takes about 340 ms (see screenshot)

This is fine for most applications but I would like to be able to move between different images with almost no delay, as I would do in a “barebone” HTML-JS implementation as this one ICON model visualization . In this particualr implementation the delay is about 150 ms the first time that the image has to be fetched but afterwards if you select the same image it will just be fetched from the cache without calling anything, so it is basically istantaneous. I’m not sure whether this kind of behaviour can be reproduced in Dash but I think most of the delay is due to the fact that the image has to be encoded in base64 every time.

Is there a simpler way to embed the image in the Dash application? or any way to speed up the update?

Thanks for the help

P.S. I’ve taken the liberty of publishing the app on heroku(https://icon-forecast.herokuapp.com/) if you want to try it out, but don’t use the preload button as it will run out of memory on the free dynos :slight_smile:

If you just want to preload the images, you could put the base64 encoded images in a Store component (i.e. stored client side) and update the src with a clientside callback. That should yield slow initial load (since all images are loaded at once, possible crashing the browser if you have too data), but i would expect the update to be more or less instantaneous.

Why can’t you do the plots in Plotly?

Because I need to do filled contour on different map projections; as far as I know there is no such capability in plotly (see e.g. here).
This is a sample of the plot that I’m doing in mpl

Sounds like a good idea, actually. I’ve used the Store component before to save data and retrieve it in a different callback.

Not necessarily, I don’t preload all images when I open the app at the beginning. It will be triggered by the slider callback.

Ah, i see. I have previouslt done color overlays using tiles in dash-leaflet,

but i haven’t played with contours. It should be doable, but will probably require (significantly) more as compated to just using mpl.

Oh My, this is exactly what I’ve be[quote=“Emil, post:5, topic:45775”]
GitHub

thedirtyfew/terracotta-dash-example

Contribute to thedirtyfew/terracotta-dash-example development by creating an account on GitHub.
[/quote]

Oh my, this is exactly what I’ve been looking for…I would really like to make everything interactive. As far as I understand I only have to do the conversion before to make the data into Geotiff. I’m also using grib2 inputs in the app, but as you can imagine the downloading and reading of the data is the most expensive operation…

You still need a system to run the conversion beforehand right? How long does it take usually?

Yes, with the dash-leaflet solution, everything will be interactive. Furthermore, as the data are loaded as tiles, you don’t need any special loading/cache/preloading on the client.

In the example i made, the data conversion time is on the order of seconds, but it will depend on the amout of data that you target, the resolution of the target raster and the HW used. That being said, it will probably take more time to download the data in the first place (at least it did in my case). And since you only need to do the conversion a few times a day, i guess thats OK.

Any idea why the memory consumption slowly increases whenever I switch between images?

As the function that does the plot returns only the filename and the loading of the image is done inside app.py I would expect all the local variables of the plotting function plot_vars, which are holding the “big” data arrays, to be cleared once I return from that function. Instead I see memory usage staying always above 400-500 MB, which is what would normally use only the plotting function while it is running.