Efficiently updating a measurement graph

Dash Python
1

I am using Dash for a GUI for my physics experiment. The experiment involves scanning a light beam over an area and measuring intensity of the returned light. The area to scan and the step size per pixel is defined by the experimenter. That results in a predefined image of U*V pixels. After the user has set all the necessary settings, the start measuring button can be pressed. This will initialize a Heatmap graph in the layout, together with a dcc.Interval component to handle the automatic updating throughout the measurement. The name of the measurement and its associated data file is saved in a hidden Div component for the graph’s update callback to process.

The measurement process itself is started in a separate thread, to isolate it from the GUI itself and I don’t want to use a callback for that. The measured data is saved in the file and consists of a U*V dataframe starting with zero’s in the beginning of the measurement. After each pixel, the corresponding cell in the dataframe is updated and the dataframe is saved to the CSV. Now back to the update graph callback. This function reads the measured dataframe and returns a go.Figure(go.Heatmap …) to the dcc.Graph component every time that it is executed.

Above process of updating the graph seems to be a bit slow. It involves replacing the whole figure instead of updating just the data that the graph contains. Is it possible to just update the existing data by means of replacement i.e. not returning a complete go.Figure component? A way I know of how to do this is by adding traces. However, I would like the graph to start in a UV sized map and not extend after a new trace line has been added. Is it possible to pre-set the size UV and add traces until the measurement is done?

1 Like
2

Hi @robbiedoes! Are all values of the UxV data array modified at each measurement? If yes, I suspect that it’s not longer to create a new Figure than to replace just the trace (which I would not know how to do anyway). What are the values of U and V, and the typical time interval of you measures? This would help thinking about possible solutions.

3

Hi Emmanuelle,

Thanks for your reply,

No not the whole array is changed at every measurement. I start with an UxV array of zero’s, which represent the points that are scanned one by one and row by row. Each point in the UxV array is a measurement and has a typical integration time of 5 ms. It is not necessary to update the graph every 5 ms, but rather every 100 ms. My update function looks like this:

@app.callback(Output('areaMeasurementGraph', 'figure'), 
          [Input('area_graph_interval', 'n_intervals'),
           Other inputs not worth mentioning.....])
def update_area_graph(n, x_range, y_range, step_per_pixel, hiddenDir, hiddenFile):
   x_values = np.arange(x_range[0], x_range[1]+step_per_pixel, step_per_pixel)
   y_values = np.arange(y_range[0], y_range[1]+step_per_pixel, step_per_pixel)
   measurements=pd.read_csv(hiddenDir+hiddenFile+".csv", index_col=0)
   z_min = measurements.min().iloc[0]
   z_max = measurements.max().iloc[0]

   return(go.Figure(data=go.Heatmap(x=x_values, y=y_values, z=measurements.values, colorscale="Hot", zmin=z_min, zmax=z_max)))

and updates this part of the layout:

dcc.Graph(id='areaMeasurementGraph', 
              figure=go.Figure(go.Heatmap(colorscale='Hot'), layout=go.Layout(width=800, height=600, margin=dict(l=20,r=20,b=0,t=10,pad=4))))

I return at every interval a new figure object with the updated data. So if my interval time is set to 100 ms, I basically add 20 new points to the graph (ignoring other time consuming tasks above the integration time of 5 ms). The rest of the points remain unchanged. Also in the function I return the x and y values again, which I guess also contributes to slowing down the process.

4

ok, thanks for the additional details (still interested in knowing the order of magnitude of U and V!). If you need a fast update of the figure, you can try dropping the go calls (which perform some validations which take time), and use instead dictionaries like
in the example of https://dash.plot.ly/interactive-graphing, this could accelerate things. I’m not sure whether the limiting factor is the validation or the rendering of all pixels…

5

I would recommend going with the extendData prop as your output. This will allow you to only send the latest measurement, which will be appended to the plot.


@app.callback(Output('areaMeasurementGraph', 'extendData'),
    [Input('area_graph_interval', 'n_intervals'), ...

You’ll need to follow the plotly.js API for extendTraces, which is not the same as how you would normally push figure data in via figure.data. I think you may have to separate out the data update from the heat scale settings, since plotly.extendTraces requires all the inputs to be extendable data.

However, this approach will be significantly more efficient than redrawing the figure each time.

An alternative is to try my third-party component: dash-extendable-graph component, which accepts data in the same way as figure.data – it’s available on pypi. It was really built to support my own needs re: time-series scatter plots, but I think it should work out of the box for heat maps as well. Unfortunately, not tested.