Live chart with extendData uses most of the client's CPU

Dash Python
1

Hi,
I created a Dash application that plots some data from a FastAPI source.
I used EventSourceResponse from sse_starlette because the data might come at different intervals (not faster than 100ms).

Here is my simplified FastAPI (for testing purposes):

import asyncio
import json
import uvicorn
from sse_starlette import EventSourceResponse
from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.middleware.cors import CORSMiddleware

middleware = Middleware(CORSMiddleware, allow_origins=["*"], allow_headers=["*"])
server = Starlette(middleware=[middleware])

async def random_data():
    while True:
        await asyncio.sleep(0.1)
        timestamp = int(time.time()*1000)
        send_msg = {
                'timestamp': timestamp,
        }
        for j in range(8):
            send_msg["var_" + str(j)] = (math.sin((timestamp + j) / 1000) + j)
        yield json.dumps(send_msg)

@server.route("/random_data")
async def sse(request):
    generator = random_data()
    return EventSourceResponse(generator)

if __name__ == "__main__":
    uvicorn.run(server, port=5000)

In my dash app, I declared an EventSource component from the dash_extensions module as follow:

EventSource(id="sse-1", url="http://127.0.0.1:5000/random_data")

with a client-side callback to bypass the Dash server:

update_graph = """function(msg) {
                  if (!msg) {
                    return {}; 
                  }
                
                  const data = JSON.parse(msg);
                
                  const x = [new Date(data.timestamp)];
                  const y = [
                    [data.var_0],
                    [data.var_1],
                    [data.var_2],
                    [data.var_3],
                    [data.var_4],
                    [data.var_5],
                    [data.var_6],
                    [data.var_7],
                  ];
                
                  return [
                    {
                      x: Array(8).fill(x),
                      y: y,
                    },
                    [0, 1, 2, 3, 4, 5, 6, 7],
                    1000,
                  ];
                } 
"""
app.clientside_callback(update_graph, 
                        Output('stress-test-graph-1', 'extendData'), 
                        Input("sse-1", "message"))

And my graph is declared as follow:

dcc.Graph(id='stress-test-graph-1',
                    figure=go.Figure(data = [go.Scattergl(
                       x = [], y = [], mode = 'lines'
                    )]*8,
                    layout=go.Layout(
                        xaxis=dict(fixedrange=True, type='date'),
                        margin=dict(l=20, r=20, t=20, b=20))),
                )

I used Scatergl to get better performances, but Google Chrome is using a lot of CPU (I5-8250U, 16Go RAM):

cpu
cpu860×254 18.5 KB

I think Dash won’t be able to measure the ressources used by my callback, as it is client-side and it shows 0:
callback
(the figure callback is used at page load to load the data history from a Redis cache).

Right now my benchmark produces 8 series in a chart, with 60% CPU average with spikes at 100%, and I might have to provide even more charts in my app.

Is there a way to improve performances in my app?

  • I thought about using Holoviews, but it is so different, I’m not sure I can use it with Dash
  • I didn’t want to create an Interval fetching data from the Redis cache because I don’t want to miss any data point and I have a lot of uncertainty about my data source (it’s going to be MQTT mostly)
  • I thought about sending data in batches from my API, but I need a real-time-like UX so I’ll need to do some work in Javascript anyway to render an illusion of real-time

Or am I using the wrong tool to develop my app?
Thanks in advance.