Dash app memory usage problem. Dash-extensions EventSource real time data streaming

Dash Python
1

Hi,

I’m trying out dash extensions for real time data streaming to a dash app. I found out that when running the app it’s memory usage increases over time. I don’t store any data and the new incoming data is always of similar size. The only thing dash app does is update a plotly graph with new incoming data. Why is memory usage increasing and can I resolve this in some way?

I use:
dash-extensions DashProxy
dash-extensions Eventsource
client side callback with EventSource as input

below is code for reproducing the behaviour (two scripts one for generating data, one for the dash app)

SCRIPT 1

import asyncio
import json
import random
import numpy as np
import uvicorn
from sse_starlette import EventSourceResponse
from starlette.applications import Starlette
from starlette.middleware import Middleware
from starlette.middleware.cors import CORSMiddleware

middleware = Middleware(CORSMiddleware, allow_origins=[““], allow_headers=[””])
server = Starlette(middleware=[middleware])

@server.route(“/random_data”)
async def sse(request):
async def event_publisher():
try:
while True:
disconnected = await request.is_disconnected()
if disconnected:
print(f"Disconnecting client {request.client}")
break

          yield json.dumps(np.cumsum(np.random.normal(0,1,1000)).tolist())
          await asyncio.sleep(0.1)
      print(f"Disconnected from client {request.client}")
    except asyncio.CancelledError as e:
      print(f"Disconnected from client (via refresh/close) {request.client}")
      raise e
return EventSourceResponse(event_publisher())

if name == “main”:
uvicorn.run(server, port=5000)

SCRIPT 2(DASH APP)

import json
import dash_bootstrap_components as dbc
import plotly.express as px
import numpy as np
from dash_extensions import EventSource
from dash_extensions.enrich import html, dcc, Output, Input, State, DashProxy

def generate_fig():
fig = px.line(np.cumsum(np.random.normal(0,1,1000)))
fig.update_layout(
yaxis={‘title’:{‘text’:‘’,‘font’:{‘size’:3}},
},
xaxis={‘title’:{‘text’:‘’,‘font’:{‘size’:3}},
},
margin=dict(l=0, r=0, t=0, b=0),
showlegend=False,
width = 360,
height = 300)
return fig

cardR1C1 = dbc.Card(
dbc.CardBody(
[
html.H5(“Card R1C1 header”),
html.Hr(),
html.P(‘Card R1C1 body’,
style={‘fontSize’: 12,‘font_family’:‘Courier New’,‘font-style’: ‘italic’}),
html.Hr(),
dcc.Graph(id=‘graph1’,figure=generate_fig())
])
)

cards = dbc.Container(
[dbc.Row([
dbc.Col(cardR1C1,className=“col-sm-4”)])])

update_graph = “”“function(msg,Figure) {
if(!msg){return {};} // no data, just return
const data = JSON.parse(msg); // read the data
return {‘data’: [{y: data,
type: “line”}],
‘layout’:Figure.layout}}; // plot the data
“””

app = DashProxy(name,
title = ‘Stream’,
update_title = None,
external_stylesheets=[dbc.themes.JOURNAL],
suppress_callback_exceptions=True)

app.layout = html.Div([cards,
EventSource(id=“sse”,
url=“http://127.0.0.1:5000/random_data”),
])

app.clientside_callback(update_graph,
Output(“graph1”, “figure”),
Input(“sse”, “message”),
State(‘graph1’, ‘figure’))

if name == ‘main’:
app.run_server(debug=False)

3

Hi,

I’m experiencing the same problem. The memory usage becomes unbelievably large (around 21GB for me)… I’ve tried gc.collect() but it did not seem to solve the problem.

Have you figured out what the problem was?

4

Hello @hzpkk520,

Welcome to the community!

It’s hard to say without seeing your code, do you make use of print statements? How are you running the server? And so on.

5

Hi! I will have to get back to you as I can’t remember. I think this problem occurred to me on a Mac and when I ran it on a windows machine later the memory problem was gone( ie memory usage stayed constant). I don’t know if I solved it for Mac though.

1 Like