Callback failed: the server did not respond - Browser keeps updating

[mcsewhoy]

Hi Dash Team,

Firstly, thank you for such a wonderful tool you have built, I have gotten some things created because of your hard work, so I really appreciate it.

I do have a question related to Dash and its performance. I have built a dashboard using a data table, and I never really have a lot of data in it (say 250 rows x 20 cols). I also only run and use it it locally all on one computer.

However for some reason, it keeps forcing my browser (Chrome or Edge) into “updating” mode, when I click around certain components or the data table cells, at a normal user’s pace. It basically freezes the dashboard up, preventing proper updates of components, also preventing further clicking around, preventing the expected callbacks from working properly, until I wait some time for what seems to be a backlog of events, i.e the “updating”, to end (it can be a 10-20 second wait quite frequently until it resolves itself). And once it resolves itself, the problem will reoccur again if I click around the datatable again with a few clicks. But if I click one cell at a time, and wait a decent amount of time before doing anything else or before clicking again (e.g. 5-10 seconds between clicks), then the problem either will be ok, or won’t last as long. It is as though the program can’t handle too many things happening at once, and the browser gets overloaded. I do notice that the longer the data table, the worse the problem (e.g. if there is only 5 rows, the problem is minor, if the datatable is 250 rows, the problem becomes quite severe, though I can’t see why 250 rows is a lot).

e.g. clicking a Radio to change the Datatable data quite often causes it if I click the radio 3 times in a row
Output(‘table’,‘data’) Input(‘single_radio’,‘value’)

or

e.g when I click around my main table, which actives “active_cell” to update a few other components, say clicking 5 different cells in a row without waiting.
Output({‘type’: ‘entry_form’, ‘index’: ALL}, ‘value’)] , Input(‘table’,‘active_cell’), entry_form being a series of dropdown components.

I don’t have enough experience points to work out what the problem is, and my program is a bit too large to drop the code here. In any case, it seems to be affecting the whole browser and not one specific component, so it is hard for me to isolate the code which has a problem. But wondering if anyone has seen something like this before (perhaps related to rendering, as my limited experience in exploring the browser Devtools may highlight - see attached pics). Unfortunately, this problem makes my whole dashboard kind of inoperable for basic use. As such, any help would be greatly appreciated, and if you need any more info, please do let me know and I would be happy to provide.

Thanks
Marcus

[chriddyp]

From the timeout calls, it looks like the callbacks are taking longer than 30 seconds. I would try inspecting these long callback requests and see if that matches your expectations.

One thing that I’ve seen in local development is where the callbacks write some data to the file system, and that triggers the server reloader. Since the server would reload mid-callback, the requests would fail.

The other thing I would try is writing a new Dash app with some dummy data and dummy callbacks that matches the shape & architecture of your existing app and see if you can reproduce the error from scratch. This is a great way to debug and get to “root cause” of issues as, as you mention, it can be difficult to debug a large codebase.

[mcsewhoy]

Thank you Chris for your prompt response.

The reason why the callback looked like it took longer than 30 seconds was that I was clicking the same callback a few times (i.e. clicking around my table with the “active_cell” input on), and the profiliing was capturing all of that as one time.

If I click very very slowly around, i.e. one click, wait 3 seconds, one click, then the callbacks are quite fast (e.g. if you see the attached pic, there are 4 callbacks activating for each click, taking a total of ~700ms together)

But when I start clicking around fast like a normal user would, and specifically, when I click when the browser says “updating”, the same callback timings blow out.

And from the below pic, it looks like it is almost all network time (99.5%), and not compute time causing the problem, even though I am running this all off a single computer locally, with the browser on the same computer as the app, related to the the server is not responding anymore error (perhaps amongst other issues)

I took your advice and checked those four callbacks for writing data to the filesystem (all I use is pd.to_csv), but it seemed none of those 4 callbacks utilize any form of filesystem saving.

Short of rewriting the dash app again as you mention, I did also try to strip my app down to the bare essentials, commenting out conditional formatting, unrelated callbacks, side features, layout, etc, to try to isolate the problem. Quite a few components/callbacks cause the same issue, mainly as a function of how fast I click them (e.g. boolean switches to change Output(‘table’,‘data’)), over and above what I have already described. Virtualization did help speed my table up, but still not to where I would like. I tried cutting the data down to 10 rows instead of 250 last night, and it sped things up quite a bit and reduced the browser “updating” time, so I know the problem is related to table size as well. But again, it is not the computation time causing issues, it seems to be network time, and I am having trouble fixing it as a result since there isn’t a very specific error message for this which tells me where to look in the code. As such, I have been trying for months now.

e.g. below pic, different callbacks circled on the right side occur when I click a boolean switch which hides columns in the table, and changes the table’s columns / data as an output. If I click this switch a lot, the timing for the associated callbacks also blows out. But clicking them slowly is fine, and all output comes out as expected. So it doesn’t seem to be callback specific, again more related to the frequency and speed of rerunning any callback over and over.

If you have seen any other potential sources of this in past (aside from saving to the filesystem, which I think I have ruled out), that would be very useful for me to go search upon. Else as you mention, I might have to rewrite a decent chunk of this again, though I can imagine it would be time consuming, and I might well hit the same issue again but not know why (as it doesn’t seem to be related to one specific callback).

Thanks
Marcus

[chang]

Hi, have you found out how to fix this issue? I encountered the same problem, and my dash app wasn’t usable because the server did not respond error kept appearing

[POBonin]

Hi everyone! I have also been encountering the same error but with a much more simple app I have developed with Plotly-Dash (under 80 lines of code). It appears the server fails to respond but after letting it run for a while, the issue sorts itself out when the browser refreshes. Hard to understand what’s causing this issue.

[adamschroeder]

hi P-O, @POBonin,
Thank you for letting us know. Can you share with us those 80 lines of code along with the data that it uses?
We can try to see if the sever fails on our end as well.

[POBonin]

Hi Adam,

It’s the time series app I’ve been developing. What I’ve noted about the bug is that:

• it only happens when I run the app through the development server a while after it was closed
• reloading it after about 2-3 minutes resolves the problem and the app works bug-free afterwards

Please find below the code:

from dash import Dash, dcc, html, Input, Output
import pandas as pd
from datetime import date
import plotly.express as px

filepath = "https://raw.githubusercontent.com/plotly/datasets/master/stockdata.csv"
df = pd.read_csv(filepath)
df["Date"] = pd.to_datetime(df["Date"])
df = df.set_index(df["Date"]).drop(columns="Date")
new_df = df.diff(1)[1:]
df = df[1:]

stock_list = list(df.columns.values)

app = Dash(__name__)

app.layout = html.Div([
    html.Div([
        html.H1('Stock Prices Time Series',
                style={'textAlign': 'center'})]),
    html.Div([
        html.H2('Enter a stock symbol, then select a date range:',
                style={"margin-left": "5em"}),
        html.Div(
            [
                dcc.Dropdown(
                    id='time-series-app-x-dropdown',
                    value='IBM',
                    options=stock_list,
                    multi=True,
                    style={'font-size': 20,
                           "margin-left": "3em"}
                )
            ], style={"width": "20%",
                      "display": "inline-block",
                      "verticalAlign": "middle"}
        ),
        html.Div(
            [dcc.DatePickerRange(
                id='time-series-app-x-date-picker',
                min_date_allowed=date(2007, 1, 4),
                max_date_allowed=date(2016, 3, 1),
                initial_visible_month=date(2016, 3, 1),
                end_date=date(2016, 3, 1)
            )
            ],
            style={"width": "49%",
                   'font-size': 20,
                   "margin-left": "4em",
                   "display": "inline-block"}
        ),
        html.Div(
            [dcc.RadioItems(
                id="time-series-app-x-radio-items",
                options=["Stock Prices", "Returns"],
                value="Stock Prices"
            )
            ], style={"width": "50%",
                      'font-size': 20,
                      "margin-left": "-21em",
                      "display": "inline-block"}
        )
    ], style={"display": "inline-block", "width": "100%"}
    ),
    dcc.Graph(id='time-series-app-x-graph',
              style={"margin-left": "2.5em"})
]
)


@ app.callback(
    Output('time-series-app-x-graph', 'figure'),
    [Input("time-series-app-x-dropdown", "value"),
     Input("time-series-app-x-radio-items", "value")])
def update_graph(stock, graph_type):
    if graph_type == "Stock Prices":
        fig = px.line(df, x=df.index,  y=stock)
        fig.update_layout()
        return fig
    elif graph_type == "Returns":
        fig = px.line(new_df, x=df.index,  y=stock)
        fig.update_layout()
        return fig


if __name__ == '__main__':
    app.run_server(debug=True)

[adamschroeder]

That’s weird, @POBonin . I’m not sure why that is happening; I couldn’t replicate the error. When I run the code on my Windows machine and VS Code, it loads instantly.

[Engr.What]

I got the same problem too.
any update or solution about this?

[betoS]

Hi Marcus @mcsewhoy,

I want to start by thanking you for your valuable contributions to this community. I also want to thank @adamschroeder and @chriddyp for your continuous support.

I am encountering an issue where I am getting a “server not responding” error randomly when running the app locally in development or on the deployed Heroku ECO dyno, a “WORKER TIMEOUT” error in the latter.

To trigger the KPI calculations, you can upload any sample data using the top-left green button as shown below, for which I have shared two sample CSV files in the sample data folder of the repository:

While the app has simple calculations, I am unable to determine why it is randomly hanging. It works at some runs but not at others. I have tried various stacks without success, and I am sharing the pip freeze command output below for your reference (exact replication in Python 3.8.16):

Brotli==1.0.9
click==8.1.3
dash==2.0.0
dash-bootstrap-components==1.0.2
dash-core-components==2.0.0
dash-html-components==2.0.0
dash-table==5.0.0
Flask==2.1.3
Flask-Compress==1.13
gunicorn==20.1.0
importlib-metadata==6.1.0
itsdangerous==2.1.2
Jinja2==3.1.2
MarkupSafe==2.1.2
numpy==1.22.1
pandas==1.4.0
plotly==5.13.1
python-dateutil==2.8.2
pytz==2022.7.1
six==1.16.0
tenacity==8.2.2
Werkzeug==2.0.0
zipp==3.15.0

Thank you for your time and support. I would greatly appreciate any help you can provide in resolving this issue. If you require any further information, please do let me know, and I will be happy to provide it.

With best regards,
Alberto

[adamschroeder]

hi @betoS
From my experience, I’ve gotten these errors when the calculations of my app take too long on heroku.
Do you have heavy calculations in your app that might take more than 30 seconds?

[betoS]

Dear @adamschroeder,

Thank you so much for your prompt response and willingness to help with the performance issue in this Dash app.

I understand that there aren’t heavy calculations involved in the app, yet the performance issue persists. Moreover, the app sometimes hangs randomly, both in local development and production environments.

For instance, when I uploaded the sample_appointments.csv from the GitHub repository, which is just 88 KB in size, the performance has been as shown in the image below. However, this response may not occur every time, and sometimes it may result in a hanged response.

I would highly appreciate if you could replicate how this behaviour would work at your local server and provide your valuable insights to help me resolve this issue.

Thank you in advance for your kind support, with best regards,
Alberto

[jinnyzor]

Hello @betoS,

In your requirements, I noticed that you are running dash 2.0, we are on dash 2.9 now.

Maybe try updating the libraries on your local and trying again.

It seems that the only spot you use the data from your csv, you are referring to it as State. This means that the new data will only be used once you update some of your date triggers.

To update the KPIs every time there is new data uploaded, you need to change this State call to Input.

beto-Sibileau/ebse-kpis-dash/blob/8de859aae3911eb00775db7d7ca3a2489aab0512/ebse_kpi_dash.py#L1107

[betoS]

Hi Bryan @jinnyzor, thank you very much for your response and engagement!

Indeed, I downgraded the stack to test a hypothesis, but unfortunately, the issue I observed during development persisted. Following your suggestion, I updated to the Python 3.11.2 runtime and re-installed the required libraries. However, the issue persists, as you can observe when running the code locally (here is a list of the libraries I have installed for exact replication):

Successfully installed Flask-2.2.3 Jinja2-3.1.2 MarkupSafe-2.1.2 Werkzeug-2.2.3 click-8.1.3 dash-2.9.1 dash-bootstrap-components-1.4.1 dash-core-components-2.0.0 dash-html-components-2.0.0 dash-table-5.0.0 gunicorn-20.1.0 itsdangerous-2.1.2 numpy-1.24.2 pandas-1.5.3 plotly-5.13.1 python-dateutil-2.8.2 pytz-2022.7.1 six-1.16.0 tenacity-8.2.2

Thank you for taking the time to review the code and for noting that the KPI calculations use the State of csv-df. However, I want to emphasize that every time the user uploads new data, they receive a modal message. Upon closing the modal, the new KPI calculations are triggered. Please refer to the link and callback diagram below that illustrate this behavior:

beto-Sibileau/ebse-kpis-dash/blob/8de859aae3911eb00775db7d7ca3a2489aab0512/ebse_kpi_dash.py#L1106

I would appreciate very much if you could replicate this behaviour on your local server and provide me with your valuable insights to help me resolve this issue.

Thank you again for your kind interest and consideration. With best regards,
Alberto

[jinnyzor]

@betoS,

Are you making sure that you are uploading unique data each time?

Your trigger is based upon the contents, which means that if the contents is exactly the same, even with a different filename, it will not trigger the rest of your callbacks.

If I try to upload the copy after doing the original, it never looks like it was changed, even though it would have pulled different information for the filename and last_modified. If you change these all to Input then I think you’d probably have the interaction that you are expecting.

Also… on a personal note, the flashing is a little over the top.

[betoS]

Thanks for your feeback Bryan @jinnyzor, I appreciate your engagement in suggesting modifications to the project. However, I feel that we may be getting off-topic and I would like to steer the conversation back to the main issue at hand.

To keep the conversation on track, I suggest we move forward by replicating the behavior on your local server and providing insights to help resolve the issue. This will help us to stay focused on finding a solution.

Once again, thank you for your engagement and feedback. I appreciate your willingness to help resolve this issue.

[jinnyzor]

@betoS ,

I tried to replicate the behavior, and it was only when uploading the same data that was in a differently named file.

Please make the adjustments to the inputs and make sure it doesn’t fix the issue.

[betoS]

@jinnyzor, thank you for your suggestion, I appreciate your willingness to help.

For the sake of testing, I have already tried the adjustment you are suggesting and it did not resolve the problem.

I understand that you are suggesting changing the inputs, however, to my best understanding a hang in the response (Browser keeps updating) has never been related to an unexpected interaction between callbacks at all.

If you have any other suggestions or ideas on how to resolve this issue, I would be grateful for your input. Thank you again for your help.

[jinnyzor]

The browser shows no errors at all when processing the response of the data?

Not in the callbacks, but in the console for the browser.

[betoS]

Thank you for bringing this to our attention, I appreciate your efforts in identifying the issue.
I apologize that I have exhausted all of my current ideas on how to troubleshoot this specific problem, which appears to be random based on my current understanding.
If you or anyone else has any additional information or insights that could help us, please do not hesitate to share them.
Bests,
Alberto