Dynamic pages with slow data loads

Dash Python
1

I have a small dash app currently hosted on Google Cloud Run, with two pages [internalFeed, adminFeed] with a handful of graphs/tables each. I am capturing the dynamic variable in the pathname (/internalFeed/). The ecosystem variable is passed through a callback to retrieve data from:

  • 1 cloud mysql table where rows range from the low 100s to edging towards 200k rows and ~10 columns
  • 3 tables on SupaBase, each ranging from the low hundreds to rarely over 1200 rows. and each ~12 columns.
    Our team is restructuring some data architecture so they will be consolidating in the next quarter.

The internalFeed page allows a dropdown to select through 1 of the many ecosystems to display data on in a stacked barchart and two tables, the adminFeed page is similar, but uses a few more tables. The data is aggregated and slimmed down to a small reporting style, rarely more than a few hundred datapoints each.

Running locally there is sometimes ~10 seconds of lag when switching between different ecosystems, but hosted on cloud run the app often fails to load, and shuts down. There are usually POST 500 errors on the logs when attempting to update a callback.

I’m not entirely where to start or what information I can provide that will be helpful. Thanks in advance!

2

Tried swapping from using the supabase client to using a direct postgres connector but I’m not sure it’s any improvement. Here’s the current app: Dash

3

Hello @nationalmyth,

Welcome to the community!

Sorry no one has responded to you yet.

What I would suggest is to see if you can break your data and/or into chunks for updating, this should help you with getting a response from the server. Since the timeout will typically be 30 seconds.

Also, 500 error I think is server error, check the network tab to see if it is timing out or if 500 it means something is crashing on your server.

Check to make sure that there aren’t any errors popping up there. One thing that can happen is a library isn’t being imported but it’s not effecting the app spin up so the app runs ok.

5

So I am reverting back the data retrieval I was using before, the Supabase client. I am having trouble identifying the bottleneck beyond this.

6

10 seconds on local is still a long time.

What server are you hosting this on?

I’d try to decrease your time on local and see if that can translate to the server.

7

This is being served on a GCR server with 4 workers, 8 threads (gunicorn) 2VPC, 2GB memory

8

Locally, it’s a sporadic dance with lag, but when launched into the Cloud Run cosmos, it’s a celestial challenge. The app occasionally falters, invoking POST 500 errors during callbacks, particularly when navigating diverse ecosystems. It’s a cosmic riddle, and I’m reaching out to the Plotly galaxy for unconventional wisdom.