I have a huge data that I am storing in a dcc.Store because it needs to be shared with other layouts (multi-page web app). I took the size of the serialized JSON for the dataframe to be stored and it’s about 3.75 MB. The documentation says there’s a limitation if using memory.
Is there a way to increase this?
Also, I know that the store component is not being updated with the serialized JSON because the component does not have a modified_timestamp when I’m checking the debugger. MORE WEIRD is that when I actually deploy the Dash app to a Heroku server, the store component stores the 3.75MB and thus can be used by other components such as graphs. However, the store does not get updated when on local. I’ve been at it for hours and can’t seem to figure it out.
There is a limit to the amount of data your can keep in a Store component due to browser limitations. If you are afraid that you might hit it (or if performance becomes an issue), you might want to consider server side caching instead. Either directly (e.g. using files or an in-memory cache) or using a ServersideOutput,
The mechanism itself doesn’t impose any limits, but your choice of backend does. If you use your local disk for example, the limit will be the available space on the disk.
I am having some stability issues with an app that uses FileSystemBackend cache with Serverside outputs, and I’m trying to pin down the reason. The server I am using is more than large enough, and each dcc.Store() stores a pandas dataframe that is sourced from a parquet file about 4MB, but in pandas size thats about 40MB. Do you think this is causing the app to crash sometimes? Are there ways to increase performance?
Yes, I’m using HPA in kubernetes, so there are currently 6 pods for my resources, although it doesn’t look like most of them have any data stored because of using the fixed keys (?). I also want to add that I have tried both session and local memory in the dcc.Store() but I am currently using local.