Do you need the storage to be persistent across page reloads? If not, storage_type='memory' would avoid the quota problem (which is browser-dependent but normally 2-10MB)
But that wouldn’t address the slowness issue, unless the slowness comes from trying to interpret a data file that got cut off in the middle.
Looks like we don’t have any examples in Part 4. Sharing Data Between Callbacks | Dash for Python Documentation | Plotly of caching uploaded files - We should definitely add that! The general pattern I’d use for that is:
- create a
dcc.Storethat will hold just cache key(s) - create a callback with the uploaded file(s) as
Input/Stateand the aboveStoreasOutput- For each file, generate a cache key. This can just be a UUID, like
file_id = str(uuid.uuid4())uuid — UUID objects according to RFC 4122 — Python 3.12.1 documentation - Insert the file (or some parsed/preprocessed version of it - Fast way to share data between callbacks - #2 by chriddyp) in the cache using this key. If you’re using
flask-caching, you can do that withcache.set(file_id, file_contents)(see Flask-Caching — Flask-Caching 1.0.0 documentation) - Return the
file_id(s) to theStore
- For each file, generate a cache key. This can just be a UUID, like
- Now to use the file contents to create some output, provide the
Storedata as anInput/Stateto another callback, and callcache.get(file_id)to retrieve the data
One more thing: Before I had just saved the original dataframe in a global variable, because in the beginning I did not understand the data storage very well.
So interestingly this has never caused problems, the app to run slow any anything therelike.
This will work fine as long as you’re the only one using the app, and you run the app in a single process. Once you have multiple users or processes though you’ll encounter problems.