I’m using ServersideOutputTransform from dash-extensions with the default backend that writes the dcc.store output to the file_system_backend folder. The app is working fine. When developing, I can see that the file_system_backend folder increases in size on my computer and plan on setting up a job to clean it periodically. My question is about when my app is deployed in a docker container in the cloud – will there be any size problems with this folder or is the file system ‘ephermeral’ and the container is refreshed each time the page is loaded, avoiding cache size problems?
Also, if anyone has tips or suggestions on using a different or better backend, please let me know.
Hi, @AIMPED. Thank you for the link. The post from January was new but I had read through the others before. @Emil has some great explanations and examples. (Though the GitHub dirty few example links are now broken, the inline code in the forum is still there.) I’m just wondering whether someone has experience with a Docker implementation of ServersideOutputTransform. Do the size issues of the file_system_backend occur there too, etc.
There is no difference between using the ServersideOutputTransform on the host system as compared to within a Docker image (i.e. you don’t need a different implementation). In both cases, files will accumulate (indefinitly) if you use the FileSystemBackend. Hence, unless you implement a cleanup mechanism, you will at some point you run out of disk space. Note that this is not an bug, but rather a deliberate design choice. Since different people may want to apply different cleanup routines for different use cases, the choice of cleanup mechanism is left to the developer.
In terms of how you would do the cleanup in a Docker context, you could run a separate process that does it as part of the image; or run small cron job that perform it; or you could place the cache in a mounted folder, which you cleanup from a different docker image. Pick whatever approach you like the best
Thank you for the very quick reply, Emil! I’ll look into how to do the docker cleanup per your suggestions. Do you have an opinion about the suitability of different backends for a Docker implementation of Dash Plotly using ServersideOutputTransform?
For most use cases, I would recommend to use a Redis cache. If you are unable to, e.g. due to cost and/or infrastructure limitations, the FileSystemBackend should be OK for production use, if you mount the cache from an external folder. If you don’t, you’ll break all running apps if the pod reboots (since the cache would then be lost). Furthermore, you’ll not be able to run more than a single pod, as their caches would become out of sync.
Thank you, Emil. I need to educate myself on Redis. If you have any tips on mounting the FileSystemBackend cache from an external folder, please let me know. I’ve only used a single container for my app thus far. I’ll start reading docs.
I’m using the ServersideOutputTransform from dash-extensions with the default backend that writes the dcc.store output to the file_system_backend folder. While developing my app locally, I’ve noticed that the file_system_backend folder grows in size over time, and I plan to set up a job to clean it periodically. However, I’m uncertain about the behavior of this folder when my app is deployed in a docker container in the cloud. Will there be any potential size problems with the file_system_backend folder, or is the file system considered ‘ephemeral’ in a containerized environment, refreshing with each page load and avoiding cache size issues? Regards
It depends on you docker setup. If you write to a folder inside the container, you will run out of disk space at some point. Hence, you’ll need to run the cleanup routine as you do now; or restart the container regularly, which would also clear the cache.
