I would like to implement a single user-app that allows the user to interact with quite big data (several GB). Since part of the interaction would be showing graphs, I am considering using Plotly dash.
However, since the data is really big and only a single user is using the app, I would rather like the app to avoid all the clumsyness that is necessary for making it stateless, all the caching mechanisms would require some sort of (slow) serialization or complex conversion.
So – if it is guaranteed that only one user is using a Dash app and the app is run using a single worker, would it be safe to simply have all my big data in global variables? I know how this would break if we have more than one worker running for more than one user, but is it also possible for this to break with a single worker and a single user?
I don’t think you need it to be stateful.
Giving it multiple workers could allow for quick and snappy responses regardless of other callbacks taking time.
Using global variables in the way you described is fine and will improve the speed for sure. You can design the callbacks to always use the global variable as well.
But if you are manipulating the global variable, then you would only use the one worker. However, it should be noted that in doing so. Anytime you reload the app, all your changes would be lost.
I am not sure I understand your answer: using a global variable WILL make the app stateful as the global variable is the state, no?
I guess whether this will work as intended with a single worker depends on the way how the underlying web server (flask?) manages a single worker process: if it is indeed one process that stays the same all the time, I guess it should work, but is that guaranteed? (I am not sure what guarantees Dash makes about that underlying server process(es), I suppose in theory there would not even have to be a guarantee for using a specific server framework unless explicitly stated?)
The global variable will be manipulated if there is just the one worker. Basically you are just emulating running it from your python ides console for testing.
The problem with this is that if the server restarts for any reason, these variable changes will be reset.
You start with df. You alter df to be filtered, server resets, df will be reset to the initial load.
OK - that is exactly what I was worrying about - but will the server reset? Why would it reset?
I am not sure if this is something that is likely, unlikely, impossible to happen under normal circumstances?
It’s not likely, under normal circumstances, but say something like a power outage, the server becomes unresponsive for some reason, etc.
Or you want to change something about the app, this also would request a reset.
I don’t know if you will be manipulating the variable though, if you are not, then you shouldn’t have anything to worry about. Since it operates stateless, in the next callback after a reset, it should load everything the right way.
Thanks! - I guess I can live with that.
Yes, the idea would be to do some massive changing manipulation of the global data, e.g. generate derived data and the like so it is essential for it to remain intact while the user interacts with the process.
In actual fact, the “server” would really be the same machine on which the user interacts with the app. Basically, I want to use Dash like a local GUI framework.
Using dash that way should work nicely.
There are also some nice libraries, dash mantine and dash bootstrap that make the process of creating user interfaces easier.