Looking for some advice on how to structure an app I want to try.
The end result is a number of charts, which are based on data retrieved from a number of APIs. The data is to be retrieved every 24 hours and some calculations will be applied (moving averages etc).
What I want to avoid, is the data being retrieved/calculations being done every time someone loads a page. I want the data/calcs done, and charts updated at a fixed time, and then when someone loads a page the chart is instantly there.
Would I be able to get away with global variables here?
So a dcc.Interval component set to every 24 hours, and that callback retrieves data, applies calcs, and creates/updates global figures for each chart. In my layout, my dcc.Graph components would use the global figures?
The data would be static once updated i.e. the app/user does not modify the data in any other way.
On my workplace, I have separated the “collect data and calculate some outcome” scripts and the Dash visualization scripts.
Roughly speaking, I have some python scripts which calls our database, do some analysis and save results in some .txt files.
In my Dash program, I read these text files and throw the figure.
I have then used Windows Scheduler to run my “collect and analyse” scripts once every 24 hours to make sure everything is up to date.
I do this as well and it’s a fine solution. I am mostly on Linux and use crontab instead. Some further things I do:
app.layout points to a function that gets the latest data
dcc.Interval runs every n seconds but only pulls an update if the csv file(s) are no older than ~n*1.5 seconds otherwise it raises PreventUpdate. E.g. if interval is 60 seconds and hasn’t been modified in over 90 seconds then don’t update. Different values may work better for you.
I also raise Prevent update then n_intervals is None or 0 as I include the data in the layout function, but you may want to think what works best for you
Include a timestamp in my app of when the data was last calculated - just in case there’s a problem with the calculations script
As long as reading the data is fast I think this is a pretty nice solution as updates to the Dash App and updates to the calculations can be done independently of each other.
Also check out celery for running scheduled jobs & redis for caching data. A dcc.Interval could poll every 15 seconds but the scheduled task would update the underlying data on its own schedule & store the data in redis. Here’s an example: https://github.com/plotly/dash-redis-celery-periodic-updates/.
Thanks everyone. I am going to use a similar approach to what Blaceus has suggested.
I’m using Pythonanywhere and I’ve seen that it comes with a job scheduler. So I will put the “get data and do calcs” bit into a separate script that exports a csv of latest data every time it is run.