Looking for some advice on how to structure an app I want to try.
The end result is a number of charts, which are based on data retrieved from a number of APIs. The data is to be retrieved every 24 hours and some calculations will be applied (moving averages etc).
What I want to avoid, is the data being retrieved/calculations being done every time someone loads a page. I want the data/calcs done, and charts updated at a fixed time, and then when someone loads a page the chart is instantly there.
Would I be able to get away with global variables here?
So a dcc.Interval component set to every 24 hours, and that callback retrieves data, applies calcs, and creates/updates global figures for each chart. In my layout, my dcc.Graph components would use the global figures?
The data would be static once updated i.e. the app/user does not modify the data in any other way.
On my workplace, I have separated the ācollect data and calculate some outcomeā scripts and the Dash visualization scripts.
Roughly speaking, I have some python scripts which calls our database, do some analysis and save results in some .txt files.
In my Dash program, I read these text files and throw the figure.
I have then used Windows Scheduler to run my ācollect and analyseā scripts once every 24 hours to make sure everything is up to date.
Itās not the most elegant solution, but it works.
I do this as well and itās a fine solution. I am mostly on Linux and use crontab instead. Some further things I do:
app.layout points to a function that gets the latest data
dcc.Interval runs every n seconds but only pulls an update if the csv file(s) are no older than ~n*1.5 seconds otherwise it raises PreventUpdate. E.g. if interval is 60 seconds and hasnāt been modified in over 90 seconds then donāt update. Different values may work better for you.
I also raise Prevent update then n_intervals is None or 0 as I include the data in the layout function, but you may want to think what works best for you
Include a timestamp in my app of when the data was last calculated - just in case thereās a problem with the calculations script
As long as reading the data is fast I think this is a pretty nice solution as updates to the Dash App and updates to the calculations can be done independently of each other.
Also check out celery for running scheduled jobs & redis for caching data. A dcc.Interval could poll every 15 seconds but the scheduled task would update the underlying data on its own schedule & store the data in redis. Hereās an example: https://github.com/plotly/dash-redis-celery-periodic-updates/.
Thanks everyone. I am going to use a similar approach to what Blaceus has suggested.
Iām using Pythonanywhere and Iāve seen that it comes with a job scheduler. So I will put the āget data and do calcsā bit into a separate script that exports a csv of latest data every time it is run.
