dcc.Interval doesn't update data

Dash Python
1

Hello colleagues!
I’ve been using dash for over a year, but recently decided to join you because the projects have become more complex :slightly_smiling_face:

I have a dashboard with graphs, filters, tables. I get the data from the own function “msg_data()” - this is a pandas dataframe obtained using SQL. I need to update data with a certain period without restarting the project. Also I update the DatePickeRange component. I chose a solution using dcc.Interval(), but it doesn’t work correctly.

I’ll try to give an example of the main elements of my code:

DATA = msg_data()

app.layout = dbc.Container(
    children=[
        ...
        dcc.Graph(id='graph1'),
        dcc.DatePickerRange(id='picker_range', ... ),
        dcc.Interval(id='interval-component', interval=3*60*1000)
        ...
    ]
)


@app.callback(
    Output('picker_range', 'max_date_allowed'),
    Output('picker_range', 'end_date'),
    Output('picker_range', 'start_date'),
    Input('interval-component', 'n_intervals')
)
def update_data(n):
    if n:
        global DATA
        DATA = msg_data()
    day = datetime.date(datetime.now())
    s_day = day - timedelta(days=29)
    return day, day, s_day


@app.callback(
    Output('graph1', 'figure'),
    Input('picker_range', 'end_date'),
    Input('picker_range', 'start_date')
)
def create_graph(e_day, s_day):
    df = DATA
    ...
    return figure

If I launch the project and open it in the browser, the data is updated every 3 minutes and everything is fine. But if I open the browser after the night, I see that the dates are correct in DatePickerRange, but there is no data for the new day in the graphs and tables. I wait 3 minutes and it appear.

I do not understand what the problem is.
Maybe the Interval component does not work without an observer (open browser)? How can I then trigger the component?
Perhaps the second callback (create_graph()) does not work offline?

The problem is that I have a project where I need to update data once every 24 hours and I don’t want to keep the browser open. I would also like to solve this without the windows task scheduler or Cron.

I also liked the idea of ​​using flask-caching with filesystem storage, but to do this I also need to trigger the function somehow

2

Hi @TemaTorg, could this be of any help: Live Updates?

Passing your layout as a function to make sure the layout is updated everytime to re-open your browser?

3

Hi, @jhupiterz , thank you for the idea, but this solution does not work.

In some of my projects, it takes 15 minutes to unload data from the database and this is a very long time for users. So I want to update the main dataframe once late at night.

4

I think it comes from the fact that your second callback, that updates your graph, uses the DATA that is defined on line 1 of your code. This line would only be executed when the program starts, not when the page is re-loaded.

Your DatePickerRange is being updated correctly because you redefine DATA in the associated callback, so DATA is being updated within the scope of your callback but isn’t stored anywhere.

To fix it, my guess would be that you can either:

  1. update DATA inside your second callback based on your DatePickerRange, or
  2. Store DATA from your first callback in a dcc.Store component that you can reuse as an input in your second callback.
5

Thank you for your answer.

I made an example based on your first sentence:

from dash import Dash, html, Input, Output, dcc
import dash_bootstrap_components as dbc
from datetime import datetime

DATA = datetime.now()

app = Dash(__name__)
# app.config.prevent_initial_callbacks = True
app.layout = dbc.Container([
    dbc.Row(
        html.Div(id='time_area', children='some text')
    ),
    dbc.Row(
        dcc.DatePickerRange(id='picker_range')
    ),
    dcc.Interval(id='interval-component', interval=15*1000, n_intervals=0)
])


@app.callback(
    Output('picker_range', 'end_date'),
    Input('interval-component', 'n_intervals')
)
def update_data(n):
    # I don't want to count it twice after "run"
    if n > 0:
        global DATA
        DATA = datetime.now()
    day = datetime.date(datetime.now())
    return day


@app.callback(
    Output('time_area', 'children'),
    Input('picker_range', 'end_date')
)
def create_graph(e_day):
    df = DATA
    return df


if __name__ == '__main__':
    app.run_server(debug=True)

If you launch the application, you can see the time in the upper left corner. I close the browser tab, wait a minute, open it again and see the old time until the interval works again.

I need a way to change data and save it when the application is closed. Now I’ll try dcc.Store().

6

So based on your example above, if in your second callback you replace df = DATA by DATA = datetime.now() then I think it does what you’re after, unless I didn’t get your issue.

7

Same problem :cry: After re-opening the browser tab, I see the old data.

from dash import Dash, html, Input, Output, dcc
import dash_bootstrap_components as dbc
from datetime import datetime

DATA = datetime.now()

app = Dash(__name__)
app.layout = dbc.Container([
    dbc.Row(
        html.Div(id='time_area', children='some text')
    ),
    dbc.Row(
        dcc.DatePickerRange(id='picker_range')
    ),
    dcc.Interval(id='interval-component', interval=15 * 1000, n_intervals=0),
    dcc.Store(id='store', storage_type='local')
])


@app.callback(
    Output('picker_range', 'end_date'),
    Output('store', 'data'),
    Input('interval-component', 'n_intervals')
)
def update_data(n):
    # I don't want to count it twice after "run"
    if n > 0:
        global DATA
        DATA = datetime.now()
    day = datetime.date(datetime.now())
    return day, DATA


@app.callback(
    Output('time_area', 'children'),
    Input('store', 'data')
)
def create_graph(data):
    df = data
    return df


if __name__ == '__main__':
    app.run_server(debug=True)
8

In a real working example, instead of datetime.now() in the DATA variable, I have a huge DataFrame that takes several minutes to count. Also, the create_graph() function in the working project receives about 10 Input arguments (filters). Therefore, I separated the function of updating data and the function of creating graphs and tables.

9

Hello @TemaTorg,

What is updating your df in the background? Is it the app or is there a process outside of the app that will update the df?

Is there no data in the current day until whatever procedure updates the df?

10

Hello, @jinnyzor

This data is from a PostgreSQL database and it is updated very frequently.

No( I always check DB. We get a lot of new data every day.

11

What is updating the DF?

Not the database.

The DF is how the info is getting into the app.

Assuming that you are relying on the app to do this, the data will not be refreshed until there is a user on the app.

This is because of how callbacks work, they have to respond to a request from the client’s browser. To schedule tasks to update data independent, you’ll need to use something like crontab or a supervisor that will run and update your data on a system based time.

Currently, with updating your df based upon user queries will continuously spam your db the more users you have on the app.

What I do in my own app is I have stored procedures that perform all the heavy lifting into set tables, these tables are then picked up by a crontab job that runs on my server and inserts the table data into a db that is on the same server as the app.

Then I use this local data to update the info in the app.

stored proc → crontab job to pull data local → interval based local queries to update the users data

Others mention storing these dfs into local parquet files as well, which can be quicker to convert into dfs.

2 Likes
12

Thank you!

You are right. I wanted to update data using the application and without users on the page.

Agree

I found an interesting example, maybe it will suit me:

import schedule
import time
import dash

app = dash.Dash(__name__)

def run_dash_app():
    # Define the layout of the Dash app
    app.layout = dash.html.Div( ... )
    app.run_server(debug=True)

# Run the Dash app every 10 minutes
schedule.every(10).minutes.do(run_dash_app)

while True:
    schedule.run_pending()
    time.sleep(1)
13

That could indeed work. Let me know how it goes. :smiley:

My only concern is about running the app this way is how long it takes your data to query from your db. Could introduce some significant downtime.

Also, its not really a way to go about it if you are using a dedicated web server with a supervisor, as these would use gunicorn which has multiple workers.

14

Hello

Schedule doesn’t work as I expected. I made a small test script:

import time
from dash import Dash, html
from datetime import datetime
import schedule

app = Dash(__name__)


def run_dash_app():
    print('--- run dash application ---')
    dt = datetime.now()
    print(dt)
    app.layout = html.Div(children=dt)
    app.run_server(debug=True)


schedule.every().minute.at(":30").do(run_dash_app)

while True:
    schedule.run_pending()
    time.sleep(1)

schedule
schedule730×257 19.5 KB

It works twice and that’s it. Is there any way to stop the server before starting it?

15

Since it seems that you dont want to do this systematically, here are two python scripts to get you going:

app.py

from dash import Dash, html, Input, Output, dcc
import dash_bootstrap_components as dbc
from datetime import datetime
import json

DATA = datetime.now()

app = Dash(__name__)
app.layout = dbc.Container([
    dbc.Row(
        html.Div(id='time_area', children='some text')
    ),
    dbc.Row(
        dcc.DatePickerRange(id='picker_range')
    ),
    dcc.Interval(id='interval-component', interval=15 * 1000, n_intervals=0),
    dcc.Store(id='store', storage_type='local')
])

@app.server.route('/updateDF', methods=['POST'])
def updateDF():
    global DATA
    DATA = datetime.now()
    return json.dumps({'status':'successful'})


@app.callback(
    Output('picker_range', 'end_date'),
    Output('store', 'data'),
    Input('interval-component', 'n_intervals')
)
def update_data(n):
    day = datetime.date(datetime.now())
    return day, DATA


@app.callback(
    Output('time_area', 'children'),
    Input('store', 'data')
)
def create_graph(data):
    df = data
    return df


if __name__ == '__main__':
    app.run_server(debug=True)

schedule.py

import schedule
import time
import requests

def updateDF():
    r = requests.post('http://127.0.0.1:8050/updateDF', headers={'content-type': 'application/json'})
    print(r.content)

# Update DF internally
schedule.every(30).seconds.do(updateDF)

while True:
    schedule.run_pending()

Just start both and watch the df update in the app as the other pings it every 30 seconds.

This will fail on a deployed server with multiple workers as it will only update the one worker.

2 Likes
16

Great! Very intresting solution, thank you!

Yes, it works!

How can this be fixed? Which worker exactly will change? I’ll test it tomorrow on deployed server.

1 Like
17

If you only have 1 gunicorn worker, you dont need to worry, but if you have more, it will be whichever one receives the request. There is no way to make a request to each worker.

18

Today we tested app on deployed server and it worked perfect (2 clients)! Thank you :+1:

Finally we did it using Threads:

from dash import Dash, html, Input, Output, dcc
import dash_bootstrap_components as dbc
from datetime import datetime
import json
import requests
import time
import schedule
import threading

DATA = datetime.now()

app = Dash(__name__)
app.layout = dbc.Container([
    dbc.Row(
        html.Div(id='time_area', children=DATA)
    ),
    dbc.Row(
        dcc.DatePickerRange(id='picker_range')
    ),
    dcc.Interval(id='interval-component', interval=10*1000, n_intervals=0)
])


@app.server.route('/updateDF', methods=['POST'])
def updateDF():
    global DATA
    DATA = datetime.now()
    return json.dumps({'status': 'successful', 'data': str(DATA)})


@app.callback(
    Output('picker_range', 'end_date'),
    Input('interval-component', 'n_intervals')
)
def update_data(n):
    day = datetime.date(datetime.now())
    return day


@app.callback(
    Output('time_area', 'children'),
    Input('picker_range', 'end_date')
)
def create_graph(data):
    df = DATA
    return df


def update_request():
    r = requests.post('http://127.0.0.1:8050/updateDF', headers={'content-type': 'application/json'})
    print(r.content)


schedule.every().minutes.at(":15").do(update_request)


def sched():
    while True:
        schedule.run_pending()
        time.sleep(1)


thread = threading.Thread(target=sched)
thread.start()


if __name__ == '__main__':
    app.run_server(debug=True)
2 Likes