Refresh dataframe on page load

Hi all,

I know everyone is probably sick of seeing this question but I just cannot get this to work and it seems like it should be so simple.

I have an app that reads data from a csv file, I just want to refresh the data on client-side page load. I have read about 25 different questions and answers and read the dash docs but as someone new to this whole setup none of it is straightforward with do this and it will work.

I have settled on the fact that I will need to use the serve.layout() - building a function to handle the refresh,

I wrapped my app from top to bottom inside this function - the data frame, the app.layout (which i renamed to layout based on another topic on here) and eventually served the app up with app.layout = serve.layout right before the app.run_server()

When I opened my app I got 32 duplicate app callback errors immediately and everytime I refreshed my webpage they multiplied 64, 96, 128 (I know I have ALOT of callback functions but by doing this it caused a multiple loop through my app.

Please please can someone assist, All I want to do is refresh my data frame on browser reload:

df = pd.read_csv(url)

Please post example code. Otherwise it is hard to help.

Hi Emil,

Sorry I just cleaned the code up and removed a massive chunk (doubt you want to edit 1200 lines…)

Here is just a sample with a csv read like I currently have it and a single dropdown contained within the bootstrap like I am using

import pandas as pd
import plotly
import plotly.express as px
import plotly.graph_objects as go

from datetime import datetime

import dash
import dash_core_components as dcc
import dash_html_components as html
import dash_bootstrap_components as dbc
from datetime import date


from dash.dependencies import Input, Output

app = dash.Dash(__name__, external_stylesheets=[dbc.themes.DARKLY],
                meta_tags=[{'name': "viewport", 'content': "width=device-width, initial-scale=1"}])

sheet_id=1
sheet_name='hello'
url = f'https://docs.google.com/spreadsheets/d/{sheet_id}/gviz/tq?tqx=out:csv&sheet={sheet_name}'

df = pd.read_csv(url)

df = df[df['CategoryName'].notna()]


df.rename(columns={'Unnamed: 8': 'Date'}, inplace=True)


app.layout = dbc.Container(
    [
        dbc.Row(
            [
                dbc.Col(

                    html.Img(
                        src=app.get_asset_url("dashly.svg"),
                        id="dashly-logo",
                    ), xs=12, sm=12, md=12, lg=4, xl=4

                ),

                dbc.Col("", xs=10, sm=10, md=10, lg=8, xl=8),

            ], justify="center",
        ),

        dbc.Row(
            [
                dbc.Col(
                    [
                        html.P(
                            "Select Date:",
                            className="control_label",
                        ),
                        dcc.DatePickerSingle(id='date_name',
                                             # date=date(2021, 7, 21),
                                             display_format='Do MMM YYYY',
                                             min_date_allowed=min(df.Date),
                                             max_date_allowed=max(df.Date),
                                             # optionHeight=35,
                                             # multi=False,
                                             # searchable=False,
                                             # search_value='',  # Most searched value
                                             placeholder='Please select a Date',
                                             clearable=True,
                                             # style={'width': "100%"},
                                             # persistence=True,
                                             # persistence_type='session',
                                             className='dash-bootstrap'
                                             ),
                    ],
                    id="cross-filter-options", xs=12, sm=12, md=12, lg=5, xl=5
                ),
            ],
            justify="center",
        ),
    ],
    id="mainContainer",
    fluid=True,
)


if __name__ == '__main__':
    app.run_server(debug=True)

If you want the data to update on pageload, you can either load them in a callback or as part of a layout function,

With your current code, the data are loaded only once (when the application starts).

Yip I see based on the app lifecycle the data will only reload once,

The problem I am faced with is how do I add them to the call back or to the Layout.

The layout option I tried yesterday and formed something like:

import pandas as pd
import plotly
import plotly.express as px
import plotly.graph_objects as go

from datetime import datetime

import dash
import dash_core_components as dcc
import dash_html_components as html
import dash_bootstrap_components as dbc
from datetime import date


from dash.dependencies import Input, Output

app = dash.Dash(__name__, external_stylesheets=[dbc.themes.DARKLY],
                meta_tags=[{'name': "viewport", 'content': "width=device-width, initial-scale=1"}])

sheet_id=1
sheet_name='hello'
url = f'https://docs.google.com/spreadsheets/d/{sheet_id}/gviz/tq?tqx=out:csv&sheet={sheet_name}'

def refresh_data():

    df = pd.read_csv(url)

    df = df[df['CategoryName'].notna()]


    df.rename(columns={'Unnamed: 8': 'Date'}, inplace=True)


    layout = dbc.Container(
        [
            dbc.Row(
                [
                    dbc.Col(

                        html.Img(
                            src=app.get_asset_url("dashly.svg"),
                            id="dashly-logo",
                        ), xs=12, sm=12, md=12, lg=4, xl=4

                    ),

                    dbc.Col("", xs=10, sm=10, md=10, lg=8, xl=8),

                ], justify="center",
            ),

            dbc.Row(
                [
                    dbc.Col(
                        [
                            html.P(
                                "Select Date:",
                                className="control_label",
                            ),
                            dcc.DatePickerSingle(id='date_name',
                                                 # date=date(2021, 7, 21),
                                                 display_format='Do MMM YYYY',
                                                 min_date_allowed=min(df.EventDate),
                                                 max_date_allowed=max(df.EventDate),
                                                 # optionHeight=35,
                                                 # multi=False,
                                                 # searchable=False,
                                                 # search_value='',  # Most searched value
                                                 placeholder='Please select a Date',
                                                 clearable=True,
                                                 # style={'width': "100%"},
                                                 # persistence=True,
                                                 # persistence_type='session',
                                                 className='dash-bootstrap'
                                                 ),
                        ],
                        id="cross-filter-options", xs=12, sm=12, md=12, lg=5, xl=5
                    ),
                ],
                justify="center",
            ),
        ],
        id="mainContainer",
        fluid=True,
    )
    return layout

app.layout = refresh_data

if __name__ == '__main__':
    app.run_server(debug=True)

The problem is this is what caused massive duplicated app callback errors (32 at a time) - Have I done anything wrong with the above that could be the reason for the duplications or is that layout correct?

Looking at the option for callback addition - from how I understand it I would need to refresh the data on each individual graph etc, something like the following:

import datetime

import dash
import dash_core_components as dcc
import dash_html_components as html
import plotly
from dash.dependencies import Input, Output

# pip install pyorbital
from pyorbital.orbital import Orbital
satellite = Orbital('TERRA')

external_stylesheets = ['https://codepen.io/chriddyp/pen/bWLwgP.css']

app = dash.Dash(__name__, external_stylesheets=external_stylesheets)
app.layout = html.Div(
    html.Div([
        html.H4('TERRA Satellite Live Feed'),
        html.Div(id='live-update-text'),
        dcc.Graph(id='live-update-graph'),
        dcc.Interval(
            id='interval-component',
            interval=1*1000, # in milliseconds
            n_intervals=0
        )
    ])
)


@app.callback(Output('live-update-text', 'children'),
              Input('interval-component', 'n_intervals'))
def update_metrics(n):
    lon, lat, alt = satellite.get_lonlatalt(datetime.datetime.now())
    style = {'padding': '5px', 'fontSize': '16px'}
    return [
        html.Span('Longitude: {0:.2f}'.format(lon), style=style),
        html.Span('Latitude: {0:.2f}'.format(lat), style=style),
        html.Span('Altitude: {0:0.2f}'.format(alt), style=style)
    ]


# Multiple components can update everytime interval gets fired.
@app.callback(Output('live-update-graph', 'figure'),
              Input('interval-component', 'n_intervals'))
def update_graph_live(n):
    satellite = Orbital('TERRA')
    data = {
        'time': [],
        'Latitude': [],
        'Longitude': [],
        'Altitude': []
    }

This is a portion taken from the Live Updates page showing that the data load is directly linked to a specific element ID - meaning I would need to have a multi-step callback calling on each element to update - this is going to cause the error of Duplicate callback ID’s because I already have elements updating each other in callbacks.

I have read multiple articles and the docs and I cannot find some sample code structure that works for my application

In you first code block, you are not registering any callbacks at all, so I don’t understand how you can get callback errors?

Sorry I removed the bulk of my app as it would be ridiculously long for a forum post.

The callbacks all work in conjunction with each other and work perfectly as the data currently loads,

However with changing my app layout code to the above and all the call backs occur afterwards I am receiving these multiple call back duplicate ID’s, almost as though it is trying to load the page again even though it is loaded already.

For example the app with callbacks took this form (This is watered down just including an example layout):

import pandas as pd
import plotly
import plotly.express as px
import plotly.graph_objects as go

from datetime import datetime

import dash
import dash_core_components as dcc
import dash_html_components as html
import dash_bootstrap_components as dbc
from datetime import date


from dash.dependencies import Input, Output

app = dash.Dash(__name__, external_stylesheets=[dbc.themes.DARKLY],
                meta_tags=[{'name': "viewport", 'content': "width=device-width, initial-scale=1"}])

sheet_id=1
sheet_name='hello'
url = f'https://docs.google.com/spreadsheets/d/{sheet_id}/gviz/tq?tqx=out:csv&sheet={sheet_name}'

def refresh_data():

    df = pd.read_csv(url)

    df = df[df['CategoryName'].notna()]


    df.rename(columns={'Unnamed: 8': 'Date'}, inplace=True)


    layout = dbc.Container(
        [
            dbc.Row(
                [
                    dbc.Col(

                        html.Img(
                            src=app.get_asset_url("dashly.svg"),
                            id="dashly-logo",
                        ), xs=12, sm=12, md=12, lg=4, xl=4

                    ),

                    dbc.Col("", xs=10, sm=10, md=10, lg=8, xl=8),

                ], justify="center",
            ),

            dbc.Row(
                [
                    dbc.Col(
                        [
                            html.P(
                                "Select Date:",
                                className="control_label",
                            ),
                            dcc.DatePickerSingle(id='date_name',
                                                 # date=date(2021, 7, 21),
                                                 display_format='Do MMM YYYY',
                                                 min_date_allowed=min(df.EventDate),
                                                 max_date_allowed=max(df.EventDate),
                                                 # optionHeight=35,
                                                 # multi=False,
                                                 # searchable=False,
                                                 # search_value='',  # Most searched value
                                                 placeholder='Please select a Date',
                                                 clearable=True,
                                                 # style={'width': "100%"},
                                                 # persistence=True,
                                                 # persistence_type='session',
                                                 className='dash-bootstrap'
                                                 ),
                        ],
                        id="cross-filter-options", xs=12, sm=12, md=12, lg=5, xl=5
                    ),
                ],
                justify="center",
            ),
        ],
        id="mainContainer",
        fluid=True,
    )

    @app.callback(
        Output('examplea', 'date'),
        Output('exampleb', 'value'),
        Output('examplec', 'value'),
        Output('exampled', 'value'),
        Input('reset_btn', 'n_clicks'))
    def reset_filters(reset_button):
        if reset_button > 0:
            return None, None, None, None
        else:
            return dash.no_update

    return layout

app.layout = refresh_data

if __name__ == '__main__':
    app.run_server(debug=True)

Your callback(s) should not be inside the layout function, but outside. Otherwise you will register the callback(s) again every time you reload the page.

Ok so if I move my callback to outside my layout function i start to get unresolved references to my df variable which stores my dataframe - this would occur on a callback which would be placed like the following:

import pandas as pd
import plotly
import plotly.express as px
import plotly.graph_objects as go

from datetime import datetime

import dash
import dash_core_components as dcc
import dash_html_components as html
import dash_bootstrap_components as dbc
from datetime import date


from dash.dependencies import Input, Output

app = dash.Dash(__name__, external_stylesheets=[dbc.themes.DARKLY],
                meta_tags=[{'name': "viewport", 'content': "width=device-width, initial-scale=1"}])

sheet_id=1
sheet_name='hello'
url = f'https://docs.google.com/spreadsheets/d/{sheet_id}/gviz/tq?tqx=out:csv&sheet={sheet_name}'

def refresh_data():

    df = pd.read_csv(url)

    df = df[df['CategoryName'].notna()]


    df.rename(columns={'Unnamed: 8': 'Date'}, inplace=True)


    layout = dbc.Container(
        [
            dbc.Row(
                [
                    dbc.Col(

                        html.Img(
                            src=app.get_asset_url("dashly.svg"),
                            id="dashly-logo",
                        ), xs=12, sm=12, md=12, lg=4, xl=4

                    ),

                    dbc.Col("", xs=10, sm=10, md=10, lg=8, xl=8),

                ], justify="center",
            ),

            dbc.Row(
                [
                    dbc.Col(
                        [
                            html.P(
                                "Select Date:",
                                className="control_label",
                            ),
                            dcc.DatePickerSingle(id='date_name',
                                                 # date=date(2021, 7, 21),
                                                 display_format='Do MMM YYYY',
                                                 min_date_allowed=min(df.EventDate),
                                                 max_date_allowed=max(df.EventDate),
                                                 # optionHeight=35,
                                                 # multi=False,
                                                 # searchable=False,
                                                 # search_value='',  # Most searched value
                                                 placeholder='Please select a Date',
                                                 clearable=True,
                                                 # style={'width': "100%"},
                                                 # persistence=True,
                                                 # persistence_type='session',
                                                 className='dash-bootstrap'
                                                 ),
                        ],
                        id="cross-filter-options", xs=12, sm=12, md=12, lg=5, xl=5
                    ),
                ],
                justify="center",
            ),
        ],
        id="mainContainer",
        fluid=True,
    )
    return layout

app.layout = refresh_data

@app.callback(
        Output('examplea', 'date'),
        Output('exampleb', 'value'),
        Output('examplec', 'value'),
        Output('exampled', 'value'),
        Input('reset_btn', 'n_clicks'))
def reset_filters(reset_button):
        if reset_button > 0:
            return None, None, None, None
        else:
            return dash.no_update

    
@app.callback(
        Output('examplee', 'options'),
        Output('examplef', 'options'),
        Output('exampleg', 'options'),
        Output('exampleh', 'options'),
        Input('examplei', 'value'),
        Input('examplej', 'value'),
        Input('examplek', 'date'))
def update_league_dropdown(examplei, examplej, examplek):
    if examplei is not None and examplej is None and examplek is None:
        dff = df[df.Date == examplei]
        return [{'label': i, 'value': i} for i in sorted(dff.examplee.unique())], \
                [{'label': i, 'value': i} for i in sorted(dff.examplef.unique())], \
                [{'label': i, 'value': i} for i in sorted(dff.exampleg.unique())],\
                [{'label': i, 'value': i} for i in sorted(dff.exampleh.unique())]
    

if __name__ == '__main__':
    app.run_server(debug=True)

You cannot use your df variable like that. You should place it in a Store and use that store as Input (or State) in the callback. You might want to take a look at the docs,

https://dash.plotly.com/sharing-data-between-callbacks

It allows you to update the visualization on an interval that you can choose. Reloading the page is an expensive operation, so I would not recommend building an app that relies on it. Unless your webpage design is dynamic, I suggest separating your static html code like in the exmaple and just use the callbacks to reload the content of the visualizations.