Recommendations on handling user session data?

[ALaughingHorse]

Hi all,

I am trying to build a Plan-Maker application with Dash. Basically the app does three things:

1. User can input their plans/goals for a certain time period (workout 10 times in the next 3 weeks, etc)
2. Every day when the user log-in to the app, the app will show what was planned for this day and provide a checklist-like interface
3. Users can also see a summary of how many tasks are planned/completed etc.

I am using the app myself as of now on the localhost. Thus I have the data output into folders in my local file system, pointing the app to the folder every time it starts.

if I’m to host this app on a server and eventually open to other users, what would be the best way to handle the data pipeline for each user?

Any help would be much appreciated.
Thanks!

[adamschroeder]

Maybe you can use PostgreSQL as your live database where you store user information and have usernames and passwords

[ALaughingHorse]

Hi Adam,

Thanks for replying. Are there any links or examples I can follow?

Maybe I can use a cloud database instead? (GCP or AWS)?

Thanks!

[adamschroeder]

AWS is also good. I don’t have experience with that, though. I’m making a tutorial next week on “connecting your app to PostgreSQL”. I’ll let you know as soon as as I post it.

[EliasCoutinho]

I saw your post on youtube but I didn’t understand how it works, from what I saw in the videos there is a table data entry for Postgres, then the graph is updated by the table and not by Postgres.

I wonder:
In a scenario where we only display tables and graphs in the Dash, the page will not insert anything, just consume Postgres data.

1 - How would you display the data by pressing F5 on the keyboard?

2 - How would you display this data in real time if there is any modification in the Postgres database?

My problem today is that I only have this update if I stop the dash and restart it.

[adamschroeder]

hi @EliasCoutinho
That’s a pretty old video so I’m not sure how accurate it is these days. What does F5 usually do?

You should be able to display data directly from PostgreSQL by adding the dcc.Interval which can trigger a callback you build every X amount of time. The callback would instruct the app to pull data from the postgreSQL DB.

If you’d like to display the data only if there were modifications to the database, you would need to compare between the current table and the pulled table on postgresql. If there are any differences, then you could pull it into your app.

[EliasCoutinho]

Exactly.
It even works, I get the postgres data and show the graphics, the problem is that it doesn’t update.

I just want to pull data from the Postgres database, if possible it updates the charts and tables from time to time.

I’ve been trying to do it for 2 months and I haven’t found anything that I can understand, so if you or someone in our community made a minimal example with a table only I would take it as a basis for mine.

I would be very grateful if you help us because here we don’t have examples of this problem I’m experiencing.

[adamschroeder]

hi @EliasCoutinho

I hope the following example where I connect to a Mongo database and show data in a DataTable is helpful. In this case I just had the interval activate once a week, since I didn’t want to pull data frequently, but you can change that obviously.

import dash                                                   
from dash import html, dcc, Input, Output, State, dash_table
import pandas as pd
import plotly.express as px
import pymongo   
from bson.objectid import ObjectId

# Connect to server on the cloud
client = pymongo.MongoClient("mongodb+srv://adam3:mypassword@...")


# Go into the database I created
db = client["xindustry"]
# Go into one of my database's collection (table)
collection = db["production"]

# Define Layout of App
external_stylesheets = ['https://codepen.io/chriddyp/pen/bWLwgP.css']
app = dash.Dash(__name__, external_stylesheets=external_stylesheets,
                suppress_callback_exceptions=True)

app.layout = html.Div([
    html.H1('Web Application connected to a Live Database', style={'textAlign': 'center'}),
    # interval activated once/week or when page refreshed
    dcc.Interval(id='interval_db', interval=86400000 * 7, n_intervals=0),
    html.Div(id='mongo-datatable', children=[]),

    html.Div([
        html.Div(id='pie-graph', className='five columns'),
        html.Div(id='hist-graph', className='six columns'),
    ], className='row'),
    dcc.Store(id='changed-cell')
])


# Display Datatable with data from Mongo database when page first loads
@app.callback(Output('mongo-datatable', component_property='children'),
              Input('interval_db', component_property='n_intervals')
              )
def populate_datatable(n_intervals):
    # Convert the Collection (Mongo table) data to a pandas DataFrame
    df = pd.DataFrame(list(collection.find()))
    # Convert id from ObjectId to string so it can be read by DataTable
    df['_id'] = df['_id'].astype(str)
    print(df.head(20))

    return [
        dash_table.DataTable(
            id='our-table',
            data=df.to_dict('records'),
            columns=[{'id': p, 'name': p, 'editable': False} if p == '_id'
                     else {'id': p, 'name': p, 'editable': True}
                     for p in df],
        ),
    ]


# store the row id and column id of the cell that was updated (from the Dash DataTable built above)
app.clientside_callback(
    """
    function (input,oldinput) {
        if (oldinput != null) {
            if(JSON.stringify(input) != JSON.stringify(oldinput)) {
                for (i in Object.keys(input)) {
                    newArray = Object.values(input[i])
                    oldArray = Object.values(oldinput[i])
                    if (JSON.stringify(newArray) != JSON.stringify(oldArray)) {
                        entNew = Object.entries(input[i])
                        entOld = Object.entries(oldinput[i])
                        for (const j in entNew) {
                            if (entNew[j][1] != entOld[j][1]) {
                                changeRef = [i, entNew[j][0]] 
                                break        
                            }
                        }
                    }
                }
            }
            return changeRef
        }
    }    
    """,
    Output('changed-cell', 'data'),
    Input('our-table', 'data'),
    State('our-table', 'data_previous')
)


# Update MongoDB with the new data and create the graphs
@app.callback(
    Output("pie-graph", "children"),
    Output("hist-graph", "children"),
    Input("changed-cell", "data"),
    Input("our-table", "data"),
)
def update_d(cc, tabledata):
    if cc is None:
        # Build the Plots
        pie_fig = px.pie(tabledata, values='quantity', names='day')
        hist_fig = px.histogram(tabledata, x='department', y='quantity')
    else:
        print(f'changed cell: {cc}')
        print(f'Current DataTable: {tabledata}')
        x = int(cc[0])

        # update the external MongoDB
        row_id = tabledata[x]['_id']
        col_id = cc[1]
        new_cell_data = tabledata[x][col_id]
        collection.update_one({'_id': ObjectId(row_id)},
                              {"$set": {col_id: new_cell_data}})
        # Operations guide - https://docs.mongodb.com/manual/crud/#update-operations

        pie_fig = px.pie(tabledata, values='quantity', names='day')
        hist_fig = px.histogram(tabledata, x='department', y='quantity')

    return dcc.Graph(figure=pie_fig), dcc.Graph(figure=hist_fig)


if __name__ == '__main__':
    app.run_server(debug=False)

[EliasCoutinho]

Friends, how lost I am in this concept!
I’ll paste the code as short as possible but it will be a little big.

If anyone can help me I would be very pleased. I’m really not understanding this concept.

This is the code where I want to show the content of df_data_generais (My dataframe)

from datetime import date
import plotly.graph_objs as go
import dash
from dash import html, dcc, Input, Output, State, dash_table
import dash_bootstrap_components as dbc
import plotly.express as px
import plotly.graph_objects as go
from plotly.subplots import make_subplots
import pandas as pd

from dash_bootstrap_templates import ThemeSwitchAIO

# Importei meu dataframe do arquivo movimento_geral.py
from movimento_geral import df_dados_gerais

# Formatação das tabelas
formatted = {'specifier': ',.2f', 'locale': {'group': '.', 'decimal': ',', }}

# ================================================================== #
from flask import Flask

dbc_css = "https://cdn.jsdelivr.net/gh/AnnMarieW/dash-bootstrap-templates/dbc.min.css"

server = Flask(__name__)

app = dash.Dash(__name__, server=server, suppress_callback_exceptions=True,
                external_stylesheets=[dbc.themes.BOOTSTRAP, dbc_css])

# ============================Styles================================ #
tab_card = {'height': '100%'}

main_config = {
    'hovermode': 'x unified',
    'legend': {
        'yanchor': 'top',
        'y': 0.9,
        'xanchor': 'left',
        'x': 0.1,
        'title': {'text': None},
        'font': {'color': 'white'},
        'bgcolor': 'rgba(0,0,0,0.0)'},
    'margin': {'l': 0, 'r': 0, 't': 20, 'b': 0}
}


template_theme1 = 'cyborg'
template_theme2 = 'spacelab'
url_theme1 = dbc.themes.CYBORG
url_theme2 = dbc.themes.SPACELAB


# To dict - Para salvar no dcc.store
df_store = df_dados_gerais.to_dict()

app.layout = dbc.Container([

    dcc.Store(id='store-data', data=[], storage_type='memory'), 
    dcc.Interval(
                id='interval-component',
                interval=10*1000, # in milliseconds
                n_intervals=0
            ),     

    dbc.Row([
        dbc.Col([
            dash_table.DataTable(
                id='datatable-data',
                editable=True,
                fixed_rows={'headers': True},
                
                style_cell_conditional=[
                    {
                        'if': {'column_id': 'CODIGO'},
                        'textAlign': 'left'
                    },

                    {
                        'if': {'column_id': 'CODIGO'},
                        'width': '80px'
                    },

                    {
                        'if': {'column_id': 'ESTOQUE'},
                        'width': '130px'
                    },

                    {
                        'if': {'column_id': 'UNIDADE'},
                        'width': '80px'
                    },                

                    {
                        'if': {'column_id': 'DESCRICAO'},
                        'textAlign': 'left', 'width': '230px'
                    },                

                    {
                        'if': {'column_id': 'GRUPO'},
                        'textAlign': 'left'
                    },   

                    {
                        'if': {'column_id': 'FAMILIA'},
                        'textAlign': 'left'
                    },

                    {
                        'if': {'column_id': 'UNIDADE'},
                        'textAlign': 'left'
                    },                                                      
                ],

                style_data={
                    'color': 'black',
                    'backgroundColor': 'white'
                },

                style_data_conditional=[
                    {
                        'if': {'row_index': 'odd'},
                        'backgroundColor': 'rgb(220, 220, 220)',
                    }
                ],            
                #'backgroundColor': 'white',
                
                style_header={'textAlign': 'center', 
                            'backgroundColor': 'rgb(210, 210, 210)', 
                            'color': 'black', 
                            'fontWeight': 'bold'},

                style_table={'height': '1000px', 'overflowY': 'auto'},                          

                filter_action='native',
                sort_action="native",            
                page_size=1000,
            )
        ], sm=12, lg=12)
    ], className='g-2 my-auto', style={'margin-top': '7px'})

], fluid=True, style={'height': '100vh'})


# ======== Callbacks ========== #
# Na aba TABELAS, conteúdo da tabela.
@app.callback(
    Output('datatable-data', 'data'),
    Output('datatable-data', 'columns'),
    Input('store-data', 'data')
    )
    
def update_table(d):
    print('---------------------- DATA update_table n ----------------------')
    print(d)


    mask = (df_dados_gerais['ANO'] == 2022) & (df_dados_gerais['OPERACAO'] == 'V') &  \
            (df_dados_gerais['MES'].isin(11)) & (df_dados_gerais['CANCELADO'] != '*')

    print('---------------------- update_table MASK ----------------------')
    print(mask)

    df_pivot = pd.pivot_table(
        df_dados_gerais.loc[mask], index=['CODIGO', 'DESCRICAO',
                'ESTOQUE', 'GRUPO',
                'FAMILIA', 'UNIDADE',],
        values='QUANTIDADE',
        columns='MES',
        aggfunc=sum).reset_index().fillna(0)
    
    df_pivot = df_pivot.rename(
        {1: 'JAN', 2: 'FEV', 3: 'MAR', 4: 'ABR', 5: 'MAI', 6: 'JUN', 
        7: 'JUL', 8: 'AGO', 9: 'SET', 10: 'OUT', 11: 'NOV', 12: 'DEZ'}, axis=1)
    
    cols = []

    textCols = ['CODIGO', 'DESCRICAO', 'GRUPO', 'FAMILIA', 'UNIDADE']

    for i in df_pivot.columns:
        if i not in textCols:
            cols.append({"name": str(i),
                    "id": str(i),
                    "type": "numeric",
                    "format": formatted})
        else:
            cols.append({"name": str(i),
                        "id": str(i),
                        "type": "text"})



    return df_pivot.to_dict('records'), cols

# Load data from PostgreSQL, you have to change database name, user and password.
@app.callback(Output('store-data', 'data'),
            Input('interval-component', 'n_intervals'))
def update_data(n_intervals):

    print(f'---------------------- FUNÇÃO update_data {n_intervals} ----------------------')
    #print(df_dados_gerais.to_dict())
    
    return df_dados_gerais.to_dict()

if __name__ == '__main__':
    app.run_server(debug=True)

What I intend to do is read this data, store it in
dcc.Store(id=‘store-data’, data=, storage_type=‘memory’),

And display in the table.

Every 10 seconds update dcc.Store and table