Dash DataTable - Updating the Number of Rows

[Stuart_Kerkhof]

Hi Guys,

I have been trying to create a DataTable with a varying number of rows, by choosing number of rows via a dropdown component. For this I used the following code:

import dash
from dash.dependencies import Input, Output, State
import dash_core_components as dcc
import dash_html_components as html
import dash_table_experiments as dt
import json
import pandas as pd
import numpy as np
import plotly
import plotly.figure_factory as ff
import plotly.graph_objs as go

app = dash.Dash()
app.scripts.config.serve_locally = True
app.css.config.serve_locally = False

DF_MCA = pd.read_csv('D:\Box Sync\My Box (907070)\Energietransitie pres\MCA_Basis.csv')

DF_MCA.loc[0:20]
DF_MCA.rename(columns={'Unnamed: 0': 'Naam Idee'}, inplace=True)

ROWS = [
    {'a': 'AA', 'b': 1},
    {'a': 'AB', 'b': 2},
    {'a': 'BB', 'b': 3},
    {'a': 'BC', 'b': 4},
    {'a': 'CC', 'b': 5},
    {'a': 'CD', 'b': 6}
]

app.layout = html.Div([

    # Page Header
    html.Div([
        html.Div([
            html.H1(
                children='MCA Tool',
                style={
                    'fontSize':60,
                    'textAlign':'center',
                    'textDirection': 'vertical',
                    'dir':'rtl'
                    },
                className='eight columns')
            ]),
        html.Div([
            html.Img(
                src = 'http://denatuurlijkestad.nl/wp-content/uploads/2016/12/RHDHV-logo-download_0.jpg',
                className='three columns')
            ]),
        ]),

    ##Dropdown menu choosing number of ideas
    html.Div([
        dcc.Dropdown(
            id = 'NrIdeas',
            options=[
                {'label':'1','value':1},
                {'label':'2','value':2},
                {'label':'3','value':3},
                {'label':'4','value':4},
                {'label':'5','value':5},
                {'label':'6','value':6},
                {'label':'7','value':7},
                {'label':'8','value':8},
                {'label':'9','value':9},
                {'label':'10','value':10}
            ],
            value=4
            )
        ]),
    
    # Match Results Grid
    html.Div([
        # Match Results Table
            html.Div(
                id='MCA-Datatable',
                className='ten columns',
            ),
        # Season Summary Table and Graph
        html.Div([
            # summary table
            html.Div(id='MCA-Totaal'),
        
        ], className="three columns")
    ], className="row"),
    html.Div([
        # graph
        html.Div(id='selected-indexes'),
        dcc.Graph(
        id='graph-MCA'
        ),
    ], className="ten columns"),
    html.Div(id='TussenTabel'),
    html.Div(id='TussenDF'),
    html.Div(
        dt.DataTable(
            id='ONZ',
            rows=[{}]),
            style={'display':'none'},
            ),
])

def make2dList(rows, cols):
    a=[]
    for row in range(rows): a += [[0]*cols]
    return a


@app.callback(Output('MCA-Datatable','children'),[Input('NrIdeas','value')])
def Resize_DF(value):
    TussenLijst1 = make2dList(value,8)
    DF_MCA_Var = pd.DataFrame(TussenLijst1)
    DF_MCA_Var.rename(columns={0: 'Naam Idee'}, inplace=True)
    DF_MCA_Var.rename(columns={1: 'Duurzaamheid (CO2-winst)'}, inplace=True)
    DF_MCA_Var.rename(columns={2: 'Betrouwbaarheid spoor (technisch)'}, inplace=True)
    DF_MCA_Var.rename(columns={3: 'Onderhoudbaarheid spoor'}, inplace=True)
    DF_MCA_Var.rename(columns={4: 'Veilig spoor (Intern & extern)'}, inplace=True)
    DF_MCA_Var.rename(columns={5: 'Imago ProRail'}, inplace=True)
    DF_MCA_Var.rename(columns={6: 'Maatschappelijke baat bij gebruik huidige infrastructuur'}, inplace=True)
    DF_MCA_Var.rename(columns={7: 'Politieke component met ministerie EZ & IM'}, inplace=True)
    RowsCount = len(DF_MCA_Var.index)

    for i in range(0,RowsCount):

        DF_MCA_Var['Naam Idee'][i]='idee '+ str(i+1)
  
    return dt.DataTable(
        rows=DF_MCA_Var.to_dict('records'),
        )

Because I created an empty DataTable object in the app.layout (as suggested in earlier posts, thanks!) This callback produces a DataTable with a varying number of rows. I would like to use the output of this table for further calculation. Because the DataTable is outputted to ‘children’ of an empty html.Div, the data can only be accessed via this ‘children’ property, instead of the much more beautiful prop ‘rows’.
It seems I am only able to access the property ‘rows’ when I output the DataTable in the callback function, to a defined DataTable in app.layout. That however, results in the ‘Error loading dependancies’.

The ‘children’ prop contains a JSON formatted version of the previous called DataTable. Python seems to think this is a dictionary, but found functions like json.read etc. can’t read the ‘dict’. It looks like this (with print(children) in cmd):

{'props': {'rows': [{'Naam Idee': 'idee 1', 'Duurzaamheid (CO2-winst)': 0, 'Betrouwbaarheid spoor (technisch)': 0, Onderhoudbaarheid spoor': 0, 'Veilig spoor (Intern & extern)': 0, 'Imago ProRail': 0, 'Maatschappelijke baat bij gebruik huidige infrastructuur': 0, 'Politieke component met ministerie EZ & IM': 0}, {'Naam Idee': 'idee 2', 'Duurzaamheid (CO2-winst)': 0, 'Betrouwbaarheid spoor (technisch)': 0, 'Onderhoudbaarheid spoor': 0, 'Veilig spoor (Intern & extern)': 0, 'Imago ProRail': 0, 'Maatschappelijke baat bij gebruik huidige infrastructuur': 0,'Politieke component met ministerie EZ & IM': 0}]},'type': 'DataTable', 'namespace': 'dash_table_experiments'}

Could someone shed some light on the ‘Error Loading Dependancies’ error? Or can someone help figure out a way to read the children prop properly?

Kind regards,

Stuart

[chriddyp]

@Stuart_Kerkhof

Could someone shed some light on the ‘Error Loading Dependancies’ error?

This is a generic error that usually corresponds to a malformed response from a callback or from app.layout. Sometimes you can learn more about this error by inspecting your terminal or your browser’s console, but not always.

@Stuart_Kerkhof

Because the DataTable is outputted to ‘children’ of an empty html.Div, the data can only be accessed via this ‘children’ property, instead of the much more beautiful prop ‘rows’.

Hm, you should be able to access the rows property even if the table is generated dynamically. Here is a simple example:

import dash
from dash.dependencies import Input, Output
import dash_html_components as html
import dash_core_components as dcc
from dash_table_experiments import DataTable
import json

app = dash.Dash()
app.config['suppress_callback_exceptions'] = True
app.layout = html.Div([
    html.Button(id='button', n_clicks=0, children='Show table'),
    html.Div(id='content'),
    html.Div(DataTable(rows=[{}]), style={'display': 'none'})
])


@app.callback(Output('content', 'children'), [Input('button', 'n_clicks')])
def display_output(n_clicks):
    if n_clicks > 0:
        return html.Div([
            html.Div(id='datatable-output'),
            DataTable(
                id='datatable',
                rows=[{'Column 1': i} for i in range(5)]
            )
        ])


@app.callback(
    Output('datatable-output', 'children'),
    [Input('datatable', 'rows')])
def update_output(rows):
    return html.Pre(
        json.dumps(rows, indent=2)
    )


if __name__ == '__main__':
    app.run_server(debug=True)

[Stuart_Kerkhof]

Hi Chris,

Thanks for the reaction. I see some mistakes I’ve made in my code. Works now!

kind regards,

Stuart

[mrose1]

Hi -

I am trying to use callbacks to dynamically change the data in the DataTable. I am able to use the example above to change the data based on dropdown fields. How would I suppress the display of the data outside of the DataTable object?

[chriddyp]

@mrose1

How would I suppress the display of the data outside of the DataTable object?

@mrose1 - This data is being displayed in the app.layout (html.Div(id='content')) and is being updated in its associated callback

@chriddyp

@app.callback(
    Output('datatable-output', 'children'),
    [Input('datatable', 'rows')])
def update_output(rows):
    return html.Pre(
        json.dumps(rows, indent=2)
    )

To remove the extra data, you can delete the element from app.layout and the associated callback.

[mrose1]

Thanks! Was able to fix my code. Appreciate the quick response!

[vonf]

Hope this is the right topic to post into… How would I use Dash to dynamically append rows into an existing table without having to pass the entire table contents to the server and then back to the webpage? For example, I might have an empty table on a webpage and want to periodically poll the server using dcc.Interval to get new data. The new data should be appended to the existing table. In traditional HTML+javascript, I would get the new data using AJAX then use table.insertRow().

Thanks!

[nedned]

Callbacks must target properties of components, so you have to target the rows property of the your DataTable, which means all you can do I, believe, is replace the entire rows property, not parts of it.

Provided you don’t add the rows property as an Input or State of the callback, you will only be getting the new data being send down in the response from the server, not as part of the request.

[vonf]

Thanks. Makes sense that I will not need to send the table data to the server, just the interval event or button click indicating I want to update the table. I see the rows property of the datatable and can send back the full replacement of the rows property with the new data plus the old data. I am new to Dash so am wondering if this is a reasonable feature request. I am trying to create a live streaming viewer that is simply a datatable where new data gets appended as new rows in the table periodically (or on-demand). Once the table is large, I would hate for a single new row entry to cause the entire table to get sent back up to the webpage. If Dash can’t do it directly, are there any recommendations (e.g. small amount of custom javascript that my Dash app talks to)? Here’s a simple app as a reference - note that it is doing it the way I would prefer to not do it.

import dash
from dash.dependencies import Input, Output, State
import dash_core_components as dcc
import dash_html_components as html
import dash_table_experiments as dt

app = dash.Dash()

app.scripts.config.serve_locally = True
app.css.config.serve_locally = True

INITIAL_ROWS = [
    {'a': 'Miami', 'b': "FL"},
    {'a': 'Redwood City', 'b': "CA"},
    {'a': 'Chicago', 'b': "IL"}
]

EXTRA_ROWS = [
    {'a': 'Naperville', 'b': "IL"},
    {'a': 'Palo Alto', 'b': "CA"}
]

app.layout = html.Div([
    dt.DataTable( rows=INITIAL_ROWS, row_selectable=False, 
                  filterable=True, sortable=True, 
                  selected_row_indices=[], id='my_cities_datatable'),
    html.Button("Get some new rows", id='get_new_row_button_id'),
    
], className="container")


@app.callback(
    Output('my_cities_datatable', 'rows'),
    [Input('get_new_row_button_id', 'n_clicks'),
     Input('get_new_row_button_id', 'id')])
def get_new_row(n_clicks, id_name):
    print "n_clicks=%s, id_name=%s" % (n_clicks, id_name)
    if n_clicks == None:
        # At initial startup, callback is called with n_clicks=None. Return INITIAL_ROWS in this case.
        return INITIAL_ROWS 
    else:
        # When n_clicks is not None, we want to add extra rows. 
        #
        # How do we return just the extra rows to the web page and have the webpage only append the extra rows.
        # I don't want to have to return the INITIAL_ROWS plus the EXTRA_ROWS.
        #
        return INITIAL_ROWS+EXTRA_ROWS # <--- this is not what I want since I want to avoid sending back INITIAL_ROWS unnecessarily.

app.css.append_css({
    "external_url": "https://codepen.io/chriddyp/pen/bWLwgP.css"
})

if __name__ == '__main__':
    app.run_server(debug=True)

[nedned]

It’s probably also worth getting confirmation with @chriddyp that my suspicion about the atomic update of the rows is correct.

How large are you talking? While it does feel like a bit unnecessary, I agree, you’re probably going to need to have a really large table before you’d actually notice the request taking any longer. Dash automatically compresses the JSON responses for one thing, and furthermore, since there’s no pagination support at the moment—so all rows of the table are always rendered–I wonder if you might run into performance issues of React trying to render a large number of rows before you’d notice increased latency on the request side. I made a Dash app recently with around 400 rows (just a handcrafted

layout not a DataTable component) that was a little sluggish to scroll through in some browsers.

Perhaps it would be better to worry about that issue if you actually run into it being a problem.

[nedned]

I although I just tried the DataTable on that 400 row CSV file and its super fast. I guess manually building a layout fragment with 400 rows starts to run into performance issues due to each element being a React component.

So I don’t have as much reason to think that the DataTable will lag before you notice latency issues with downloading. But you will still need a large number of rows… for some definition of ‘large’

[kons]

Hi Everyone,

I am just reading through here and wondering what the status of the project is to get into Beta. I would really love to use it but am not advanced enough to do so at this point, until it is stable. Thank you for your hard work!

[chriddyp]

@kons

I am just reading through here and wondering what the status of the project is to get into Beta

We’re currently looking for some commercial sponsors to help bring the project to a production-level readiness. If your organization or company is interested in sponsoring, please reach out: Get Pricing

@kons

I would really love to use it but am not advanced enough to do so at this point, until it is stable

Are there any features in particular that are missing for you? i.e. this issue with “Updating the Number of Rows”? If you are interested in other features besides the one in this thread, please open a new thread

[kons]

Thanks for the really fast reply, Chris!

My Pandas DataFrame contains rows that have some string data (Name, Title, etc) but then also each row has 8 columns of time-series data. I really want to be able to plot each row’s time series as a line inside the figure but I only see the DashTable enabling me to do Bar graphs with single 1:1 correspondence. How could I plot each row’s line graph in the figure?

[kons]

Hi Chris,

I just wanted to follow up to seek any advice you may have.

My Pandas DataFrame contains rows that have some string data (Name, Title, etc) but each row also has 8 columns of time-series data. I really want to be able to plot each row’s time series as a line inside the figure but I only see the DashTable enabling me to do Bar graphs with single 1:1 correspondence for, say Name to Population. Instead, I’m hoping to have the same 8 X-Axis (Just the name of my 8 columns) as well as the Y-Axis as lines for each of the rows (about 100 of them) and on hover we get the name. Do you have any idea how I could do that programmatically?

[chriddyp]

@kons

Do you have any idea how I could do that programmatically?

See the plotly.py graphing documentation here: Plotly Python Graphing Library. There are many examples of multi-series graphs there

[kons]

Definitely! But none in terms of systematically creating the full time-series from each row in the dataset. Do you know any examples of that by chance? Thanks, Chris.

[zhanghuanhuan]

I meet problem about the atomic update of the rows. first time updating is correct, but the next time is worry. Do you know how to solve it, can you give me some advice. I am new to that. Thank you very much.

[chriddyp]

@zhanghuanhuan

I meet problem about the atomic update of the rows. first time updating is correct, but the next time is worry. Do you know how to solve it, can you give me some advice. I am new to that. Thank you very much.

Could you create a small, reproducable example that shows your problem?