Loading pandas dataframe into Data table through a callback

Dash Python
1

So, I have been excited by the continued improvement of Data Tables, but despite looking through many threads around here as well as guides, I can’t figure out the proper way to simply import a pandas dataframe into a data table.

Most examples illustrate how to manually pick certain columns/rows taken from a dataframe that is hardcoded within the example, and display that, but I have a dataframe that needs to be created via a callback, which will then be sent to dash_table.DataTable().

How can I make this work? Using the references, I’ve tried the following code to send a dict of my dataframe, but nothing displays.

## Imports
import dash
import dash_core_components as dcc
import dash_html_components as html
import dash_table

from dash.dependencies import Input, Output, State

import datetime as dt
import pandas as pd
import numpy as np
from twitter_functions import id_extractor, old_tweets, continuous_stream

app = dash.Dash(dev_tools_hot_reload=True)
app.scripts.config.serve_locally = True
app.config['suppress_callback_exceptions'] = True


app.layout = html.Div(children=[

    html.H3('Twitter App'),
    
    dcc.Input('ScreenName_Input', type='text'),
    
    html.Button(id='screenNames_submit_button', children='Submit'),
    
    dash_table.DataTable(id='tweet_table')
    
])
                      
@app.callback(
    Output(component_id='tweet_table', component_property='data'),
    [Input(component_id='screenNames_submit_button', component_property='n_clicks_timestamp')],
    [State(component_id='ScreenName_Input', component_property='value')]
)
def display_tweets(submit_button, screen_names):
    tweets = old_tweets(screen_names)
                      
    return tweets.to_dict(orient='records')

if __name__ == '__main__':
    app.run_server(debug=True)

How can I do this?

2

If I’m not mistaken, you may have to define the DataTable with the columns you want and an empty list for data i.e.:

dash_table.DataTable(
    id='computed-table',
    columns=[
    {'name': 'Column 1', 'id': 'column1'},
    {'name': 'Column 2', 'id': 'column2'},
    {'name': 'Column 3', 'id': 'column3'},
    {'name': 'Column 4', 'id': 'column4'},
    {'name': 'Column 5', 'id': 'column5'},
    ...
                ],
    data=[],
    ...

where obviously the structure of you DataTable in declaration matches the df you are trying to display

2 Likes
3

This ended up working. Thank you so much, sterlingbutters; you saved a fellow coder in a great time of need.

Plotly really needs to do a better job of documenting these bedrock-basic features. I think I’ll make a forum post at some point documenting all the little tricks I’ve learned over the last couple months, in the hopes possibly they’ll help others like me in the future.

3 Likes
4

yup, only way i figured it out was because I had two DataTables that were exactly the same except that was the only difference haha so I guess you could say I got lucky

1 Like
5

Thanks, this however seems to put DataTable virtually useless for me…

6

@markus_zhang While it may be a pain, you could likely create a function that parses your DataFrame to determine the shape of your data to create the list of dictionaries that occupy the columns property. Sure, it’d be nice if DataTable did everything for you but it could be a temporary solution for you. There could also be some potential issues depending on how “dynamic” your app is.

1 Like
7

Thanks, this is what I’m doing at the moment. Basically I just save one row of data from SQL Server and parse the columns into the table. I’m new to Dash so I find it a bit awkard to pass objects around. I wish I could do everything with a controller program as a communication centre.

8

Dash has a bit of a learning curve but it gets really fun after awhile

1 Like
9

Yeah it’s extremely versatile. I know nothing about web dev and basically am leveraging the power of Dash to dev quick solutions for the other teams. When they ask I just say “Ask Dash, I know nothing…” lol

10

Its a game-changer no doubt - Even if you knew web dev, it’d be a real pain to inject python in there. Probably one of the best tools for the scientific data and computing community

1 Like
11

@sterlingbutters So do I understand it right. I have to define the columns before I really know what columns I get from a callback?

12

@Varlor Correct. In your callback you can likely determine the shape of your data you wish for DataTable to display and then return dcc.DataTable to the children property of a Div element. The point is that it doesn’t appear that DataTable is able to create it’s columns dynamically based on the data, this must be specified prior to loading the data into the table with a callback.

13

But what I tested now was to pass data and columns as output in the callback functions to the datatable and this worked for me! But now my data seems to large and the pagiation thing do not work for me. But if i shorten the dataframe to lets say 10 columns it works fine now!

14

@sterlingbutters I did as you suggestet and defined the columns in my layout and and empty list for my data. I pass the actual data trough a callback. I can see the datatable with the columns now, but they remain empty. I used the the same data without a callback and it worked perfectly. I just can’t figure it out. What do I need to pass to my callback return statement?

16

@kna Would you mind sharing your code?

17

hey, here’s a fully working example that works for me, using Upload component.

import dash
import dash_table
import pandas as pd
import dash_core_components as dcc
import dash_html_components as html

app = dash.Dash()
app.css.config.serve_locally = True
app.scripts.config.serve_locally = True

app.layout = html.Div([
	dcc.Upload(
		id='upload-data',
		children=html.Div([
			'Drag and Drop or ',
			html.A('Select Files')
		]),
		style={
			'width': '100%',
			'height': '60px',
			'lineHeight': '60px',
			'borderWidth': '1px',
			'borderStyle': 'dashed',
			'borderRadius': '5px',
			'textAlign': 'center',
			'margin': '10px'
		},
		multiple=False
	),
	html.Div(id='output-data-upload'),
])

@app.callback(dash.dependencies.Output('output-data-upload', 'children'),
			 [dash.dependencies.Input('upload-data', 'contents'),
			  dash.dependencies.Input('upload-data', 'filename')])
def update_output(contents, filename):
	if contents is not None:
		df = pd.read_csv(filename, sep='\t')
		return html.Div([
			dash_table.DataTable(
				id='table',
				columns=[{"name": i, "id": i} for i in df.columns],
				data=df.to_dict("rows"),
				style_cell={'width': '300px',
				'height': '60px',
				'textAlign': 'left'})
			])
1 Like
18

Thank you for your help and the additional code. Turns out it was a mistake in my dropdown menu options. I now testet your code and it works perfectly.

19

that code hugely helped. i don’t know why, but, only the files in the same directory are being displayed(“file not found” error for files that are not in local directory).

what i think is happening, is the file that’s being dragged/chosen is parsed as “file_name.csv”(which works fine for files in local directory) instead of “C:\Users* ** \file_name.csv”. and that “file not found” error will not persist only if full path will be taken.

how do i fix this @simonb?

20

how to return two dataframes as data tables in same call back function?