Dash Upload Component - Decoding Large Files

Dash Python
28

Hi @skinner4dinner, @maulberto3 and apologies for a late response. I check this forum only occasionally, but now turned on the email notifications.

1. Getting dataframe out of get_a_list()

I’m not exactly sure what you want, but maybe something like this:

def get_a_list(user_id):
    stock_files = []
    folder = Path(UPLOAD_FOLDER_ROOT) / user_id

    for csv_file in folder.glob('*.csv'):
        if csv_file.is_file():
            stock_files.append(csv_file)

    # create dataframe from uploaded csvs:
    df = pd.concat((pd.read_csv(file) for file in stock_files),
                   ignore_index=True)

    return dash_table.DataTable(data=df.to_dict('records'),
                                columns=[{
                                    'name': i,
                                    'id': i
                                } for i in df.columns])


@du.callback(
    output=Output('callback-output', 'children'),
    id='dash-uploader',
)
def call_after_file_upload(filenames):
    return get_a_list(user_id=Path(filenames[0]).parent.name)
  • Now you can also just use pass in call_after_file_upload and call get_a_list with the user_id from callback of some another component. Note that the HTTP requests are not “checked” in any way in dash-uploader, so anyone can get anyones tables visible (by guessing the user id), if you do not create some sort of login and user authentication system yourself to your app.
  • If you want to know the user_id in the callback for du.Upload, then you may want to use the @app.callback method described here . It is more verbose but gives you a bit more control (access to user_id in callback).

2. dash-uploader on Heroku

I have never tried Heroku myself. The dash-uploader needs (currently) to have write permissions to the hard disk on the server that it is running on. Your current code just uploads to the same directory as the code is, under uploads, since you have given UPLOAD_FOLDER_ROOT = 'uploads'. Maybe test it out first, how it works. I do not know how much disk space Heroku gives.

If you want to try to improve the dash-uploader and add some other way to save the data, you can check it out. It is not too complicated to setup the development environment for dash-uploader (see here). If I would like to change the way the files are saved, I would start at the configure_upload.py.

3. About time.sleep

I did not encounter this myself, but If can be reproduced, then it might be good idea to actually create an issue for it. It could be possible to add some sort of “while file is not ready, wait” there to ensure that the file is on filesystem before callback is fired.

I hope this helps! :smiley:

-Niko