Black Lives Matter. Please consider donating to Black Girls Code today.

Show and Tell - dash-uploader (Upload large files)

Hi all,

I had some problems with uploading large data files using Dash, and I bumped to the awesome package dash-resumable-upload (0.0.3) and the improved version (0.0.4) by github user westonkjones. I decided to build my own package based on these, and published it on GitHub at

It is also pip installable (pip install dash-uploader) . I tried to make the documentation clear, so it would be easy
for anyone using Dash to upload large data files. The size of the data file should be only limited by the hard disk drive.

A complete MWE with a callback after upload would look something like this


from pathlib import Path

import dash_uploader as du
import dash
import dash_html_components as html
from dash.dependencies import Input, Output

app = dash.Dash(__name__)

UPLOAD_FOLDER = r"C:\tmp\Uploads"
du.configure_upload(app, UPLOAD_FOLDER)

app.layout = html.Div(
    [
        html.H1('Demo'),
        html.Div(
            du.Upload(
                text='Drag and Drop files here',
                text_completed='Completed: ',
                pause_button=False,
                cancel_button=True,
                max_file_size=1800,  # 1800 Mb
                filetypes=['zip', 'rar'],
                css_id='upload-files-div',
            ),

            style={
                'textAlign': 'center',
                'width': '600px',
                'padding': '10px',
                'display': 'inline-block'
            },
        ),
        html.Div(id='callback-output')
    ],
    style={
        'textAlign': 'center',
    },

)

@app.callback(Output('callback-output', 'children'),
              [Input('upload-files-div', 'fileNames')])
def display_files(fileNames):

    if fileNames is not None:
        out = []
        for filename in fileNames:
            file = Path(UPLOAD_FOLDER) / filename
            out.append(file)
        return html.Ul([html.Li(str(x)) for x in out])
    return html.Ul(html.Li("No Files Uploaded Yet!"))

if __name__ == '__main__':
    app.run_server(debug=True)

And the resulting page will look something like this:

I have tested it with the Dash 1.11.0 and Python 3.7.2.

Hope that you like it!

2 Likes

v.0.1.1 Update

  • Bugfix: Now callback is called after every upload
  • The callback syntax is slightly different now, since now there is a isCompleted boolean flag that tells when the upload process is completed, and fileNames list (currently max length is one) that has the name(s) of the uploaded files. Seems that the fileNames itself can not be used as input, as appending files to the list did not trigger the callback.

A complete MWE with a callback on v.0.1.1 would be:

from pathlib import Path

import dash_uploader as du
import dash
import dash_html_components as html
from dash.dependencies import Input, Output, State

app = dash.Dash(__name__)

UPLOAD_FOLDER = r"C:\tmp\Uploads"
du.configure_upload(app, UPLOAD_FOLDER)

app.layout = html.Div(
    [
        html.H1('Demo'),
        html.Div(
            du.Upload(
                text='Drag and Drop files here',
                text_completed='Completed: ',
                pause_button=False,
                cancel_button=True,
                max_file_size=1800,  # 1800 Mb
                filetypes=['zip', 'rar'],
                css_id='upload-files-div',
            ),
            style={
                'textAlign': 'center',
                'width': '600px',
                'padding': '10px',
                'display': 'inline-block'
            },
        ),
        html.Div(id='callback-output')
    ],
    style={
        'textAlign': 'center',
    },
)


@app.callback(
    Output('callback-output', 'children'),
    [Input('upload-files-div', 'isCompleted')],
    [State('upload-files-div', 'fileNames')],
)
def display_files(isCompleted, fileNames):

    if not isCompleted:
        return
    if fileNames is not None:
        out = []
        for filename in fileNames:
            file = Path(UPLOAD_FOLDER) / filename
            out.append(file)
        return html.Ul([html.Li(str(x)) for x in out])
    return html.Ul(html.Li("No Files Uploaded Yet!"))


if __name__ == '__main__':
    app.run_server(debug=True)
1 Like

Sounds awesome.
First observation while trying it out: You’ve been very restrictive on the requirements which leads to downgrading all my dash packages. Is there a specific reason for this or could you open those up like e.g. dash>=1.11?

Thanks for trying out and giving feedback! You’re right the requirements on v0.1.1. were very restrictive. I changed the dash requirement to be dash>=1.1.0, but it might work on older dash versions, too. I updated the requirements in the v.0.1.2. update. I included also a progressbar:

3 Likes

Now there’s v.0.2.0, update available which includes

  • Upload folder for each file defined with a upload id, which may be defined by the user. In the example below, a simple uuid.uuid1() is used, which can be also converted back to a timestamp. This way, uploads with same filename from concurrent users will be in different folders.
  • Bugfix: Uploading file with similar name now overwrites the old file (previously, file chunks were uploaded, but never merged.)

2 Likes

:tada: dash-uploader v.0.3.0

Im am happy to announce you the v.0.3.0 release of dash-uploader!

What is new?

  • Added proper Documentation page for the package.
  • Added new @du.callback decorator for simple callback creation.
  • Added (experimental) max_files support for the du.Upload component: Now it is possible to drag multiple files to the upload component.
  • Bugfix: Working behind a proxy now possible. I.e., if app is run at
    http://myhost/myproxy , and dash instance is initiated with app = dash.Dash( __name__, requests_pathname_prefix='myproxy', )
    it is taken care out-of-the-box.
  • Bugfix: Uploading files with same filename repeatedly is now possible.

The new callback decorator @du.callback greatly simplifies (the simple) callbacks; instead of the

@app.callback(
    Output('callback-output', 'children'),
    [Input('dash-uploader', 'isCompleted')],
    [State('dash-uploader', 'fileNames'),
     State('dash-uploader', 'upload_id')],
)
def callback_on_completion(iscompleted, filenames, upload_id):
    if not iscompleted:
        return

    out = []
    if filenames is not None:
        if upload_id:
            root_folder = Path(UPLOAD_FOLDER_ROOT) / upload_id
        else:
            root_folder = Path(UPLOAD_FOLDER_ROOT)

        for filename in filenames:
            file = root_folder / filename
            out.append(file)
        return html.Ul([html.Li(str(x)) for x in out])

    return html.Div("No Files Uploaded Yet!")

you may just use

@du.callback(
    output=Output('callback-output', 'children'),
    id='dash-uploader',
)
def get_a_list(filenames):
    return html.Ul([html.Li(filenames)])
1 Like

Hi!
I’ve noticed that the uploader doesn’t work if I initialize the app with “url_base_pathname” parameter, is there any specific reason for that?
I just replaced app initialization in usage demo for this:

app = dash.Dash(__name__,
                url_base_pathname='/page/'
                )

…and it stopped working. Maybe configure_upload function should be called differently or…?