I need to read csv from different external url that change based on a value range that the user sets. For example, the user may say 1-10 and url 1-10 are read and a csv returned. I then need to allow the user to download a single csv file created from all the downloaded csv (something like pd.concat(list)). The final and intermediate files don’t need to be stored or cached.
The limitations I have so far are:
- The final joined CSV is large so I can’t just fill out the href property
- The choice of URL to download the CSV is generated dynamically by the user (not a global df)
- I am deploying the app on Heroku and so filesytem cache and save to disk may not be the best option
My current solution has been to return a link in a callback to app.server.route. The value in the link contains a range of values which determines the number of files to download. The “long_function” reads the csv from the url and creates a single dataframe which is then returned.
This solution works BUT the issue is when a user clicks the download button there is a pause (shown here as 10 second sleep) with no indication of progress or loading. Is there a way to add a loading spinner or progress bar in this scenario? Is there another solution and maybe this is not the right course?
import dash
import dash_core_components as dcc
import dash_html_components as html
from dash.dependencies import Output, Input
import flask
import io
import time
import pandas as pd
app = dash.Dash(__name__)
def long_function(x):
# Would really be loop pinging multiple URL that deliver CSV in return [pd.read_csv(URL)]
time.sleep(10)
# Would be a df formed from multiple URL [pd.concat(li)]
return pd.DataFrame({'a': [1, 2, 3, 4]})
app.layout = html.Div([
# Value would be used to generate multiple URL paths
dcc.Dropdown(id='my-dropdown', value='default',
options=[{'label': 'Path', 'value': 'Path'}]),
# Need icon to indicate long function process
dcc.Loading(id='loading-icon',
children=[html.Div(html.A('Download', id='download-link', download='data.csv', href="", target="_blank"))])
])
@app.callback(Output('download-link', 'href'), [Input('my-dropdown', 'value')])
def update_link(value):
return '/dash/urlToDownload?value={}'.format(value)
@app.server.route('/dash/urlToDownload')
def download_csv():
value = flask.request.args.get('value')
df = long_function(value)
str_io = io.StringIO()
df.to_csv(str_io)
mem = io.BytesIO()
mem.write(str_io.getvalue().encode('utf-8'))
mem.seek(0)
str_io.close()
return flask.send_file(mem,
mimetype='text/csv',
attachment_filename='downloadFile.csv',
as_attachment=True)
if __name__ == '__main__':
app.run_server(debug=True, port=8000)```