I am trying to come up with a workaround for the Heroku H12 30 second timeout.
My Dash application has a process that reads a number of files from an FTP site. I have the code written so that one callback reads all PNs and returns a data frame but that takes over 30 seconds causing an H12 error when deployed on Heroku.
I am trying to break this into two callbacks since each individual file read is less than 30 seconds.
- callback 1 : creates a list of all files to be read from FTP site
- callback 2 : takes in the list of files, downloads the first file from the list from the FTP site, returns the list of files with the downloaded file removed, repeats until all files are read
@app.callback(Output('files', 'children'), [Input('pn-dropdown', 'value')]) def get_file_list_from_ftp(part_number): get_files_matching_part_number(part_number) # code that reads the name of files that match the partnumber return file_name_string # comma separated list of file names
@app.callback(Output('files', 'children'), [Input('files', 'children')]) def read_file_from_ftp(file_name_string): file_name_list = file_name_string.split(',') if len(file_name_list) == 0: raise PreventUpdate else: file_name = file_name_list.pop(0) process_ftp_file(file_name) # function reads ftp file and appends to a data frame new_file_name_list = ','.join(file_name_list) return new_file_name_list # list is one file less than last itteration
When I run a program with this structure I do not get any errors but the program “locks up” and doesn’t process anything.
Any suggestions for how to resolve my problem without using something like redis and celery?