Time consuming initial background process

I need to load a set of large set of cPickle or json files in the beginning. Then I want to keep those values in main memory in variables. I need a set of suggestion regarding this:

  1. How can I show verbose output to the user, for example file 1 loaded, now loading file 2, etc?
  2. How can I load them just once and use them for any instances without reloading again?

Regarding showing verbose output to the user, you could try having an html.Div component and a dcc.Interval like this:

        "Upload status",
            interval=1 * 1000,

Then attach a callback to the interval that updates the status:

    Output('status-display', 'children'),
    [Input('update-status-interval', 'n_intervals')]
def update(_):
    ... # get the files that have been uploaded 

Hope this helps!

1 Like

I just found that, if something is being processed in another callback, then the callback for the verbose output does not work. I thought it will work like multi-thread.

My case was like this:
User upload a file and I display it in table and dynamically generate a button.
When the button is pressed, I take I process the input table. While processing the table, I wanted to show the current status of the process with some verbose output periodically. Unfortunately, with the above code, I only get the verbose output after completing the the full process.