General Question About Figure Loading Times

[etonblue]

I have a fairly complex dashboard with nine pages. Each page has a collection of charts and tables. For example, page (1) has seven line charts (px.line), three bar charts (px.bar), four grouped bar charts (px.bar) and seven tables (dataTable). Page (2) has up to thirteen tables and six stacked bar charts (px.bar). The other pages are all somewhere in between.

Selections are made through a series of dropdowns. Data is stored in SQLite. I am using plotly 5.6.0 and dash 2.10.2. And the app is running locally.

My observation is that each time someone logs into the app for the first time, the very first fig, regardless of page, takes between 3-5 sections to load (timed with the ‘time’ import).

So if I start the app and open Page (1) (the one with 14 figs), the first of the seven line charts takes 3.5 seconds to load. All of the other 13 figs take between .04 and .08 seconds to load.

When I start the app and open Page (2) (the one with 6 stacked bar charts), the first stacked bar chart takes 3.4 seconds to load. The other 5 stacked bar charts take a combined .3 seconds to load.

Once that first page is loaded, while faster, the loading times still feel slow. Changing a dropdown on Page (1) still takes between 2.5 - 3 seconds and Page (2) takes around 1 - 2 seconds.

It isn’t loading or processing the data either. Even at it’s slowest, the data doesn’t take more than .3s to load and process. The dataTables are also very fast.

Is this par for the course? Does dash/plotly perform some sort of long initialization whenever the first figure is loaded in an app? And are my ‘post-load’ rendering times typical? If this all isn’t working as intended, i’d be happy to post some code to see what might be causing this.

[jinnyzor]

Hello @etonblue,

I believe that you are actually running into the browser needing to download the JS in order to render the figures.

There are tricks that you can use to help the process along, browser caching, flask caching, so that the browser can potentially load it quicker next time.

[etonblue]

Oho! Sounds interesting. I reviewed the Dash Performance Page but didn’t see much about caching. Any examples or references would be appreciated.

I’m not the most sophisticated browser console guy, but I took a look at the ‘Network’ tools (using chrome) and the delay almost always shows up as ‘_dash-update-component’.

[AIMPED]

Hi @etonblue I experiemnted a bit with chaching a while ago, but I’m pretty sure that this experiment is not going to help you in your case. I’ll leave it here anyways, just in case:

The forum topic:

Multipage App with different datasets for each page

Hello Dash-Plotly community friends: I would need some guidance with my issue with dash multipage with different datasets for each page. It takes forever to load in the local computer. On deployment to Cloud Foundry, it justs displays “Loading …” forever. Log file has no error other waiting the container to start the instance. There are 3 pages with 3 different datasets, each approximately 13MB. Datasets are cached and loaded in each page using Polars and in th end for plotting gets converte…

[etonblue]

Thanks @AIMPED. I’ll take a look. One other thing I thought of that might be a factor, I am using the new Pages functionality in tabs and subtabs and in order to get it to work, I have to enclose each layout in a function. e.g.,

def layout ():
  return html.Div( . . . . )

See: Dash Pages - Multi-Page App with Subtabs using dbc.NavLinks.

Does is matter if layout is a function instead of a constant? And if it does- is there a way to fix it?

[jinnyzor]

This shouldnt be an issue.

Also, note, that responses to request from the server get served in a first come first serve basis, so caching will have a trickle down effect.

The more things that the server doesnt have to handle, the shorter the line becomes for responses.

[etonblue]

Okay, ran the app through the Werkzeug profiler and then snakeviz. When testing on my Page (2) (the one with 6 stacked bar charts from my initial post), I get an icicle chart that looks like this:

Top Level

Seventh Level (from basedatatypes.py:4764)

The biggest three blocks without text on the left side are all validate_coerce (4.0s). The blocks on the right below ‘importers.py’ without text are, in descending order (all around 5.0s): ‘import_module’, ‘_gcd_import’ (frozen importlib._bootstrap), ‘find_and_load’ (frozen importlib._bootstrap), and ‘find_and_load_unlocked’ (frozen importlib._bootstrap), and it’s ‘frozen importlib._bootstrap’ all the way down (_find_spec, _get_spec, etc.)

Any thoughts? I’m curious about the validation and the bootstrap import. Why is the latter taking 5.0 seconds?

[jinnyzor]

Is your layout a function?

[etonblue]

Yes, because of the pages functionality. See three posts up.

[jinnyzor]

Pages functionality != layout().

When I say layout function, I mean it like this:

app.layout = layout

def layout():
    return #custom layout

[etonblue]

I’m not sure I understand the difference, but I think so. in app.py, I have some dropdowns, Navlink tabs, and the dash.page_container. Layout is:

app.layout = html.Div(
  ....
)

The first page (dash.register_page(name, path=‘/’, order=0, top_nav=True)) has:

layout = html.Div(
  ....
)

All other pages have:

def layout():
    return html.Div(
  ....
)

[etonblue]

Actually I was wrong. I changed app.py to:

app.layout = layout

def layout():
    return #custom layout

And am testing now. Is layout as a function supposed to make it faster or slower?

[jinnyzor]

Having a static one would mean that you dont have to validate anything because it would be loading the layout automatically.

But not entirely sure, haha.

[etonblue]

Well, I’ve continued to run profiles using both a static layout and a function and I’m not sure that there is much of a difference. Pages with tables only are fast. Pages with one or two charts are also quick to load. But any page with four or more charts starts to bog down and often the culprits appear to be: ‘get_validator’, ‘validate_coerce’ and finding/loading (I’m not using bootstrap, so I assume dash it loading it?).

Any last thoughts about optimization? The issue isn’t with the data, as the individual data sets are quite small.

[jinnyzor]

Hmm, ok.

Try separating your figures loading into a callback instead of the layout loading.

Once you do that, test the speed, if it still isnt improved, you are welcome to try this in a js file:

const CACHE = "pages_cache";

const { fetch: originalFetch } = window;

const stored_data = {};

function updateCache(request, response) {
    if (window.location.href.substr(0,4).toLowerCase() == 'https' || window.location.href.includes('127.0.0.1')) {
        caches.open(CACHE).then(function (cache) {
        cache.put(request, response);
        });
    } else {
        stored_data[request] = response
    }
}

async function testCache(url, that, args, cache_loc = CACHE) {
    var cachedResponse;
    if (window.location.href.substr(0,4).toLowerCase() == 'https' || window.location.href.includes('127.0.0.1')) {
        const cache = await caches.open(cache_loc)
        cachedResponse = await cache.match(url)
    } else {
        cachedResponse = stored_data[url]
    }

    // Return a cached response if we have one
    if (cachedResponse) {
        savedResponse = cachedResponse.clone()
        return savedResponse;
    }

    const result = originalFetch(that, args);

    result.then((response) => {
        if (response.ok) {
            updateCache(url, response.clone())
        }
    })
    return result
}

window.fetch = async (event, payload) => {
    var result;
    // store all pages
    if (event == '/_dash-update-component') {
        data = payload.body
        if (data.includes('_pages_store')) {
            loc = JSON.parse(data)['inputs'][0].value.toLowerCase()
            result = await testCache(loc, event, payload)
        }
    }

    if (!result) {
        result = originalFetch(event, payload)
    }

    return result
}


// sets up process for pre-loading page layouts
async function fetchHandler (url) {
    var cachedResponse;
    if (window.location.href.substr(0,4).toLowerCase() == 'https' || window.location.href.includes('127.0.0.1')) {
        const cache = await caches.open(CACHE)
        cachedResponse = await cache.match(url)
    } else {
        cachedResponse = stored_data[url]
    }

    if (!cachedResponse) {
        payload = {"output":".._pages_content.children..._pages_store.data..",
        "outputs":[{"id":"_pages_content","property":"children"},
        {"id":"_pages_store","property":"data"}],
        "inputs":[{"id":"_pages_location","property":"pathname","value": url.toLowerCase()},
        {"id":"_pages_location","property":"search","value":""}],
        "changedPropIds":["_pages_location.pathname","_pages_location.search"]}

        args = {
            "credentials": "same-origin",
            "headers": {
                "Accept": "application/json",
                "Content-Type": "application/json"
            },
            "method": "POST",
            "body": JSON.stringify(payload)
            }
            const result = originalFetch('../_dash-update-component', args);

            result.then((response) => {
                if (response.ok) {
                    updateCache(url.toLowerCase(), response.clone())
                }
        })
    }
}

async function preloadPages (preload) {
    for (y=0; y<preload.length; y++) {
        const url = preload[y]
        setTimeout(function () {fetchHandler(url)}, y*5000)
    }
}

// preloading layouts
preloadPages(['./test_page'])

Only use this if you can separate your dynamic loading data from your page’s layout.