Dynamic component performance

Dash Python
1

*NOTE: A full working demo can be found here - GitHub - dales/dash-sw-demo

My dash application allows for users to dynamically add data visualizations. This is done by passing a list of “views” to the backend callback which then gets the new view appended to this list and returned to client.

This works fine for small tables and graphs but when dealing with large datasets the performance takes a huge hit since all of the current views need to be sent to the backend and then returned (this can be 10s of MB of data just to add a new empty view).

To get around this I have created a JavaScript ServiceWorker that intercepts all requests that match _dash-component-update url. It then inspects the POST payload to see if it is a request to add a new view. If it is, then it strips the current views from the payload and sends the request without them. It also waits for the response and adds them back in also adjusting the ids of the new component.

This works perfectly at the moment and is very efficient.

Can anyone think of another way to do this that is more dashy? I havent see the use of service workers anywhere but for my use case this solution works well as it kind of strips the large datasets from the callback

Any other solutions or suggestions would be appreciated. Here is the basic flow of my code

===============================

/assets/sw/sw_register.js

window.addEventListener('load', () => {
  if (!('serviceWorker' in navigator)) {
    // service workers not supported 😣
    return
  }

  navigator.serviceWorker.register('/service_worker', {scope: '/'}).then(
    () => {
     console.log("Service Worker Registered Successfully 👍")
    },
    err => {
      console.error('Service Worker registration failed! 😱', err)
    }
  )
})

*Needs to be on root scope to get access to /_dash-update-component requests

To load the service worker from file add the following view to your dash app

app.py

@app_flask.route("/service_worker")
def service_worker():
    file_name = "serviceworker.js"
    return send_file(file_name)

serviceworker.js

self.addEventListener("install", event => {
    console.log("Service worker installed 👍");
});

self.addEventListener("activate", event => {
    event.waitUntil(clients.claim());
    console.log("Service worker activated 👍");
});

self.addEventListener("fetch", function(event) {
    if (event.request.url.includes("_dash-update-component")) {
        return event.respondWith(remove_data_obj(event.request));
    }
});

async function remove_data_obj(initialRequest) {
    const request = await initialRequest.clone();
    const payload = await request.json();
    // Change the payload being sent
   ......
   response = await fetch(destURL, {
        method: request.method,
        headers,
        body: JSON.stringify(Object.assign(payload))
    });
   // Adjust the response payload
   ....
   return new Response(JSON.stringify(fixedJson), {
                headers: response.headers
            });

My magic happens inside the remove_data_obj function

If you look in the network tab you can see the service worker now handles the fetch

2 Likes
2

Love the example of how to use service workers in that way :slight_smile:
In a more dashy way you could probably work with pattern-matching callbacks so that each view is handled individually without having to pass all the others.

3

If I use pattern matching how can I think update the list of views in my frontend? The user can add several of the same views to the list then adjust each individually. They can even use drag and drop to reorder them which is easy if they are all in a single container as children

4

Here is some feedback from @alexcjohnson

Neat approach :slightly_smiling_face: That’s essentially what request_pre and request_post were created to do Adding CSS & JS and Overriding the Page-Load Template | Dash for Python Documentation | Plotly

but we haven’t really promoted their use, as they’re pretty awkward. What would make this kind of use case much easier is an API for list operations & other partial updates.

In some cases you can do something with clientside callbacks to get a similar result - ie have a callback sending minimal info to and from the back end, then fit the new pieces into the larger data structure all in the browser. I suspect there’s a problem with that for this particular use case, that we don’t currently allow clientside callbacks to create or destroy components, just update the props of existing components. That limitation wouldn’t be particularly difficult to remove.

5

With the pre and post option how would I retain context in the post to get the data I removed I’m the pre? Besides storing in memory

6

Even though my code works, I have reverted to use the library dash-extensions by @Emil.
The partial add and pop is exactly what i needed without my team needing to understand service workers.

But perhaps there is still room for http interceptors in the future that i might use

1 Like