Black Lives Matter. Please consider donating to Black Girls Code today.

Possible to serialize a Dash app?

I’m using Dash to create apps on-the-fly using an API I wrote and I’ve got it all working fine, except for when I serve it using gunicorn with multiple workers.

My API creates a dash.Dash() object and loads it into an instance of werkzeug.wsgi.DispatcherMiddleware.mounts so that it is served on a new URL.

Unfortunately, each workers has its own server, so the wsgi_app.mounts dictionary is only updated for that one worker that handled the API request. The other ones don’t know about the new dash.Dash() app or from what URL it should be accessed.

My thought was that I’d fix this by modifying DispatcherMiddleware to read it’s mounts object from Redis and just keep a master version in Redis, updating it whenever a new dash.Dash() is created and reading from it each time a worker tries to serve something.

Unfortunately, I’m having trouble serializing dash.Dash() objects with dill if they include any of the objects from dash_html_components or dash_core_components, which obviously they do:

_pickle.PicklingError: Can't pickle <class 'Div'>: it's not found as builtins.Div

Also, PyCharm gives me an inspection error on all these classes:

Cannot find reference to 'Div' in __init__.py

I don’t know if those issues are related, but it seems suspicious to me.

Here’s a quick code to reproduce this:

from dash import Dash
import dash_html_components as html
from dill import dumps

dash = Dash(__name__)
dash.layout = html.Div('Dash App')

serialized = dumps(dash)

Anyone know how I might go about storing these objects? Is there some other way to make all the workers in gunicorn aware of the changes?

Using multiple worker processes in Gunicorn means that your process is forked and you wind up with separate copies of the process – as you’ve noticed. If you need to share state across these workers then using some kind of shared cache like Redis is a good approach. There’s no way to make workers synch with each other as they’re entirely separate processes.

I suspect your error in PyCharm is just a side-effect of the fact that Div etc from dash-html-components have up until recently been dynamically generated at runtime. As of dash-html-components 0.12.0, they are now pre-generated Python classes, which will make IDEs happy. This has yet to be pushed to PyPI however.

That’s interesting about the change in dash-html-components 0.12.0 because I have been doing some reading about dill and the author wrote in some thread I can’t find right now that it can’t handle dynamically generated classes. I was planning on doing some experimentation with my own fork of dash-html-components on Monday to see if I could get it to work and maybe do a PR later. I’ll try the new version on Monday, hopefully it allows serialization.

Thanks for the info!

Just for completeness here: I just confirmed that the 0.12.0rc2 version of dash-html-components allows serialization using dill.