Modify a dataframe across uwsgi workers

Dash Python
1

Hi all,

Some context first:

I’m using Dash to make a simple annotation app. The user is presented with the contours of bacteria images.

a vue of the app
Capture du 2017-10-03 16-04-49.png1782×732 50.2 KB

Upon clicking on a contour, it assigns it the currently selected phenotype. This is stored as a phenoype column in a pandas dataframe.
Once the contour is clicked, all the contours are re-drawn, changing color based on that phenotype value.

The issue

This works fine locally, but in a container with nginx+flask+uwsgi, I have multiple processes (uswgi workers) between which the dataframe is not shared, so the modification is only effective for the process that received the callback.

The question

How can I make sure that my data is correctly synchronized between processes in the app?

Hope I’m clear,

Thanks for the good work!

2

Great question, and really sweet app! Would love to see this if it’s publically available :slight_smile:

So in Dash, it is not safe to modify any variable that is outside of the scope of the function because, as you mention:

Instead, you’ll have to store this data in an intermediate component or perform the computation on every callback (add the click event to every callback).

Here’s a simple example showing how to store data in an intermediate component (a hidden div):

import dash
from dash.dependencies import Input, Output
import dash_html_components as html
import dash_core_components as dcc

import pandas as pd

df = pd.DataFrame({
    'x': [1, 2, 3, 4, 5, 6],
    'y': [3, 1, 2, 3, 5, 6],
    'z': ['A', 'A', 'B', 'B', 'C', 'C']
})

app = dash.Dash()

app.layout = html.Div([
    dcc.Dropdown(
        id='dropdown',
        options=[{'label': i, 'value': i} for i in ['A', 'B', 'C']],
        value='A'
    ),

    dcc.Graph(
        id='graph'
    ),

    html.Div(id='cache', style={'display': 'none'})
])


@app.callback(Output('cache', 'children'), [Input('dropdown', 'value')])
def update_cache(value):
    filtered_df = df[df['z'] == value]
    return filtered_df.to_json()


@app.callback(Output('graph', 'figure'), [Input('cache', 'children')])
def update_graph(cached_data):
    filtered_df = pd.read_json(cached_data)
    return {
        'data': [{
            'x': filtered_df['x'],
            'y': filtered_df['y'],
            'type': 'bar'
        }]
    }


if __name__ == '__main__':
    app.run_server(debug=True)
3

Hey @chriddyp thanks for the swift answer. I was just looking into hidden Divs :smiley: (after wide searches on parallel computation, and trying to understand data management with uwsgi workers :-/ ). I’ll refactor the code to make the JSON serialized data as small as possible - might be good for the overall architecture of the app anyway.

As for the publicity of the app, that’s not up to me, but I try to push my collaborators this way :wink:

Oh while I’m at it, in one of my attemps, I sub classed dash.Dash to embark a container (with the image, the contours and other data). Is this a good idea, at least for the read only parts? It simplifies the code a bit.