Fast callback - slow rendering. What is the problem?

Dash Python
1

Hi guys,

we’re developing a small dash application running on Azure and have some problems with slow rendering.

In our app we want to display a list of items with some meta information. The list is loaded in real-time from our database after clicking a button in out app.
Showing only some list items (0-40) everything works fine. Showing more than around 50 items the page refresh becomes slow. With more than 80-100 items the page is unusable.
Debugging our app we see that the callback itself is fast enough.

The list items look like the following

Row(children=[Col(children=FormGroup(children=[Checkbox(id={'role': 'leading-business-partner-checkbox', 'index': 'k-odr100114'}, checked=False, className='custom-control-input', disabled=False), Label(children='', className='form-check-label custom-control-label big-checkbox ml-5 mt-1', html_for='{"index":"k-odr100114","role":"leading-business-partner-checkbox"}')], className='custom-control custom-checkbox'), className='bg-light', width=1), Col(Row(children=[Col(children=[Div(B(children='k-xyz', id={'role': 'business-partner-id-label', 'index': 'k-xyz'})), Div(children='Related:', id={'role': 'assign-major-item-for-duplicate-label', 'index': 'k-odr100114'}), Dropdown(id={'role': 'assign-major-item-for-duplicate', 'index': 'k-xyz'}, disabled=False, options=[{'label': 'd-z', 'value': 'd-z'}, {'label': 'd-z', 'value': 'd-z'}, {'label': 'k-xyz', 'value': 'k-xyz'}])], align='center', width=2), Col(children=[Div(B('dummy ag')), Div(B('')), Div(B('')), Div(B('')), Div('dummy'), Div(['12345', ' ', 'dummy'])], align='center', width=4), Col(children=[Div([B('IBAN: '), 'dummy']), Div([B('USt-ID: '), 'dummy'])], align='center', width=3), Col(children=[FormGroup(children=Checklist(id={'role': 'business_partner_options', 'index': 'k-xyz'}, options=[{'label': 'Direkter Ansprechpartner', 'value': 'is_direct_contact', 'disabled': False}, {'label': 'Bankinformationen übernehmen', 'value': 'keep_finance_information', 'disabled': False}, {'label': 'Umsatzsteuer-ID übernehmen', 'value': 'keep_sales_tax_information', 'disabled': False}], value=[]), className='mb-1 mt-1')], align='center', width=3)], id='business_partner_meta_data', className='pt-2 pb-2')), Col(children=Button(children='Cluster ändern', id={'role': 'change-cluster-btn', 'index': 'k-xyz'}, disabled=False, style={'display': 'none'}), width=1)], align='center', className='mb-2 border border-primary rounded large-font-size')

(We use dash-bootstrap-components for generating this html)

In our most problematic case we want to display a list of 260 (!) of these html-items. This seems to be far too much for the renderer.

Is there any possibility to debug the dash renderer?
Do you see any possibility to speed up the renderer?
Is the renderer really the problem?

Thanks in advance!

2

Did you check the network overhead?

3

Hi Emil,

thanks for your response.

This ist what the traffic looks like:

Unbenannt
Unbenannt1782×393 36.3 KB

As you can see the actual callback time is abount 6s, which is long but okay (we already see some improvements here). Overall the page rendering (“Updating…”) finished after about 33s.
8.2kB are transferred.

Is that what you meant?

4

Jep, that was what I meant. Wow, that’s a lot of time… My guess is that the poor performance is due to too many React nodes. I encountered a similar problem while developing dash leaflet. Representing markers as React nodes, this got laggy at ~ 100 markers and unusable at ~ 1000. The solution was to wrap all the “small elements” (in my case markers) into a single React node (in my case a GeoJSON component).

5

Hi Emil,

thanks for your response again.

I’m not very familiar with react. Is every html-objekt in dash a react node?
Do you have an example for “wrapping” elements in a single react node? Do we need to decrease the number of our html-objects returned by our callback and so decrease the depth of our DOM?

6

As I understand, every object in your Dash layout (html elements included) is mapped into a React node. By ‘wrapping’ I mean rewriting bundling the components into fewer components in the JavaScript layer, i.e. not in Dash.

7

I’m surprised it would slow down in the order of hundreds of elements. I would expect rendering to slow down more in the order of thousands or tens of thousands.

Could you try creating a minimal reproducible example? Something simple like render 100 dbc.Row components in a loop.

8

I’m absolutly clueless.
I build a small example which works fine and is very fast.
The app only has a button. Onclick 200 dbc.Row get loaded. I see the only difference between our real app and this example that this example is very light weight. Our real app has overall more containers stecked in each other.
I try to investigate more differences.

import dash
import dash_html_components as html
import dash_core_components as dcc
import dash_bootstrap_components as dbc
from dash.dependencies import Output, Input

app = dash.Dash(__name__, title='dummy')

app.layout = html.Div(children=[
    html.Button('Click me', id='btn'),
    html.Div(id='main-container')
])

@app.callback(Output('main-container', 'children'),
    Input('btn', 'n_clicks'), prevent_initial_call=True)
def btn_clicked(n_clicks):
    container_elements = []
    for i in range(200):
        container_elements.append(dbc.Row(children=[
            dbc.Col(children=dbc.FormGroup(children=[
                dbc.Checkbox(id={'role': 'leading-business-partner-checkbox', 'index': 'x'}, checked=False, className='custom-control-input', disabled=False),
                dbc.Label(children='', className='form-check-label custom-control-label big-checkbox ml-5 mt-1',
                          html_for='{"index":"x","role":"leading-business-partner-checkbox"}')],
                className='custom-control custom-checkbox'), className='bg-light', width=1),
            dbc.Col(dbc.Row(children=[
                dbc.Col(children=[
                    html.Div(html.B(children='k-xyz', id={'role': 'business-partner-id-label', 'index': 'k-xyz'})),
                    html.Div(children='Related:', id={'role': 'assign-major-item-for-duplicate-label', 'index': 'k-odr100114'}),
                    dcc.Dropdown(id={'role': 'assign-major-item-for-duplicate', 'index': 'k-xyz'}, disabled=False,
                                 options=[{'label': 'd-z', 'value': 'd-z'},
                                          {'label': 'd-zz', 'value': 'd-zz'},
                                          {'label': 'k-xyz', 'value': 'k-xyz'}])], align='center', width=2),
                dbc.Col(children=[html.Div(html.B('dummy ag')), html.Div(html.B('')), html.Div(html.B('')), html.Div(html.B('')), html.Div('dummy'), html.Div(['12345', ' ', 'dummy'])], align='center', width=4),
                dbc.Col(children=[html.Div([html.B('IBAN: '), 'dummy']), html.Div([html.B('USt-ID: '), 'dummy'])], align='center', width=3),
                dbc.Col(children=[dbc.FormGroup(children=dbc.Checklist(id={'role': 'business_partner_options', 'index': 'k-xyz'},
                                                               options=[{'label': 'Direkter Ansprechpartner', 'value': 'is_direct_contact', 'disabled': False},
                                                                        {'label': 'Bankinformationen übernehmen', 'value': 'keep_finance_information', 'disabled': False},
                                                                        {'label': 'Umsatzsteuer-ID übernehmen', 'value': 'keep_sales_tax_information', 'disabled': False}], value=[]),
                                            className='mb-1 mt-1')], align='center', width=3)], id='business_partner_meta_data', className='pt-2 pb-2')),
            dbc.Col(children=html.Button(children='Cluster ändern', id={'role': 'change-cluster-btn', 'index': 'k-xyz'}, disabled=False, style={'display': 'none'}), width=1)],
            align='center', className='mb-2 border border-primary rounded large-font-size'))

    return container_elements


if __name__ == '__main__':
    app.run_server(host='0.0.0.0', debug=False)

Thank you all for your help!!

9

Hi guys,

i think i have another approach.
The rendering time increases with dynamic callbacks. In the following example, the list in generated on button click. In every list item there is a checkbox. For each checkbox we need a callback. This is why we use a dynamic callback (with pattern matching) here.

Is there something wrong with this code? Do we use dynamic callbacks the wrong way?

import dash
import dash_html_components as html
import dash_bootstrap_components as dbc
from dash.dependencies import Output, Input, ALL

app = dash.Dash(__name__, title='dummy')

app.layout = html.Div(children=[
    html.Button('Click me', id='btn'),
    html.Div(id='main-container')
])

@app.callback(Output('main-container', 'children'),
    Input('btn', 'n_clicks'), prevent_initial_call=True)
def btn_clicked(n_clicks):
    container_elements = []
    for i in range(150):
        container_elements.append(dbc.Row(children=[
            dbc.Col(children=dbc.FormGroup(children=[
                dbc.Checkbox(id={'role': 'leading-business-partner-checkbox', 'index': i}, checked=False, className='custom-control-input', disabled=False),
                dbc.Label(children='Item #{}'.format(i), className='form-check-label custom-control-label big-checkbox ml-5 mt-1',
                          html_for='{{"index":{},"role":"leading-business-partner-checkbox"}}'.format(i))],
                className='custom-control custom-checkbox'), className='bg-light', width=1)],
            align='center', className='mb-2 border border-primary rounded large-font-size'))

    return container_elements


@app.callback(Output({'role': 'leading-business-partner-checkbox', 'index': ALL}, 'checked'),
              Input({'role': 'leading-business-partner-checkbox', 'index': ALL}, "checked"))
def leading_business_partner_checkbox_items(items_checked):
    print('Fired')
    return items_checked


if __name__ == '__main__':
    app.run_server(host='0.0.0.0', debug=False)
10

I have experienced the same thing. @chriddyp that was one of the main reasons I was skeptical about your proposal about building reusable component in Dash. I guess the performance issues originate from the dash renderer code?

1 Like
11

Hi @CSS_Fanboy

The second callback looks odd. Why have the same input and output for items checked?

Try running this (it’s much faster). It still gets a little slow when there are more than 500, but this should be OK for your use case of about 260.


import dash
import dash_html_components as html
import dash_bootstrap_components as dbc
from dash.dependencies import Output, Input, ALL

app = dash.Dash(__name__, title="dummy")

app.layout = html.Div(
    children=[
        html.Div(id="output"),
        html.Button("Click me", id="btn"),
        html.Div(id="main-container"),
    ]
)

@app.callback(Output('main-container', 'children'),
    Input('btn', 'n_clicks'), prevent_initial_call=True)
def btn_clicked(n_clicks):
    container_elements = []
    for i in range(150):
        container_elements.append(dbc.Row(children=[
            dbc.Col(children=dbc.FormGroup(children=[
                dbc.Checkbox(id={'role': 'leading-business-partner-checkbox', 'index': i}, checked=False, className='custom-control-input', disabled=False),
                dbc.Label(children='Item #{}'.format(i), className='form-check-label custom-control-label big-checkbox ml-5 mt-1',
                          html_for='{{"index":{},"role":"leading-business-partner-checkbox"}}'.format(i))],
                className='custom-control custom-checkbox'), className='bg-light', width=1)],
            align='center', className='mb-2 border border-primary rounded large-font-size'))

    return container_elements


@app.callback(
    Output("output", "children"),
    Input({"role": "leading-business-partner-checkbox", "index": ALL}, "checked"),
)
def leading_business_partner_checkbox_items(items_checked):
    return html.Div(
        [
            f"  item# {i} is checked"
            for (i, checked) in enumerate(items_checked)
            if checked
        ]
    )


if __name__ == "__main__":
    app.run_server(host="0.0.0.0", debug=True)
1 Like
12

Hi @AnnMarieW

thanks for your demo. Actually we want to use the checked box to trigger the display of a button for each item. That means, if the checbox of a item is selected, its button will be displayed. In this case, the item checked will not be returned as your recommendation. I have tested it and it looks okay.

So do you know why it runs very slowly when the item checked is also be returned?

import dash
import dash_html_components as html
import dash_bootstrap_components as dbc
from dash.dependencies import Output, Input, ALL

app = dash.Dash(__name__, title='dummy')

app.layout = html.Div(children=[
    html.Button('Click me', id='btn'),
    html.Div(id='main-container')
])

@app.callback(Output('main-container', 'children'),
    Input('btn', 'n_clicks'), prevent_initial_call=True)
def btn_clicked(n_clicks):
    container_elements = []
    for i in range(150):
        container_elements.append(dbc.Row(children=[
            dbc.Col(children=dbc.FormGroup(children=[
                dbc.Checkbox(id={'role': 'leading-business-partner-checkbox', 'index': i}, checked=False, className='custom-control-input', disabled=False),
                dbc.Label(children='Item #{}'.format(i), className='form-check-label custom-control-label big-checkbox ml-5 mt-1',
                          html_for='{{"index":{},"role":"leading-business-partner-checkbox"}}'.format(i))],
                className='custom-control custom-checkbox'), className='bg-light', width=1),
            dbc.Col(dbc.Button("Button for item #{}".format(i), id={'role': 'show-item-btn', 'index': i},
                           style= {'display': 'none'}), width=1)],
            align='center', className='mb-2 border border-primary rounded large-font-size'))

    return container_elements


@app.callback(Output({'role': 'show-item-btn', 'index': ALL}, 'style'),
              Input({'role': 'leading-business-partner-checkbox', 'index': ALL}, "checked"))
def leading_business_partner_checkbox_items(items_checked):
    print('Fired')

    button_style_result = []
    for checked in items_checked:
        if checked:
            button_style_result.append(None)
        else:
            button_style_result.append({'display': 'none'})
    return button_style_result, items_checked


if __name__ == '__main__':
    app.run_server(host='0.0.0.0', debug=True)
13

Hi @jianqiao and @CSS_Fanboy

Try changing the ALL to MATCH in the callback. It’s much much faster. Even with 5000 items, the initial render is a little slow, but the callback to display the button has no lag:

import dash
import dash_html_components as html
import dash_bootstrap_components as dbc
from dash.dependencies import Output, Input, MATCH

app = dash.Dash(__name__, title='dummy')

app.layout = html.Div(children=[
    html.Button('Click me', id='btn'),
    html.Div(id='main-container')
])

@app.callback(Output('main-container', 'children'),
    Input('btn', 'n_clicks'), prevent_initial_call=True)
def btn_clicked(n_clicks):
    container_elements = []
    for i in range(500):
        container_elements.append(dbc.Row(children=[
            dbc.Col(children=dbc.FormGroup(children=[
                dbc.Checkbox(id={'role': 'leading-business-partner-checkbox', 'index': i}, checked=False, className='custom-control-input', disabled=False),
                dbc.Label(children='Item #{}'.format(i), className='form-check-label custom-control-label big-checkbox ml-5 mt-1',
                          html_for='{{"index":{},"role":"leading-business-partner-checkbox"}}'.format(i))],
                className='custom-control custom-checkbox'), className='bg-light', width=1),
            dbc.Col(dbc.Button("Button for item #{}".format(i), id={'role': 'show-item-btn', 'index': i},
                           style= {'display': 'none'}), width=1)],
            align='center', className='mb-2 border border-primary rounded large-font-size'))

    return container_elements


@app.callback(Output({'role': 'show-item-btn', 'index': MATCH}, 'style'),
              Input({'role': 'leading-business-partner-checkbox', 'index': MATCH}, "checked"),
              prevent_initial_call=True)
def leading_business_partner_checkbox_items(checked):
    return None if checked else {'display': 'none'}

if __name__ == "__main__":
    app.run_server(host="0.0.0.0", debug=True)
2 Likes