Plotly randomly fails at reading data and generates empty graphs

I’m using a multipage Dash app and sometimes the pages won’t load the graph data, it just renders an empty dcc graph, usually because it fails to read the x or y axis data, but there’s no problem with the code, it just happens randomly but all the time. Usually, the problem is solved by refreshing the main page or switching pages until one of them works, then the others will work too (not if I keep switching though).

Here’s a video showing what happens:

https://screenrec.com/share/p5PzgnOqGm

The code is rather long to post it here (but if it’s really necessary I’ll do it or a link to the repository) and it’s possible to see in the video that the code itself shouldn’t have any problem as it works as expected sometimes.

Hey @Galliard I think this is not necessarily a problem with plotly. You get error messages from pandas.

Might be related to your data or the way you do the filtering. Without not information it’ll be difficult to help you. Cheers!

I see, I’ll post the link to the repository, it’d be too much of a trouble posting all the code necessary for context here:

The grids are created on the files in the Reusable folders. The images below are headers from the xlsl files so you don’t have to download them:

I’ll assume that I should have posted the code instead and try again to get an answer to this problem because I really want to solve it, this has been happening for a long time.

So I’ll use the first page as an example:

The code in ‘Dist_notas.py’, to read the data and create the grid for page 1 app (it’s reading data from the sheet in the first screenshot on my last post):

# Selecionando os dados a serem lidos
dfraw = pd.read_excel('https://github.com/Oesterd/Dash-learning-analytics/raw/master/dados_teste.xlsx')
df = dfraw.iloc[:, 0:10]



# Formatação dos números
locale_pt_BR = """d3.formatLocale({
  "decimal": ",",
  "thousands": ".",
  "grouping": [3],
  "currency": ["R$", ""],
  "thousands": "\u00a0",
})"""



numformat = {"function": f"{locale_pt_BR}.format(',.2f')(params.value)"}


clndef = [


    {'field': 'Alunos'},

    {'field': 'Sexo'},

    {'field': 'Etnia'},

    {'field': 'Escola'},

    {'field': 'Renda (R$)',
     'filter': 'agNumberColumnFilter'},

    {'field': 'Med aluno',
     'valueFormatter': numformat,
     'filter': 'agNumberColumnFilter'},

    {'field': 'Med turma',
     'valueFormatter': numformat,
     'filter': 'agNumberColumnFilter'},

    {'field': 'Freq',
     'valueFormatter': numformat,
     'filter': 'agNumberColumnFilter'},

    {'field': 'Resultado'},

    {'field': 'Professor'}


]






dfclndef = {
    'headerClass': 'center-aligned-header',
    'cellClass': 'center-aligned-cell',
    'filter': True,
    'filterParams': {
        "alwaysShowBothConditions": True,
    },
    'floatingFilter': True,
    'suppressMenu': True,
}


grid = dag.AgGrid(
    id='grid',
    rowData=df.to_dict('records'),
    columnDefs=clndef,
    defaultColDef=dfclndef,
    dashGridOptions={'pagination': True},
)





# Opções do dropdown
nullop = 'Nenhuma'
op2 = df.columns[1:4].to_list()  # Divisão por colunas
op2.insert(0, nullop)

The code for page 1 application:

# Importando as bibliotecas
import dash, orjson
from dash import html, dcc, Input, Output, callback
import dash_ag_grid as dag


import seaborn as sns
import plotly.express as px
import plotly.graph_objects as go

import pandas as pd
import numpy as np
import openpyxl


# Iniciando o aplicativo
dash.register_page(__name__, name='Distribuição de notas', path='/')


filename = 'Reusables/Dist_notas.py'
exec(open(filename).read())




Ops = {
   'Nenhuma': ['Nenhuma', 'Sexo', 'Escola', 'Etnia'],
   'Sexo': ['Nenhuma', 'Escola', 'Etnia'],
   'Escola': ['Nenhuma', 'Sexo', 'Etnia'],
   'Etnia': ['Nenhuma', 'Sexo', 'Escola']
}



# Layout da página

layout = \
    html.Div([
        html.Div([
            grid
        ]),

        # Menus de dropdown
        html.Div([


            html.Div([
                'Escolha o eixo x:',
                dcc.Dropdown(id='dropdown11', value='Med aluno', options=['Med aluno', 'Resultado'], clearable=False)
            ]),


            html.Div([
                "Divisão por cores:",
                dcc.Dropdown(id='dropdown12', value='Nenhuma', options=['Nenhuma', 'Professor'], clearable=False),
            ]),


            html.Div([
                "Divisão por colunas:",
                dcc.Dropdown(list(Ops.keys()), 'Nenhuma', id='dropdown13', clearable=False),
            ]),


            html.Div([
                "Divisão por linhas:",
                dcc.Dropdown(id='dropdown14', value='Nenhuma', options=['Nenhuma', 'Sexo', 'Etnia', 'Escola'], clearable=False),
            ]),


            html.Div([
                "Tipo de gráfico:",
                dcc.Dropdown(id='dropdown15', value='Histograma', options=['Histograma', 'Cumulativo'], clearable=False),
            ]),


            html.Div([
                'Tipo de normalização:',
                dcc.Dropdown(id='dropdown16', value='Nenhuma', options=[
                    {'label': 'Nenhuma', 'value': 'Nenhuma'},
                    {'label': 'Porcentagem', 'value': 'percent'},
                    {'label': 'Densidade de probabilidade', 'value': 'probability density'},
                ], clearable=False)
            ], style={'width': '20%'}),

            # html.Div([
            #     "Tipo de agrupamento:",
            #     dcc.Dropdown(id='dropdown14', value='layer', options=[
            #         {'label': 'Normal', 'value': 'layer'},
            #         {'label': 'Empilhar', 'value': 'stack'}], clearable=False),
            # ])
        ], style={'display': 'flex', 'flexDirection': 'row', 'gap': 50, 'flex': 1}),

        # Gráfico
        html.Div([
                dcc.Graph(id='displot')
        ])

    ])



# -------------------------------------------------------------------------------------
# Chained callback
@callback(
    Output('dropdown14', 'options'),
    Input('dropdown13', 'value')
)


def drop_chain(drop4value):
    return [{'label': i, 'value': i} for i in Ops[drop4value]]


@callback(
    Output('dropdown14', 'value'),
    Input('dropdown14', 'options'))


def drop4init(available_options):
    return available_options[0]['value']





# -------------------------------------------------------------------------------------
@callback(
    Output(component_id='displot', component_property='figure'),
    Output(component_id='dropdown15', component_property='options'),
    Input(component_id='grid', component_property='virtualRowData'),
    Input(component_id='dropdown11', component_property='value'),
    Input(component_id='dropdown12', component_property='value'),
    Input(component_id='dropdown13', component_property='value'),
    Input(component_id='dropdown14', component_property='value'),
    Input(component_id='dropdown15', component_property='value'),
    Input(component_id='dropdown16', component_property='value'),
)



def matplot_html(rows, drop1, drop2, drop3, drop4, drop5, drop6):

## Criando o gráfico

    # Modificando os dados conforme a filtragem do usuário
    dff = pd.DataFrame(rows)


    if drop2 == 'Nenhuma':
        drop2 = None
    else:
        drop2 = f'{drop2}'


    if drop3 == 'Nenhuma':
        drop3 = None
    else:
        drop3 = f'{drop3}'


    if drop4 == 'Nenhuma':
        drop4 = None
    else:
        drop4 = f'{drop4}'

    if drop6 == 'Nenhuma':
        drop6 = None
    else:
        drop6 = f'{drop6}'


    if drop1 == 'Med aluno':
        drop5val = ['Histograma', 'Cumulativo']
    else:
        drop5val = ['Histograma']
        drop5 = 'Histograma'

    if drop5 == 'Cumulativo':
        drop5 = True
    else:
        drop5 = False




    fig = px.histogram(dff, x=drop1, color=drop2, facet_col=drop3, facet_row=drop4, cumulative=drop5, histnorm=drop6, barmode='group')


    return fig, drop5val

Note that despite this example having a kind of chained callback inside the function that changes the output and value from another dropdown based on the input in the dropdown that select the x axis data (which is where the problem lies), this problem happened with all pages as shown in the video and not all of them have any chained callback structure.

In fact, this problem has been happening since the early stages of my code, but I couldn’t figure out why and it’s probably not related with Ag Grid itself, because it also happened when I used Dash Datatable

Honestly, I did not fully understand your problem. What I might look into, is reducing the Input() on this callback. Right now it gets triggered, if the dropdown13 or dropdown14 values change. For the following callbacks, the same components and props are used for triggering the callback. I could imagine, that this leads to unwanted behavior.

I was able to narrow down the cause a little:

https://screenrec.com/share/dMALNEWGVa

  • It’s not random;
  • It’s related with the multi-page feature;
  • Pages that use the same data and grid conflict with each other, pages using different data and grid don’t conflict with each other;
  • Single page app does not display this behavior (not recorded);
  • Single page app still show the same error message (the ones shown in my first video) but are able to render the graph just fine.

So the problem is related to the initial rendering of the page, I guess the app doesn’t expect a change in the output using the same input (since pages with unrelated data and grid are fine)? I can’t think of a workaround though because I still want to make them different pages.

I’m not sure what is to be seen in the video you shared. I see figures loading correctly. Once you start clicking at a higher rate, the figures do not load anymore. You could wrap your figure in a dcc.Loading() and check if the loading gets displayed. If so, wait a bit longer until finished, then switch pages.

The speed is not the problem, I clicked at a higher rate because I knew the figures wouldn’t load so that the video wouldn’t be unnecessarily long. I already tested it many times before recording it and it’s really not about the speed of the clicks, but I recorded it again with different speed rates so that you don’t have to trust my word:

https://screenrec.com/share/VJgBRS17c4

It’s also shown in the video the behavior I mentioned in the last bullet point in my previous post.

I believe your word, why shouldn’t I :blush:

Honestly I think you should create a simple app which reproduces your points.

Keep in mind that everyone willing to help you is most likely simply overwhelmed from the amount of code to debug. Time of most users is quite limited, so try to make it as easy as possible for us to help you.

1 Like

Hello @Galliard,

To elaborate on your observations on pages.

If the same component exists inside the page, the component will only be updated vs being a completely new component. This applies even if sending completely different pages.

A way to avoid this is to utilize some sort of pattern-matching, be it the outside container, basically, something that will force dash to rerender the components you are talking about.

1 Like

I initially marked this answer as the solution but I was really investigating, the problem seems trickier than I thought.

I will start with another video, which is self-explanatory and very detailed of what the problem really is:

https://screenrec.com/share/5Hukm6yAQW

Honestly I think you should create a simple app which reproduces your points.

The simplest yet most detailed and clear app I could make had 6 pages, to complete explain everything. Here are the files of this MWE with the venv for debugging (zip was too large to upload it in my git repo):

https://www.mediafire.com/file/rw6x0jtlir0hl82/MWE_Files.zip/file

Note: I ran this app with Pycharm in Windows 10, in case there are any problems.

So to summarize:

  1. This wasn’t a problem with all Dcc components, but with Dash AG Grid;
  2. More specifically, with the virtualRowData component, when I use rowData instead it works;
  3. As it’s shown in the video, pages containing datatable are completely identical, aside from the id of the output component;
  4. The callback in the first page has a html.Div -> Children as output, the callback in the second page has a dcc.Textarea -> value as output, yet they conflict;
  5. I used buttons and States for the callbacks with DAG because of what’s written in: https://dash.plotly.com/dash-ag-grid/editing-and-callbacks, it helped specially with callbacks that used rowData, but didn’t solve the main problem;
  6. All pages use the prevent_initial_call=True parameter in the callback, the only thing it does is displaying less error messages, as far as I have seen;
  7. The first and second pages have different ids for the input of their callbacks, that’s also not the cause of the problem;
  8. The pages 1 and 2 only work when I change to a page with a different dcc component (dag to datatable), different inputs and outputs weren’t enough, even from page 1 to 4 or 2 to 3, as it’s shown in the video;
  9. It’s not that the virtualRowData return as None when this errors happens, but the whole callback fails, otherwise it would still show The virtual rowData is: in the text box;
  10. For some reason, this only happens in the third rendering of a page with DAG + VRD, the first two always work;

Ifaik I need to use virtualRowData to fetch filtered data from the AG Grid, which is something I want to do in my main project, I really tried solving this problem by myself, specially trying the 4th point, that I interpreted as the thing this:

If the same component exists inside the page, the component will only be updated vs being a completely new component. This applies even if sending completely different pages.

was talking about.

I should have asked it before but could you provide a concrete example? The 4th and 8th points show that it’s really more about a problem with DAG than with Dash itself, but I still want to use DAG in my main code.

I finally solved the problem:

I just needed to give a different id to the grid being used in each page, in this MWE page 1 grid id is ‘daggrid’ and page 2 is ‘daggrid2’ and everything worked fine, my main project was reusing the same grid in a separate file, that’s why this problem happened.

So for anyone having the same problem, don’t use the same id for Dash Ag Grids in different callbacks, specially if the input is virtualRowData.

Edit: I couldn’t understand what the previous answer was about and I didn’t ask for clarification, that’s why I went extra steps debugging the behavior, I’ll be careful next time.

Edit2: The error messages shown in the videos are unrelated to this problem, to fix this, add prevent_initial_call=True as the last line of each @callback.

1 Like

This was what my post was saying. :stuck_out_tongue_winking_eye: Glad you figured it out.

This is a dash renderer issue with some wrapper components because of how dash treats the info. This is also the case when using something like cytoscape.

1 Like