Shapes and annotations are not pruned / cleared when new figure returned to Graph

Very simply, if I send a complete new figure to dcc.Graph, and the new one has fewer annotations or shapes in it, old items will be left displaying in the graph.

My guess is this is based on some diff’ing behavior of plotly.react, and it assumes I’m only sending the elements of the list that I want to overwrite, so it should keep the rest?

Is there some flag or metadata or terminator value needed to tell the front end "I know i only sent you 2 shapes. Please display only those shapes, and not also items 3 through end of the old list?

Thanks,

This sounds right, it also sounds like a bug! Could you create a very small example to reproduce this behaviour? If we can reproduce, we’ll create in issue in the plotly.js repository.

Many thanks as always for reporting!

@chriddyp Here you go:
Notably, in constructing this I found that the problem only exists when Graph.animate = True, which would be consistent with the idea of diff’ing behavior being the cause.

import json
from textwrap import dedent as d
from copy import copy

import dash
import dash_core_components as dcc
import dash_html_components as html
from dash.dependencies import Input, Output, State
import plotly.graph_objs as go

app = dash.Dash(__name__)
app.css.append_css({'external_url': 'https://codepen.io/chriddyp/pen/bWLwgP.css'})
styles = {
    'pre': {
        'border': 'thin lightgrey solid',
        'overflowX': 'scroll'
    }
}

black_square = dict(
    xref = 'x',
    yref = 'y',
    xsizemode = 'pixel',
    ysizemode = 'pixel',
    type = 'path',
    layer = 'below',
    line = {'width':'5'},
    fillcolor = 'rgb(0,0,0)',
    path = 'M 0 0 L 20 0 L 20 20 L 0 20 Z',
    xanchor = 0,
    yanchor = 5,
    )
down_arrow = dict(
    x= 0,
    y= 6,
    xref='x',
    yref='y',
    showarrow=True,
    arrowhead=1,
    arrowcolor='rgb(200,0,100)',
    ax=0,
    ay= 6.5,
    axref='x',
    ayref='y'
)

initial_shapes = []
for x_pos in range(10):
    black_square['xanchor'] = x_pos
    initial_shapes.append(copy(black_square))
initial_annotations = []
for x_pos in range(10):
    down_arrow['x'] = x_pos
    down_arrow['ax'] = x_pos
    initial_annotations.append(copy(down_arrow))

app.layout = html.Div([
    html.Button('reduce and shift', id='do-it'),
    dcc.Graph(
        animate = True,
        id='shapes-and-annotations',
        figure={
            'data': [{
                    'x': [0, 20],
                    'y': [0, 6],
                    'name': 'Trace 1',
                    'mode': 'markers',
                    'marker': {'size': 10}
                }],
            'layout': go.Layout(
                title = 'Figure title bar',
                titlefont = dict(size=16),
                showlegend = False,
                hovermode = 'closest',
                margin = dict(b=0,l=0,r=0,t=0),
                annotations = initial_annotations,
                shapes = initial_shapes,
                autosize = True,
                dragmode = 'pan',
                xaxis = dict(showgrid=False, zeroline=True, showticklabels=True),
                yaxis = dict(showgrid=False, zeroline=True, showticklabels=True)
            ),
        }
    ),
    html.Div(className='row', children=[
        html.Div([
            dcc.Markdown(d("""
                **Count**

                Current number of shapes according to the application.
            """)),
            html.P(id='output-one')
        ], className='three columns'),

        html.Div([
            dcc.Markdown(d("""
                **Shapes data**

                contents of Graph['figure']['layout']['shapes'] as shown to the application.
            """)),
            html.Pre(id='output-two', style=styles['pre']),
        ], className='nine columns'),
    ])
])


@app.callback(
    Output('shapes-and-annotations', 'figure'),
    [Input('do-it', 'n_clicks')],
    [State('shapes-and-annotations', 'figure')])
def reduce_and_shift_down(clicks, figure):
    del figure['layout']['shapes'][-1:] # remove the last shape on the list
    for shape in figure['layout']['shapes']:    # move all the remaining shapes down one.
        shape['yanchor'] -= 1
    del figure['layout']['annotations'][-1:] # remove the last annotation on the list
    for annotation in figure['layout']['annotations']:    # move all the remaining annotations down one.
        annotation['y'] -= 1
        annotation['ay'] -= 1
    return figure



@app.callback(
    Output('output-one', 'children'),
    [Input('shapes-and-annotations', 'figure')])
def display_shapes_count(figure):
    return 'len(figure[\'layout\'][\'shapes\']) : ' + str(len(figure['layout']['shapes']))


@app.callback(
    Output('output-two', 'children'),
    [Input('shapes-and-annotations', 'figure')])
def display_shapes_content(figure):
    return json.dumps(figure['layout']['shapes'], indent=2)




if __name__ == '__main__':
    app.run_server(debug=True, port=8051)

It’s very important to my use case that I can keep animations working, so the active question still is:
Is there some other way to format the list so this behavior doesn’t happen?

Alright, with a little more poking, there’s a potential answer:

It turns out that smacking None into the list is totally valid, and effectively removes that element position from the graph. It does not however do what I’d hoped, which would be to wipe out every old element beyond the None position.

Substitute the following into the example above to see this property in action.

@app.callback(
    Output('shapes-and-annotations', 'figure'),
    [Input('do-it', 'n_clicks')],
    [State('shapes-and-annotations', 'figure')])
def reduce_and_shift_down(clicks, figure):
    del figure['layout']['shapes'][-3:] # remove the last 3 shapes on the list
    for shape in figure['layout']['shapes']:    # move all the remaining shapes down one.
        shape['yanchor'] -= 1
    figure['layout']['shapes'].append(None)
    del figure['layout']['annotations'][-3:] # remove the last 3 annotations on the list
    for annotation in figure['layout']['annotations']:    # move all the remaining annotations down one.
        annotation['y'] -= 1
        annotation['ay'] -= 1
    figure['layout']['annotations'].append(None)
    return figure

I’m guessing there’s some logic to this behavior that I don’t see, but given that shapes an annotations don’t seem to actually animate anyway, it does seem odd.

Accepting this behavior as is, the only major problem is that now in order to write a fresh, guaranteed set of shapes to the graph, you either have to pad the list with more None’s than your app may ever use, or have special knowledge of how many shapes you used last time so you can fill to that index. Manageable but not ideal. Also if this behavior is documented somewhere, I could not find it based on my problem description.

Thoughts?

And now for extra good measure, This part is definitely a bug:

If you compose the figure using simple dictionaries, as shown in the above code, everything is good.
If you use the graph object Layout builder like so,

import plotly.graph_objs as go
...
'layout': go.Layout(
    ....
    annotations = [],
    )

and then try to fig['layout']['shapes'].append(None), it throws an error on encountering None in the list, like so:

  File "C:\......\Python36-32\lib\site-packages\plotly\graph_objs\graph_objs.py", line 214, in _value_to_graph_object
    raise exceptions.PlotlyListEntryError(self, path)
plotly.exceptions.PlotlyListEntryError: Invalid entry found in 'annotations' at index, '12'

Path To Error: ['annotations'][12]

Valid items for 'annotation

Despite a None element being totally valid for the front end.

Demonstration code, to swap in to the full code from above:

@app.callback(
    Output('shapes-and-annotations', 'figure'),
    [Input('do-it', 'n_clicks')],
    [State('shapes-and-annotations', 'figure')])
def reduce_and_shift_down(clicks, figure):
    figure['layout'] = go.Layout(figure['layout'])  # this is totally unneccesary here, but easier to force it here than rebuild the example. Immagine you were building up a fresh layout.
    del figure['layout']['shapes'][-3:] # remove the last shape on the list
    for shape in figure['layout']['shapes']:    # move all the remaining shapes down one.
        shape['yanchor'] -= 1
    figure['layout']['shapes'].append(None)
    del figure['layout']['annotations'][-3:] # remove the last annotation on the list
    for annotation in figure['layout']['annotations']:    # move all the remaining annotations down one.
        annotation['y'] -= 1
        annotation['ay'] -= 1
    figure['layout']['annotations']+= [None for i in range(2)]
    return figure

The go.Layout(), go.Scatter() etc methods are in the documentation in several places, but do they actually offer any value over dict()?

Only validation. Otherwise, they get converted to the same underlying JSON which gets fed into plotly.js

Is there a fix for this yet? Padding with None isn’t really an option for me.