How to edit / delete shapes created using a callback?

Hello. The jpeg below shows the circles that are added to an image when I click on a location in the image (say a blemish I want to remove):

Below is the code I used to do this. I need help knowing how to access these shapes so I can edit or delete them? I found a way of using plotly callbacks to create these shapes, but if you look at the image - all shapes have trace 0 and 0 as a trace_name and trace_index. The only thing that changes are the x and y cordinates.

I can select and edit any shape I want using the mouse, but I can’t delete the shape, nor can I access any information on these shapes.

I’m sure plotly must be storing this somewhere. Can someone please help me understand how to access these shapes, and shape properties so I can edit or delete them?

Thank you!


# LIbraries used to run this code:

# Package that will be used to make graphs and capture user interactions 
import plotly.graph_objects as go

# Package to create interactive controls
import ipywidgets as widgets

# Library used to handle array and array-like objects
import numpy as np

# OpenCV Library
import cv2


# Load an image using ipywidgets and opencv to create an image
# Object to store loaded files
uploader = widgets.FileUpload(accept = '',  # Accepted file extension e.g. '.txt', '.pdf', 'image/*', 'image/*,.pdf'
                              multiple = False  # True to accept multiple files upload else False
                             )

# Display the button to load an image
# This code is orginally run in a notebook, so there is a pause in the remaining code below
# This code will cause an error if attempting to "run-all" or outside a jupyter environment
display(uploader)


# Grab the file content and store it in a new object
uploaded_file = list(uploader.value.values())[0]

# Load image into OpenCV
image = cv2.imdecode(np.asarray(bytearray(uploaded_file['content'])), cv2.IMREAD_UNCHANGED)

# If image is not read properly, return error
if image is None:
    print('Failed to load image file: {}'.format(uploaded_file['metadata']['name']))
else:
    print('Succesfully loaded image file: {}'.format(uploaded_file['metadata']['name']))


# Convert from bgr to rgb
image = image[:, :, :: -1]


# Create a figurewidget object that will be used to 
# handle the image
fig = go.FigureWidget()


# Add the image to the figure object
fig.add_image(z = image);


# Update the layout 
fig.update_layout(height = 800,
                  width = 1200,
                  title = f"Image filename: {uploaded_file['metadata']['name']}",
                 );

# Image figure object
image_fig = fig.data[0]

# Create an output object
out = widgets.Output()

# Callback function
@out.capture(clear_output = True)
def get_mouse_clicks(trace, points, selector):
    
    size = 10
    
    ind = points.point_inds[0]
    
    x = ind[1]
    y = ind[0]
    
    trace_name = points.trace_name
    trace_index = points.trace_index
    
    message = f"The mouse points are:x = {x}, y = {y}.\nThe trace name is: {trace_name}. \nThe trace index is: {trace_index}."
    
    print(points)
    print()
    print(selector)
    print()
    print(message)
    
    # When the left button is clicked
    if selector.button == 0:
        # Draw a circle
        fig.add_shape(editable = True,
                      type = "circle",
                      x0 = x - size / 2, 
                      y0 = y - size / 2, 
                      x1 = x + size / 2, 
                      y1 = y + size / 2, 
                      xref = 'x', 
                      yref = 'y',
                      fillcolor = "grey",
                      line_color = "black",
                     )

# Register the callback
image_fig.on_click(get_mouse_clicks)

# Display output
widgets.VBox([fig, out])

I’m not interested in finding a solution using Dash. At this point I’m trying to put something conceptual together first, which is why I’m interested in using ipywidgets and plotly to do this.

Thank you!

–Igor

Hi @iacisme the shape has a name attribute which you can use to identify shapes in your image. It is actually not intended for this, see also here. I used it in the past for finding shapes in an image by iterating thru the list of annotations.

Here an example:

    base_shape = {
        'editable': True,
        'xref': 'x',
        'yref': 'y',
        'layer': 'above',
        'opacity': 1,
        'line':
            {
                'color': '#E2F714',  # default color
                'width': 2,
                'dash': 'solid'
            },
        'fillcolor': 'rgba(0, 0, 0, 0)',
        'fillrule': 'evenodd',
        'type': 'rect',
        'x0': 0,    # default values
        'y0': 0,
        'x1': 0,
        'y1': 0,
        'name': 'default_name'
    }

Deleting the shapes is as easy as deleting an item from a list after finding the correct index (searching for the name).

Hi @AIMPED thanks for your respones. If I understand what you’re saying correctly - when creating a shape, include a custom_name in the name attribute, that way I can create a list of names?

You can then delete a name from the list, and update the shapes based on the list (ie: using a for loop using name in list)?

Hopefully my understanding is correct. I believe I’m clear on what you’re saying, except the part of using the list to update the shapes. Could you please elaborate?

Here’s what I’ve learned so far, from the example you provided as well as this example here:

The following codes tells you how many shapes you have in your figure object:

# Find out how many shapes have been stored
len(fig.layout.shapes)

In my case it returned 5, the number of circles I have in the image.

I can get the properties of each shape as follows:

# Get the property of an added shape
# https://community.plotly.com/t/the-layout-shapes-property-does-not-update-after-drawing-shapes-with-the-mouse/61739
fig.layout.shapes[0]

OUTPUT:

layout.Shape({
‘editable’: True,
‘fillcolor’: ‘crimson’,
‘line’: {‘color’: ‘black’},
‘opacity’: 0.5,
‘type’: ‘circle’,
‘x0’: 296.0,
‘x1’: 306.0,
‘xref’: ‘x’,
‘y0’: 132.0,
‘y1’: 142.0,
‘yref’: ‘y’
})

# Properties of the last shape in the list
fig.layout.shapes[-1]

OUTPUT:

layout.Shape({
‘editable’: True,
‘fillcolor’: ‘crimson’,
‘line’: {‘color’: ‘black’},
‘opacity’: 0.5,
‘type’: ‘circle’,
‘x0’: 337.0,
‘x1’: 347.0,
‘xref’: ‘x’,
‘y0’: 199.0,
‘y1’: 209.0,
‘yref’: ‘y’
})

I can even do things like change the fill color of each circle using a for loop:

for index in range(len(fig.layout.shapes)):
    print(index)
    print(fig.layout.shapes[index])
    print()
    
    fig.layout.shapes[index].fillcolor = 'green'

However it’s still not clear to me how to delete a shape. I know I can make the list you mention, but I’m not sure how to implement it in updating the shapes. In the future the circle sizes are going to vary according to the size of the blemish, so I have to be careful when doing updates of properties to each shape.

I noticed there’s a pop method, but the doc string isn’t clear to me:

Signature: fig.layout.activeshape.pop(key, *args)
Docstring:
Remove the value associated with the specified key and return it

Parameters

key: str
Property name
dflt
The default value to return if key was not found in object

Returns

value
The removed value that was previously associated with key

Raises

KeyError
If key is not in object and no dflt argument specified
File: ~/.virtualenvs/computer_vision/lib/python3.10/site-packages/plotly/basedatatypes.py
Type: method

I’d really appreciate it if you could show me your method of deleting a shape, I think I got the rest covered now.

Cheers!

Hi @iacisme this is a quick & dirty example in dash. I know you do not want to use dash, I will try to answer your question by just using plotly, but I actually never used shapes in pure plotly.

Anyways, the section where I delete the shapes is in the callback. After the name has been introduced in the dcc.Input() and the delete button has been clicked.

from dash import Dash, dcc, html, Input, Output, State
import dash_bootstrap_components as dbc
from dash.exceptions import PreventUpdate
import numpy as np

import plotly.graph_objects as go

# prepare data
data = go.Scatter(
    x=[1, 10],
    y=[1, 10],
    mode='markers',
    marker={
        'size': 8,
        'symbol': 'circle-open',
    },
)

# create figure
fig = go.Figure(data=data)

# add some shapes
for i in range(1, 6):
    fig.add_shape(
        {
            'type': 'rect',
            'x0': np.random.randint(1, 5), 'x1': np.random.randint(6, 11),
            'y0': np.random.randint(1, 5), 'y1': np.random.randint(6, 11),
        },
        editable=True,
        name=f'shape_{i}',
        line={
            'color': ['red', 'yellow', 'blue', 'pink'][np.random.randint(0, 4)],
            'width': 2,
            'dash': 'solid'
        },
    )


# update layout
fig.update_layout(
    template='plotly_dark',
    plot_bgcolor='rgba(0, 0, 0, 0)',
    paper_bgcolor='rgba(0, 0, 0, 0)',
    width=700,
    height=500,
    margin={
        'l': 0,
        'r': 0,
        't': 20,
        'b': 0,
    }
)

# Build App
app = Dash(
    __name__,
    external_stylesheets=[dbc.themes.SLATE],
    meta_tags=[
        {
            'name': 'viewport',
            'content': 'width=device-width, initial-scale=1.0'
        }
    ]
)

# app layout
app.layout = dbc.Container(
    [
        dbc.Row(
            dbc.Col(
                dcc.Graph(
                    id='graph',
                    figure=fig,
                    config={
                        'scrollZoom': True,
                        'displayModeBar': False,
                    }
                ),
                width={'size': 5, 'offset': 0}
            ), justify='around'
        ),
        dbc.Row(
            [
                dbc.Col(
                    [
                        html.Button(
                            'Delete',
                            id='delete'
                        ),
                        dcc.Input(
                            id='box',
                            type='text',
                            value='shape_x',
                            className='input'
                        ),
                    ], width={'size': 5, 'offset': 0}
                ),
            ], justify='around'
        )
    ], fluid=True
)


@ app.callback(
    Output('graph', 'figure'),
    Input('delete', 'n_clicks'),
    State('graph', 'figure'),
    State('box', 'value'),
    prevent_initial_call=True
)
def get_click(click, current_figure, shape_to_delete):
    if not click:
        raise PreventUpdate
    else:
        # get existing shapes
        shapes = current_figure['layout'].get('shapes')
        
        # delete shape, aka keep only the shapes which are not to be deleted
        shapes[:] = [shape for shape in shapes if shape.get('name') != shape_to_delete]

        # update figure layout
        current_figure['layout'].update(shapes=shapes)
    return current_figure


if __name__ == '__main__':
    app.run_server(debug=True, port=8053)

Hi @AIMPED, thanks for this example!

I ran it using jupyter-dash. It makes sense why I need to add custom names to added shapes.

The gist also makes sense:

  1. shape_to_delete holds the name of the shape the user choses to delete (from shape_1 to shape_5)
  2. The callback triggers you to get all the shapes and store them in a list
  3. Update the shapes object based on the list of names found in shapes, as long as the name is != shape_to_delete

There’s has to be a way of doing the same thing through plotly.

You code has given me some ideas.

Thank you!

–Igor

Hi @iacisme , you are very welcome.

You are iterating thru the same list here, just that instead of the “name” you are interested in the “fillcolor”.

Hi @AIMPED, although I haven’t yet solved this issue, I have figured a few more things out, like how to automatically name each created shape. Here is the updated callback function:

@out.capture(clear_output = True)
def get_mouse_clicks(trace, points, selector):
    
    size = 10
    
    print(points.point_inds)
    
    ind = points.point_inds[0]
    
    x = ind[1]
    y = ind[0]
    
    trace_name = points.trace_name
    trace_index = points.trace_index
    
    # When the left button is clicked
    if selector.button == 0:    
            
        # Object to store the total number of shapes
        # added to the image
        total_shapes = len(fig.layout.shapes) + 1
        
        # Draw a circle
        fig.add_shape(editable = True,
                      type = "circle",
                      x0 = x - size / 2, 
                      y0 = y - size / 2, 
                      x1 = x + size / 2, 
                      y1 = y + size / 2, 
                      xref = 'x', 
                      yref = 'y',
                      line_color = "black",
                      fillcolor = "crimson",
                      opacity = 0.5,
                      name = f"shape_{len(fig.layout.shapes)}",
                     )
    
    message = f"The mouse points are: \n\nx = {x}, \ny = {y} \n\nTotal shapes on image = {total_shapes}"
    
    print(message)

I can even make a list of shapes:

shape_list = list(fig.layout.shapes)

Here is the output of doing a print(shape_list):

[layout.Shape({
     'editable': True,
     'fillcolor': 'crimson',
     'line': {'color': 'black'},
     'name': 'shape_0',
     'opacity': 0.5,
     'type': 'circle',
     'x0': 132.0,
     'x1': 142.0,
     'xref': 'x',
     'y0': 101.0,
     'y1': 111.0,
     'yref': 'y'
 }),
 layout.Shape({
     'editable': True,
     'fillcolor': 'crimson',
     'line': {'color': 'black'},
     'name': 'shape_1',
     'opacity': 0.5,
     'type': 'circle',
     'x0': 296.0,
     'x1': 306.0,
     'xref': 'x',
     'y0': 128.0,
     'y1': 138.0,
     'yref': 'y'
 }),
 layout.Shape({
     'editable': True,
     'fillcolor': 'crimson',
     'line': {'color': 'black'},
     'name': 'shape_2',
     'opacity': 0.5,
     'type': 'circle',
     'x0': 310.0,
     'x1': 320.0,
     'xref': 'x',
     'y0': 233.0,
     'y1': 243.0,
     'yref': 'y'
 }),
 layout.Shape({
     'editable': True,
     'fillcolor': 'crimson',
     'line': {'color': 'black'},
     'name': 'shape_3',
     'opacity': 0.5,
     'type': 'circle',
     'x0': 338.0,
     'x1': 348.0,
     'xref': 'x',
     'y0': 200.0,
     'y1': 210.0,
     'yref': 'y'
 })]

I figured it out by reading this post: 35794

What I need to now figure out to select a shape to delete it. I know that in Dash I could build another callback, but I want to figure this out in plotly first.

Any suggestions?

Cheers!

Hi @iacisme, as I said, I have never worked with shapes and plotly but I do like a little brain exercise.:joy:

think for me it’s important to understand, how you would like to delete the shape. Is there a button, is it done by code? Where do you get the information about which shape to delete?

I will have to create a plotly MRE because I do not really understand your setup with the widget stuff. :see_no_evil: