Bring Drag & Drop to Dash with Dashboard Engine. 💫 Learn how at our next webinar!

Duplicate Callback Outputs - Solution & API Discussion

:wave: Hello everyone –

In yesterday’s Dash 2.0 webinar, there were a surprising (to me!) number of questions asking if we “allow multiple callbacks to update the same output.”

I wanted to kick off a discussion of this topic.

How to do this today

Here is the canonical working example that allows you to update an output. This works! It allows you to target a single input from multiple inputs.

from dash import Dash, callback, Input, Output, State, callback_context, html, dcc
import plotly.express as px
import plotly.graph_objects as go

app = Dash(__name__)

app.layout = html.Div([
    html.Button('Draw Graph', id='draw'),
    html.Button('Reset Graph', id='reset'),
    dcc.Graph(id='graph')
])

@app.callback(
    Output('graph', 'figure'),
    Input('reset', 'n_clicks'),
    Input('draw', 'n_clicks'),
    prevent_initial_call=True
)
def update_graph(b1, b2):
    triggered_id = callback_context.triggered[0]['prop_id']
    if 'reset.n_clicks' == triggered_id:
         return reset_graph()
    elif 'draw.n_clicks' == triggered_id:
         return draw_graph()

def draw_graph():
    df = px.data.iris()
    return px.scatter(df, x=df.columns[0], y=df.columns[1])

def reset_graph():
    return go.Figure()

app.run_server(debug=True)

Multiple Callbacks Targeting Same Output - This Syntax Doesn’t Work

Here is what I believe users expect to be able to do:

from dash import Dash, callback, Input, Output, State, callback_context, html, dcc
import plotly.express as px
import plotly.graph_objects as go

app = Dash(__name__)

app.layout = html.Div([
    html.Button('Draw Graph', id='draw'),
    html.Button('Reset Graph', id='reset'),
    dcc.Graph(id='graph')
])


@app.callback(
    Output('graph', 'figure'),
    Input('draw', 'n_clicks'),
    prevent_initial_call=True
)
def draw_graph(_):
    df = px.data.iris()
    return px.scatter(df, x=df.columns[0], y=df.columns[1])

@app.callback(
    Output('graph', 'figure'),
    Input('reset', 'n_clicks'),
    prevent_initial_call=True
)
def reset_graph(_):
    return go.Figure()


app.run_server(debug=True)

If you run this example, you’ll receive this error message:

Duplicate callback outputs

In the callback for output(s):
  graph.figure
Output 0 (graph.figure) is already in use.
Any given output can only have one callback that sets it.
To resolve this situation, try combining these into
one callback function, distinguishing the trigger
by using `dash.callback_context` if necessary.

Trade-Offs

We could support the unsupported example as an official part of the Dash API. Community member’s @Emil 's excellent dash-extensions library has MultiplexerTransform which implements this with a dcc.Store technique. I suspect that we if we supported this in Dash we would try to wire in the triggered option above into the dash library so that no extra requests are made and no dcc.Store's are necessary.

That being said, there are some API challenges and trade-offs to supporting multiple callbacks targeting the same output:

  • Which callback to fire at start?

    Do you fire all of the callbacks? Or none of them? This behavior is ambiguous and In the face of ambiguity, [we should] refuse the temptation to guess.

    In the triggered example, you can have a final else statement that handles this case.
    In order for this to be unambiguous, we’d need the user to set prevent_initial_call for N-1 of their callbacks. At that point, writing Duplicate Callback Outputs would probably be the same lines of code as using triggered and multiple functions. If the user didn’t specify prevent_initial_call for N-1 callbacks then we’d raise an exception.

    And what if you had multiple outputs in one of the callbacks? prevent_initial_call would prevent the entire callback from being fired, so you would need to split up this callback. Unless we added prevent_initial_call to Output: Output('graph', 'figure', prevent_initial_call=True).

    Alternatively, we could by default call none of the outputs and then the user could add initial_call=True to just a single callback output.

  • It doesn’t add any new functionality.

    As far as I understand, every use case for Duplicate Callback Outputs can be solved using triggered as in the example above.

    In the spirit of the Zen of Python, “There should be one-- and preferably only one --obvious way to do it.”

    However you could make the argument that using triggered isn’t obvious. It certainly requires some mental gymnastics. A canonical documentation example could help here.

  • Clean code & ergonomics

    The triggered example is only 2 lines longer than the multiple callback example.

    In the first example, the callback is a multiplexer that calls different functions. This is pretty clean, unambiguous, and organized.

    However, if you put the contents of draw_graph and reset_graph into the if/elif/else statements then I can imagine that this function could become quite unwieldy.

    From a mental gynmastics perspective, we’ve certainly seen that users naturally write multiple callback Output, so there’s something to be said about supporting the “natural”/“self-discoverable” thing.

  • More broadly, Dash’s reactive architecture is based off of a Directed-Acyclic-Graph where a node only exists “once”. This is similar to e.g. Excel. You can see this graph in the Dash devtools. If we built this into Dash, we would either need to relax the DAG or wire this into the callback processing logic on the backend, in which case the DAG diagram in the devtools wouldn’t exactly match what the structure of the user’s Dash app code (the output node would only exist once, even though it would have multiple app.callback declarations).

  • triggered syntax - Another solution would be to provide a more ergonomic

    triggered_id = callback_context.triggered[0]['prop_id']
    

    For example, we could make a new triggered_dict that would have the same content but be a dictionary instead of a list. The logic would become:

    def update_graph(b1, b2):
        if 'reset.n_clicks' in dash.callback_context.triggered_map:
            return reset_graph()
        elif 'draw.n_clicks'  in dash.callback_context.triggered_map:
            return draw_graph()
    

    which would save another line of code.

Moving Forward

First off, we need to publish the canonical example above in our documentation to make it more discoverable. And we should link the error message to this documentation page. The current error message leaves a lot of the details out :slight_smile:

Beyond that, I’d love to hear the community’s thoughts on the tradeoffs above and if you have any other suggestions for this problem. Thanks for reading!

3 Likes

Re: DAG, it’s possible that certain “output” nodes are targeted by multiple “input” nodes. Here’s an example:

Theoretically, I think of components*props as nodes, callbacks as directed edges, and callback functions as colors of the edges. So if two callback functions target the same components, it’d be simply two differently colored edges targeting the same node. So the b->d arrow would have a different color from the c->d edge.

1 Like

There was lots of discussion about this in Github here: Duplicate Outputs[Feature Request] · Issue #850 · plotly/dash · GitHub

One particularly thorny variant that was discussed was to have two callbacks that are meant to target the same output but also have shared inputs.

Having introduced Dash to a few people, I can confirm that the duplicate callback outputs issue is typically their first complaint - so I am not surprised that this particular question was raised. They implemented more-or-less exactly the structure of your “what users expect to be able to do” example, and when I then presented (a form of) your “canonical example” they responded with something along the lines of “but why does it have to be that complicated?”. They don’t want to know about a (complicated) context object, nor pollute their code with loads of conditional statements. And to be honest, I kind of feel the same.

I understand the design choice from an implementation perspective, but the syntax is not intuitive, nor simple. It feels like the technical solution is driving the syntax, but I generally think that it should be the other way around. Since people kind-of-agree on how the syntax should be (your “what users expect to be able to do” example) I would vote for adopting this syntax (who doesn’t love intuitive frameworks?), and then just solve whatever technical challenges arise. This design philosophy was also the driver of the design of the MultiplexerTransform.

7 Likes

I tend to agree with @Emil on this. I understand the technical considerations, and the potential tradeoffs, and as someone reasonably well-versed in managing complexity in a Dash app, this is still the feature that I spend the most time thinking about.

For me, Dash is about making serious web app development approachable for someone in my position, who is comfortable in Python but has minimal web background. This is (for me) a more important feature than the component design ‘guide’ presented in Dash 2.0 (which I also consider a step in the right direction). If there is a way to make a more intuitive syntax available (while also making clear the tradeoffs to consider when electing to use it), I would be absolutely for it. Even a @dash.multiplexed_callback() or similar that offers a solution to composability that makes it clear that it’s (under the hood) different to a regular callback, is something I would be very happy with.

1 Like

In my scenario, I am saving state based on a different sections of my webpage. For example

As you can see I have a different Bootstrap card where there are buttons and many input fields like dropdown, input, DateTime picker, etc. Now sync all of them (ex. keeping a global state in the app) I want to modify the Store().

Here I can use the callback context approach but the function will become huge. Like I have to give Input() for all of them. Instead What I did is I use the button callback to modify the Store() as well as the other properties in the Card. And this resulted in the duplicate callback.
test.drawio (1)

I think there could be a better solution for this. Can anyone point me out?

Did you try the MultiplexerTransform?

1 Like

I can’t believe I missed this gem in @chriddyp original post, but I’d like to give it :+1: :+1: :+1: and a :star2:

I think making a new triggered dict would make the code so much more readable. To me, this is meaningless boilerplate:

triggered_id = callback_context.triggered[0]['prop_id']
4 Likes

I agree that a better way of interacting with the callback_context is desirable. Especially when working with pattern-matching callbacks. Doing things like this is not fun and it creates a huge space for hard-to-discover bugs.

1 Like

Hey @sislvacl - I remember that post and I agree. :+1:

There is a lot of exciting work being done right now to reduce boilerplate and make Dash easier to use. Dash V2.1 is going to be really cool. I just added this feature request on Github and included your comments. Hopefully this will make it into a future Dash release!

1 Like

My use case for multiple callbacks with the same output is the following:

In a multi-page app, I save the state of a “project” via a Store component. The callbacks that update that store:

  • Use inputs that are specific to each page and therefore cannot be all put in the same callback
  • Update other outputs that are specific to that page and putting everything in one callback would make things messy with a lot of no_update outputs.

I have been using an equivalent of the MultiplexerTransform with multiple stores that synchronise with the main store but even this has its limitations in my context.

1 Like

In this scenario, which output is the “same output”? Does each page have e.g. the same graph?

Also, do certain controls/inputs in one page update outputs in other pages (via store)?

It’s more to keep track of the changes made by the user during their session so that they can save them to the cloud (database, json file or other, in my case I’m using a nosql db) once they are happy with the work. It’s a bit of a staging area keeping track of everything that has changed.

For illustration:

  • On the overview page the users define some general project info
  • On the components page,users can add / remove components and define their specifications
  • Once the changes have been made some results can be computed and allow the users to access a variety of graphs and outputs
  • Users can save their progress to update the remote with current changes (eventually I’ll probably also add a save as option)

Another use case for multi-callback for an output in this app is URL redirection. Certain actions on certain pages need to redirect the app to a page (e.g. if not logged in redirect to the login page, after logging in the user is redirected to the home page, after creating a new project the user is redirected to the project page). To do that, I need to update my Location component but can’t do it from different pages due to the variety of Inputs. I am also doing that with the Store synchronisation at the moment.

1 Like

In the case of the master callack dispatching what to do based on callback context, I got the feeling that the changes induced by the callback are not able to trigger again the master callback (in case where some of the outputs are also inputs). Is that the case ?

Here is a schematic example:

layout
    import_button IB
    checklist CL
    Div1
    Div2
    Div3

callbacks
    @app.callback(
        outputs=[CL Div1, Div2, Div3],
        inputs=[IB, CL],
        states=[Div1, Div2, Div3]
    )
    def master_callback(IB, CL, Div1, Div2, Div3):
        if callback_trigger == CL:
            --> set Div1, Div2, Div3 visible or not following the value list of CL
        if callback_trigger == IB:
            --> set value of CL based on imported YAML file

At the end of the import procedure, it re-assigns a new value to the checklist CL… which should in turn trigger again the master callback to update the visibility of the 3 Divs based on the box that are checked. But it’s not calling the master callback again according to my observations…