Could Output() be moved inside the components to improve readability?

Dash Python
1

An idea for an API change that I’d be interested in your opinions on. I am assuming it is technically possible, but correct me if I’m wrong.

Rather than requiring Output() to be specified against a callback, could it be specified at the property that it is actually changing?

Here’s what it might look like to replace the version in the user guide

app.layout = html.Div([
    dcc.Input(id='my-id', value='initial value', type='text'),
    html.Div(id='my-div', children=dash.compute(update_output_div))
])


@app.callback(
    [Input(component_id='my-id', component_property='value')]
)
def update_output_div(input_value):
    return 'You\'ve entered "{}"'.format(input_value)

The benefits I see are:

  1. With any reasonably large Dash app, it would be clear to see which components had dynamic properties and which were static. You can also use IDE features to get to the callbacks instantly, or to navigate to all uses of a callback.
  2. There might be a clear way to specify the default value (e.g. children=dash.Compute(theCallback, default="Loading..."), whereas at present it’s unclear if we specify a value for a component whether that gets overridden immediately by the callback.
  3. An obvious way to add multiple outputs from a callback without too much noise above the callback.
  4. More consistency with React, which is helpful for those familiar with React or who want to try making new Dash-React components.

I guess a similar idea would be to be able to specify Input components by their python reference, rather than relying on their Id. That would also improve IDE support, but I don’t think would be as popular as the Output idea, because you would have to declare and then import variables to take advantage of it.

1 Like
2

Yeah, this is pretty cool. It certainly makes the code easier to read in this example.

I’m generally pretty hesitant to make changes to the user-facing API like this and in general I’d like there to be 1 recommended way to do things. However, you bring up some good points and so I think these proposals are really good thought experiences and maybe we’ll end up incorporating some of these features in future versions of Dash.

In any case, here are some architectural notes and some other thoughts:

Architecturally, declaring the Output / callback signatures up-front made things a lot easier. On app-load, Dash’s frontend (dash-renderer) gets a list of all of the input/output relationships in the app (the /_dash-dependencies API call) and creates a directed-acyclic-graph (DAG) out of them. During the app’s lifecycle, Dash’s front-end logic is constantly checking this DAG to see if it needs to fire callbacks or not, both when properties change (typing into an input box) and when components are rendered (navigating into a new tab where children gets updated with a whole new tree of components).

In this scenario, the DAG would be dynamic: when new component trees get updated (i.e. from a callback updating children), dash-renderer would need to crawl that tree, check to see if any of the component properties will be updated via a callback, and then update the DAG. It seems like the backend would still be stateless, and so I don’t think that this is out of the realm of possibility!

Now, there are a few other features in the pipe that we would need to consider.

  1. We’re adding a DAG tree to Dash (https://github.com/plotly/dash/issues/423). If these relationships were dynamic, then we wouldn’t be able to display this entire tree on page load.
  2. We’re also working on wildcard / dynamic property updates: https://github.com/plotly/dash/issues/475. Not sure how this would fit in or not.
  3. It’s actually not clear to me how this fits in to multiple outputs in the same callback (https://github.com/plotly/dash/pull/436). What would the function return?
  4. While the code is easy to read with a self-contained snippet like this, it might actually be harder to read or maintain in larger projects when the callback functions are in separate files from the markup. I’ve found it helpful to know which property/component is getting updated with the decorator.
  5. Your point about default properties is a good one. I wonder if there are other ways that we can make this more clear.
  6. Your point about IDEs is a good one too. I wonder if there are other ways we can improve this as well (like using constants that are shared across layout and callbacks).

For better IDE support, we need to somehow link the function name to the markup. Maybe we can do something with IDs here?

app.layout = html.Div([
    dcc.Input(id=update_output.ids['inputs'][0]),
    html.Div(id=update_output.ids['output'])
])

@app.callback(Output('output', 'children'), [Input('input', 'value')])
function update_children(value):
    pass

That’s a little odd, because your “creating” the IDs in the callback functions rather than in the markup, but it would be backwards compatible and users wouldn’t have to use it, it’d just be a code-quality/IDE support trick.

1 Like
3

Thanks for your thorough response and great to understand a bit more of the mechanics. Totally appreciate this would be a big change.

Presumably, this could be dynamically generated instead, since you would need to generate the dynamic DAG anyway

I hadn’t come across that yet, but it looks to me that the need would be better served by employing my idea and pointing the set of components at the callback in a loop. Perhaps it would need to be able to dynamically generate callbacks too, but that could presumably fall out of the dynamic DAG. I think the resultant code would be much more easier to reason about and debug with no new DSL to learn:

[Input('store', 'theme')]
def warningColor(theme):
  return theme.warningColor

[Input('store', 'theme')]
def infoColor(theme):
  return theme.infoColor

def entry_color(entryType):
  if(entrytype == "warning"):
    return dash.Compute(infoColor)
  else:
    return dash.Compute(warningColor)


[html.Div(color=entryColor(entry.type) for entry in entries]

It would also have better IDE support and be less at risk of id clashes, a big risk as a dashboard gets bigger and gets maintained by multiple people.

I hadn’t twigged that this was going to target tupled outputs. What if you want to use the first entry for multiple outputs, which I thought was more common? I guess it would just need to be repeated in the current proposal. Personally that would be a more common case that we hit for something like the theme colour above. I guess in my scenario, it would need to be targeted by index:

dash.compute(multiOutput, 1)

This is where it’s very natural for me using the IDE to navigate to the usages of a function, which would take me to the target component with a simple keyboard shortcut. I find that easier than having to do a ctrl+f to find the specific ID which might not work at all if wildcards are introduced.

I may be more IDE dependent than most pythonistas as I come from C# with tools like Resharper. For me it is also very natural to look at the layout first and if I am interested in how a particular dynamic output is generated navigating to it using the IDE tools. That is what I’d do in a regular function that called off to some other function for some of it’s inputs.

Thanks again and hope my thoughts are useful :slight_smile:

4

For improving IDE support, I may have misunderstood your idea:

As this would be string based, I’m not sure it would light up the IDE helpers. However, if we put the IDs all in a separate python module then that would work I think.

#ids.py
someDiv = "someDiv"
someOtherDiv = "someOtherDiv"

import ids

html.Div(id=ids.someOtherDiv)
5

If you have the time, would you mind commenting in that issue? In particular, it’d be helpful to see a few full sample apps written out, like how I did with the TODO app: Dynamic Layouts (Wildcard Props) · Issue #475 · plotly/dash · GitHub

1 Like
6

In this case, with update_output.ids['output'], update_output would be the name of the function, so you could click on it to get routed to the function. ids would be a new property on that function that the decorator would add automatically. Er, that’s the idea at least :man_shrugging:

7

Very interesting @mungojam - I certainly sympathize with the benefits you’ve laid out from linking the callback directly into the layout. We could even imagine (again, pending making the dep graph itself dynamic) including the whole callback linkage in the layout and making the callback itself a bare function:

app.layout = html.Div([
    dcc.Input(id='my-id', value='initial value', type='text'),
    html.Div(id='my-div', children=dash.callback(update_output_div, Input('my-id', 'value'),
                                                 default='Loading...'))
])

def update_output_div(input_value):
    return 'You\'ve entered "{}"'.format(input_value)

That still wouldn’t prevent us from needing to refer to components by ID, but it would at least keep that reference nearby - note that you only created the name 'my-id' one line above where you used it, which I think is fairly common though by no means universal.

That said, it seems to me that while this helps identify dynamic props and link them to their callbacks in a way IDEs can follow, it makes the callback itself harder to write, because it no longer has all the info about its output (or inputs, in my extrapolation) right there in its decorator. In the case of multiple outputs this problem gets really hairy (using some syntax like dash.compute(multiOutput, 1) as you mention), as you’d need to find all users of the callback, combine them into a list in your mind (or in a code comment which you hope does not become obsolete), and use that to construct the return value.

Re @chriddyp’s idea about helping IDEs by pulling the id from the callback def - note that he had a typo writing update_output in the layout but update_children as the function name - they were intended to be the same.

Re dynamic/wildcard callbacks (which may also help with the multiple-consumers-of-one-output use case) I’ve been mulling the comments over there from the last few days to update the proposal - and this thread has given me a few more ideas to improve it - very much appreciated, and stay tuned to https://github.com/plotly/dash/issues/475!

8

I normally end up organizing large apps like this

# components.py
InputComponent = dcc.Input(id='my-id', value='initial value', type='text')
OutputComponent = html.Div(id='my-div', children=None)

# callbacks.py
import app from myApp
import InputComponent, OutputComponent from components

def update_output_div(input_value):
    return 'You\'ve entered "{}"'.format(input_value)

app.callback(
    Output(OutputComponent.id, 'children'),
    [Input(InputComponent.id, 'value'])(update_output_div)

This can sort of get your last point without any changes to the API (and makes it easier to unit test callbacks).

1 Like
9

@rnarren1 Thanks, that’s a neat idea yeah and I may well use that. That does solve the problem of using strings and would allow for IDE navigating in both directions. It has a slight downside that every component that needs lighting up with a callback has to be declared and importable at the top level. It would be a little frustrating if you had a layout with a nice amount of nesting (I often have 2 layers) and then you later wanted to add a callback to the inner layer as you’d then have to sacrifice a natural layout structure.

10

@chriddyp

gotcha, clever

I think it’s a little too odd for me especially when you have multiple callbacks targeting different properties of the Output element. Which would would you point to in the component id? I think there still wouldn’t be an easy way to find all callbacks that linked to a particular output component but at least it would highlight components that had at least one callback.

Another backwards compatible idea could be an IDE extension that knew about Dash and could highlight elements that have callbacks associated with them and helped you navigate through to them.

11

@chriddyp Thanks for the encouragement, see my attempt at TODO app

@alexcjohnson personally I think the trade off would be worth it, and it is something we accept any time that we write functions in other areas. Sometimes things that are easier to write are much harder to read and I would tend to prioritise ease of reading.

If multiple outputs used a named tuple or a dictionary for its return value, then it could be made into something nicer: style=dash.compute(multiOutput, 'style')

12

Nice to see my compromise idea going live in latest version :slightly_smiling_face:

13

Could you add a link?

14

I’m thinking this feature in 2.1:

3 Likes
15

Ah, yes. That looks nice :+1:

2 Likes