📣 Composite Components Draft Proposal - Pt 2

[chriddyp]

Hi folks –

Here is Pt 2 of the 📣 Modular & Reusable Python Components (With Attached Callbacks) discussion, aka “Composite Components”.

I’ve made some progress since the last discussion and written out two real-world examples:

1. TabbedGraphComposite - A tabbed graph + data component similar to Our World in Data’s UI: Which countries have mandatory childhood vaccination policies? - Our World in Data
2. DataTableComposite - A DataTable component for large Pandas dataframes that filters, sorts, and pages on the server in Python rather than on the client.

These examples are here: composite.py · GitHub

I came up with some conventions and abstractions along the way. Here is a draft of composite component documentation. Would love your feedback!

Note - This documentation uses two features that aren’t in Dash yet!
They are:

• dash.callback - A way to register callbacks with the dash module rather than the app object.
• dash.composite - Some helper functions for writing Composite Components

Composite Components (DRAFT DOCUMENTATION)

Composite Components combine multiple Dash components and callbacks into a single, modular interface that can be reused within your app or across projects.

Because Composite Components include their own, built-in callbacks, they are often used to encapsulate server-side, Python-based data processing (like filtering Pandas DataFrames) or casting Python data structures into the lower-level, JSON-serializable format that traditional Dash components accept.

The Dash component libraries that Plotly maintains contain several composite components that you can use without modification. You can also create your own Composite Components using the same patterns.

Composite Components are written entirely in Python, no React or JavaScript required.

This page explores Composite Components, outlines some examples available within Dash and shows how to create your own.

Demonstrating Composite Components with DataTableComposite & TabbedGraphComposite

Here we look at two composite components, DashTableComposite and TabbedGraphComposite, that are part of the Dash library, explore what they do and how they can be customized with keyword arguments and used with callbacks.

Overview

The DataTableComposite component encapsulates the Pandas-based filtering, sorting, paging, and data type logic described in the DataTable - Python Callbacks chapter.

app.layout = DataTableComposite(df)

Writing the same component and callbacks with the regular Dash DataTable requires around 150 lines! See the second to last example in DataTable - Python Callbacks chapter. DataTableComposite encapsulates this logic into the single line of code shown above.

The TabbedGraphComposite component displays a graph and a table in two separate dcc.Tabs.

df = px.data.iris()
fig = px.scatter(df, x=df.columns[0], y=df.columns[1])

app.layout = TabbedGraphComposite(figure=fig, df=df)

Keyword Arguments

All of the same DataTable keyword arguments are available in DataTableComposite. For example:

app.layout = DataTableComposite(
    df,
    editable=False
)

DataTableComposite also introduces two new keyword arguments:

• df - A Pandas dataframe. This is an alternative to passing in the data or columns argument.
• composite_id - See below.

TabbedGraphComposite is composed of three main components: dcc.Tabs, dcc.Graph, DataTable. The keyword arguments to customize these individual components are made via table_kwargs, tabs_kwargs, tab_kwargs, and graph_kwargs. These kwargs are passed directly into the underlying components.

For example, making the table of a TabbedGraphComposite component not editable, while applying height styling to the tab:

TabbedGraphComposite(
    figure=fig,
    df=df,
    table_kwargs=dict(
        editable=False
    ),
    tab_kwargs=dict(
        style=dict(height: 30)
    )
)

Attaching Callbacks

Composite components are implemented with Pattern-Matching Callbacks (see below for more detail) and frequently encapsulate multiple Dash components.

Because of this, the id= property cannot be set as the Composite component creates its own PMC, dictionary IDs. Instead of id=, use composite_id= and DataTableComposite.ids.datatable(composite_id).

The example below demonstrates using the input to the filter_query of the DataTableComposite component to update a html.Div that has the id ‘my-summary’.

Note: The DataTableComposite uses the composite_id to identify it and this is then referenced in the Input by DataTableComposite.ids.datatable('my-table')

app.layout = html.Div([
    DataTableComposite(
        df,
        composite_id='my-table'
    ),
    html.Div(id='my-summary')
])

@app.callback(
    Output('my-summary', 'children'),
    Input(DataTableComposite.ids.datatable('my-table'), 'filter_query')
)
def update_summary(filter_query):
    # ....

By convention, all Composite components have the .ids class. The properties of this class are functions that reference the IDs of the underlying components. In this case, the DataTableComposite.ids.datatable refers to the underling DataTable component. The name datatable is arbitrary and is decided by the author of the Composite component.

Unlike DataTableComposite, TabbedGraphComposite is composed of multiple components. Each of these components has its own ID: TabbedGraphComposite.ids.tabs, TabbedGraphComposite.ids.tab, TabbedGraphComposite.ids.datatable, TabbedGraphComposite.ids.graph.

Accessing the Underling Callback Functionality

Some Composite components will expose the Python function(s) that are used by the callback, allowing you to write your own callbacks that run similar functionality on different outputs.

For example, the DataTableComposite component filters, sorts, and pages a Pandas dataframe and updates the table with the result. These functions are provided in the public API via:

• DataTableComposite.filter_df(df, filter_query)
• DataTableComposite.sort_df(df, sort_by)
• DataTableComposite.page_df(df, page_current, page_size)

If you wanted to update another component with that filtered data, you could write your own callback that listens to the filter_query property as an input, filters the data using DataTableComposite.filter_df(df, filter_query), and then updates your output with that data:

@app.callback(
    Output('my-summary', 'children'),
    Input(DataTableComposite.ids.datatable('my-table'), 'filter_query')
)
def update_summary(filter_query):
    dff = DataTableComposite.filter_df(df, filter_query)
    return str(dff.describe())

Reading the Source Code and Forking Composite Components

The beauty of Composite Components is that they are written in Python using standard Dash components and callbacks. This means that the underlying source code should look familiar and approachable.

We encourage you to read the source code of the components if you are curious about which components make up a Composite Component, how the arguments are threaded down, what the underlying callback functions are actually doing.

If a Composite Component isn’t sufficiently flexible for you, then you can copy the source code into your own project and make your own modifications.

---

Writing Your Own Composite Components

Composite Components are built with standard Dash components, pattern-matching callbacks, and a few composite helpers.

Basic Example Without Callbacks

This is a simple example of a Composite Component composed of two tabs with a dcc.Graph in the first tab and a DataTable in the second tab.

The user can customize any of the underlying components with the table_kwargs, tabs_kwargs, tab_kwargs, and graph_kwargs arguments.

class TabbedGraphComposite(html.Div):
    def __init__(self, figure=None, df=None,
                 table_kwargs=None, tabs_kwargs=None, tab_kwargs=None,
                 graph_kwargs=None):
        super().__init__(   # < This is basically calling `html.Div`
            children=dcc.Tabs(
                children=[
                    dcc.Tab(
                        label='Graph',
                        children=dcc.Graph(
                            figure=figure,
                            **graph_kwargs
                        ),
                        **tab_kwargs
                    ),
                    dcc.Tab(
                        label='Data',
                        children=dash_table.DataTable(
                            data=df.to_dict('r'),
                            **table_kwargs
                        ),
                        **tab_kwargs
                    ),
                ],
                **tabs_kwargs
            )
        )

Basic Example With Callbacks

class TabbedGraphComposite(html.Div):
    _component_name = __qualname__
    ids = dash.composite.create_id_class(
        _component_name, ['table', 'tabs', 'graph', 'store', 'tabs_content'])

    def __init__(self,
                 figure=None,
                 df=None,
                 composite_id=None,
                 table_kwargs=None,
                 tabs_kwargs=None,
                 tab_kwargs=None,
                 graph_kwargs=None,
                 storage='client'):

        if composite_id is None:
            composite_id = str(uuid.uuid4())

        if ((tabs_kwargs and ('id' in tabs_kwargs)) or
                (graph_kwargs and ('id' in graph_kwargs)) or 
                (table_kwargs and ('id' in table_kwargs))):
            raise dash.composite.IDNotAllowedException(__qualname__, self.ids)

        # Store data in Redis so that it can be retrieved later on the fly 
        # when displaying the tab or downloading the data.
        # Use the data's hash as the unique key - 
        # This will prevent multiple processes from writing duplicate data.
        store_data = {}
        for (name, obj) in [
                ('figure', figure),
                ('df', df), ('graph_kwargs', graph_kwargs),
                ('table_kwargs', table_kwargs),
                ('composite_id', composite_id),
            ]:
            store_data[name] = dash.composite.composite_store.save(obj)

        super().__init__(
            children=[

                dcc.Store(
                    id=self.ids.store(composite_id),
                    data=store_data
                ),

                dcc.Tabs(id=self.ids.tabs(composite_id), children=[
                    dcc.Tab(
                        label='Graph', value='graph', **tab_kwargs
                    ),
                    dcc.Tab(label='Data', value='data', **tab_kwargs)
                ], **tabs_kwargs),
                
                html.Div(id=self.ids.tabs_content(composite_id))
            ],

        )

    @dash.callback(
        Output(ids.tabs_content(MATCH), 'children'),
        Input(ids.tabs(MATCH), 'value'),
        State(ids.store(MATCH), 'data'),
    )
    def display_tab(tab, store):
        composite_id = dash.composite.composite_store.load(store['composite_id'])
        if tab == 'graph':
            figure = dash.composite.composite_store.load(store['figure'])
            graph_kwargs = dash.composite.composite_store.load(store['graph_kwargs'])

            return html.Div([
                'Graph',
                dcc.Graph(
                    figure=figure,
                    **dash.composite.omit(['figure', 'id'], graph_kwargs, 'graph_kwargs', TabbedGraphComposite._component_name, TabbedGraphComposite.ids)
                )
            ])

        else:
            df = dash.composite.composite_store.load(store['df'])
            table_kwargs = dash.composite.composite_store.load(store['table_kwargs'])

            return html.Div([
                'DataTable',
                dash_table.DataTable(
                    columns=[{'name': i, 'id': i} for i in df.columns],
                    data=df.to_dict('r'),
                    **dash.composite.omit(['id'], table_kwargs, 'table_kwargs', TabbedGraphComposite._component_name, TabbedGraphComposite.ids)
                )
            ])

Passing Data from Constructor to Callback

Composite Components should be able to be returned within a user’s callbacks.

Dash callbacks need to be defined upfront when the application starts.

A Composite Component’s callbacks will often need to access the data passed into the constructor. Dash is stateless, which means that the process that runs the callback may be a different process than the process that initialized the component. These processes could even be on separate servers in a horizontally scaled deployment.

So, for the callback to access the data passed into the constructor, it needs to either be:

• Stored in the client in a dcc.Store, and passed into the callback via State
• Stored on the backend in a shared memory database like Redis, and loaded within the callback.

Storing data in dcc.Store will pass the data over the network from the server to the browser and hold the data in memory in the browser. This is OK if the data is relatively small: <5MB.

If the data is larger than 5MB, then it’s often more performant to store it in Redis.

The composite_store.save and composite_store.load provide a higher level API for storing data in Redis.

The example above demonstrates how this is done. It involves:

• Saving the data in Redis via composite_store.save
• Storing the keys to this data in Redis in a dcc.Store so that they are associated with the session
• Passing these keys to the PMC callback
• Loading the data from Redis using the keys in the PMC callback with composite_store.load

The keys are autogenerated from a hash of the underlying data. This means that if the same dataset is used in multiple browser sessions, it will only be saved once.

IDs

Pattern matching callbacks use dictionary IDs.

dash.composite.create_id_class is a helper function that creates these dictionary IDs as a class:

class TabbedGraphComposite(html.Div):
    _component_name = __qualname__
    ids = dash.composite.create_id_class(_component_name, ['table', 'tabs'])

This code is equivalent to:

class TabbedGraphComposite(html.Div):
    class ids:
        table = lambda instance: ({
            'component': 'TabbedGraphComposite',
            'subcomponent': 'table',
            'instance': instance
        })
        tabs =  lambda instance: ({
            'component': 'TabbedGraphComposite',
            'subcomponent': 'tabs',
            'instance': instance
        })

The PMC MATCH class is used to define the generic callbacks within the Composite Component. TabbedGraphComposite.ids.table(MATCH) is simply shorthand that generates a PMC-compatible dictionary:

>>> TabbedGraphComposite.ids.table(MATCH)
{
    'component': 'TabbedGraphComposite',
    'subcomponent': 'table',
    'instance': MATCH
}

This allows you to write your generic Composite Component callback like:

@dash.callback(
    Output(ids.tabs_content(MATCH), 'children'),
    Input(ids.tabs(MATCH), 'value'),
    State(ids.store(MATCH), 'data'),
)
def display_tab(tab, store):

Binding a Callback to a Particular Composite ID

The composite_id paramater allows the Dash developer to create their own callbacks with inputs or outputs of the underlying components. See the Attaching Callbacks example above for details.

>>> TabbedGraphComposite.ids.table('my-graph')
{
    'component': 'TabbedGraphComposite',
    'subcomponent': 'table',
    'instance': 'my-graph'
}

As a Composite Component author, allow composite_id as a parameter of your component.

If it is None, then use a uuid.uuid4() as the component’s composite_id. uuid.uuid4() is a random ID with a virtually impossible chance of ID collision.

Defining the Composite Callback

Within the Class

Define the callback within the Composite Component class but outside of any of the class’s functions or __init__ (see the example above).

Dash callbacks must be defined during initialization. They cannot be defined during runtime.

By defining the callback within the class, it will get registered when the Composite Component’s class is imported which usually happens during app initialization.

Use PMCs

By using PMCs with MATCH, you can allow multiple Composite Components to be defined within the same layout while all updating via the same PMC callback.

dash.callback

Use dash.callback instead of app.callback. This allows your composite callbacks to be imported before the app is defined and prevents the user from needing to pass an app object into the Composite Component constructor.

Conventions

There are some patterns and conventions in writing Composite Components to make them customizable and to make their API predictable and familiar.

Naming

By convention, Composite Components have the suffix "Composite". For example, DataTableComposite, TabbedGraphComposite.

Exposing Underlying Component’s Properties

If a Composite Component is composed of several components, provide <sub_component>_kwargs= arguments and pass these kwargs down to the underlying component.

For example, TabbedGraphComposite is composed of dcc.Tabs, dcc.Tab, dcc.Graph, and DataTable. Its constructor has:

• table_kwargs - Passed down to the DataTable
• tabs_kwargs - Passed down to the dcc.Tabs
• tab_kwargs - Passed down to the dcc.Tab
• graph_kwargs - Passed down to the dcc.Graph

Exposing the Underlying Callback Functions

Some Composite Component’s callback’s computations may be useful for other components written by the developer. If so, then consider making the underlying functions part of the Composite Component’s API. This will enable Dash developers to write their own callbacks that perform the same computations but update different components.

There is no naming convention or namespace recommendations for these functions - just name them something meaningful!

As an example, see the DataTableComposite.filter_df example above.

[chriddyp]

Some discussion topics:

• Composite Suffix Naming Convention - I like having Composite in the name since there are some important nuances (like id= vs composite_id=) that make them different from regular components. Do we like the Composite suffix? It doesn’t really roll of the tongue but it is explicit. We could also do a .composite. namespace within components packages like data_table.composite.DataTable vs dash_table.DataTable but that feels like it overloads DataTable too much.
• dash.composite namespace. I’m not set on the dash.composite namespace yet, nor the method names within that namespace. This will likely change, I’m open to suggestions.
• Appending or Prepending Components? The *_kwargs properties provide lots of flexibility for the underlying components, but one thing that’s not customizable is the ability to prepend or append other items to children. For example, in the TabbedGraphComposite, what if you wanted to add a custom header to the Graph Tab? What’s should be the API for doing that?
• Doc Strings - How can we make it easy for composite component authors to write docstrings? Especially for things like tabs_kwargs. Can we import those docstrings from the underlying components? Or just refer people to the docs for the underlying components?

[Emil]

Maybe a minor point, but I believe that your usage of random IDs,

composite_id = str(uuid.uuid4())

will break apps running across multiple workers as you are essentially making the server stateful.

[RenaudLN]

Yes this has bitten be in the past, my solution was to start a module-level counter (could be a module-level random seed) to get a new id in a repeatable manner:

counter = itertools.count()

...
composite_id = str(next(counter))

And I would love to see this come to Dash!

[chriddyp]

@Emil

Maybe a minor point, but I believe that your usage of random IDs,

composite_id = str(uuid.uuid4())

will break apps running across multiple workers as you are essentially making the server stateful.

This is actually OK in this scenario because we aren’t using this particular ID anywhere - The callbacks that are registered by the Composite Component bind to any component with the ID via MATCH. The ID just needs to be unique to prevent collisions if multiple components are embedded in the same layout.

The callback itself is stateless and doesn’t depend on that composite_id. The key of the data that the callback loads is from dcc.Store and associated with the session.

[chriddyp]

Another naming proposal from the wild: “Compound Components” (DataTableCompound) rather than “Composite Components” (DataTableComposite). Two syllables, nice alliteration, and unambiguous pronunciation.

[AnnMarieW]

Here’s another naming proposal: “Component Pack” (DataTablePack). Short, catchy, descriptive.

[RenaudLN]

I gave a good go at the composite components and I like it

Two suggestions coming from my messing around with it:

• I feel like in some cases we will need more levels of nesting in the creation of the component. E.g. in my case I wanted to create a pagination component which has a number of page buttons that will change the page (see below) and the only way I got this to work was to be able to add more keys to the id dictionary
• It would be great to have something that can be picked up by Intellisense for the ids dictionary, it would pick up typos and allow autocomplete which would greatly simplify dev work.

Here’s the stuff I did with composite component which includes

• Card component
• Edit component
• Delete component
• Pagination component

And here’s the tweak I made to create_id_class (basically you can pass a tuple for the instance):

def create_id_class(composite_component_name, subcomponent_names):
    class ids:
        pass

    def create_id_function(name):
        def create_id(instance):
            if isinstance(instance, tuple):
                return {
                    "instance": instance[0],
                    **{f"instance{i + 1}": val for i, val in enumerate(instance[1:])},
                    "component": composite_component_name,
                    "subcomponent": name
                }
            return {
                "instance": instance,
                "component": composite_component_name,
                "subcomponent": name
            }

        return create_id

    for name in subcomponent_names:
        setattr(ids, name, create_id_function(name))

    return ids

Which allowed me to do a callback like this:

    @app.callback(
        [Output(ids.page_store(MATCH), "data"), Output(ids.wrapper(MATCH), "children")],
        [Input(ids.page((MATCH, ALL)), "n_clicks")],
        [State(ids.wrapper(MATCH), "data-n_items")],
    )
    def update_page(...):

[chriddyp]

Nice!! Thanks for giving this a go. And AWESOME component!

@RenaudLN

• It would be great to have something that can be picked up by Intellisense for the ids dictionary, it would pick up typos and allow autocomplete which would greatly simplify dev work.

Agreed!

I was hoping create_id_class would do this, but I guess that’s too dynamic. We might have to declare the class members statically.

So I guess this is the simplest it could be?

class MyComponent(html.Div):
    class ids:
        page_store = dash.pmc_id('page_store', 'MyComponent')
        page = dash.pmc_id('page', 'MyComponent')
        wrapper = dash.pmc_id('wrapper', 'MyComponent')

Where pmc_id is:

def pmc_id(component, subcomponent):
    return lambda instance: {'instance': instance, 'component': component, 'subcomponent'}

Open to suggestions here!

[brianw]

Bump. Did these ideas for a composite component make it into dash or dash extensions?

[AnnMarieW]

Hi @brianw and welcome to the Dash community

Check out:

All-in-One Components | Dash for Python Documentation | Plotly

[brianw]

Thanks very much!

[someaveragepunter]

Hello,
Apologies I’m late to the game and have been building my own version of AIOs oblivious to the conventions above, but would now like to conform to the community standard.
Questions:
A) is the convention to only subclass html.Div or are there valid cases for subclassing other components as well?
B) Is there a reason classmethods preferred over instance methods for callbacks?

@property
def callback_myfunc(self):
    return [Output(self.ids.COMPONENT1, 'value'), 
            Input(self.ids.COMPONENT2, 'value'), State(self.ids.COMPONENT3, 'data')
    ], dict(
        prevent_initial_output=True
    )
def myfunc(self, component2, component3):
    return component3[component2]    

I favour instance methods for tweaking bespoke functionality through subclassing

1. allows me to override either the callback signature and actual function independently
2. allows me to reference instance properties in “self” so I can configure the “flavour” of my AIO through __init__or instance properties rather than having to subclass
3. Doesn’t require MATCH or ALL (potential performance gain)

(Note: I’ve not included the code that registers the callbacks onto the app upon __init__but that’s a simple implementation detail that can be tucked away in a common superclass)

[alaimo]

Hello,
I’m just getting started with dash so please forgive me if I am asking something that should be fairly obvious.

I was following the aio components convention to build some composite components that I plan to use frequently.
Inside of these components I wanted to make use the holoviews components and architecture but going through the documentation of dash holoviews there seems to be the need to call

components = to_dash(
app, [layout], reset_button=True, use_ranges=False
)

to finally be able to use it. Now I’m unsure on how to incorporate this inside my encapsulated component that does not know of “app”.
Could someone please push me in the right direction to resolve this?

thanks in advance!

[Reveur]

Any way using @dataclass or Pydatinc BaseModel instead init?