Feature Request: HTML Embedded layout and dependency lookup for standalone HTML Dash Reports

Dash Python
1

I’m not exactly sure how to best start this so I’ll just dive right in and hope for the best. This is a show-and-tell since I have working examples but more importantly, I’d like this to be the start of a feature request discussion for reading optionally embedded declarative application content instead of waiting for the subsequent client requests.

Save As…
Through its offline module and the ‘include_plotlyjs’ functionality, Plotly.py is capable of creating standalone plotlyjs html reports. This feature request and associated code changes enable offline standalone html Dash reports by looking for dash’s declarative application information as named script elements in the originally served, savable, html page, thereby enabling standalone dash clientside application start.

(Standalone html dash reports also face the issue of including enough of the required javascript libraries to function correctly, but as a developer I can track and work around code dependencies so library issues are mostly ignored in this.)

Proposed Changes
The proposal is to make dash-renderer look locally first for named script elements to use for initial page layout and dependency declarations. If found, those declarations are used instead of fetching the data from the server. If not found, everything proceeds as before. Once processed, everything also proceeds as before. This is only about _dash-layout and _dash-depdendencies being processed clientside from embedded content if present.

The changes are straightforward and somewhat follow the lead set by how dash-config is embedded and consumed. In APIController.react.js, just before each data fetch is attempted, look locally first for named elements and use that content if it exists.

This is the layout’s check local first proposed code:

    if (isEmpty(layoutRequest) && isEmpty(layout)) {
        const layout_element = document.getElementById('_dash-layout')
        if ( layout_element ) {
            layoutRequest.content = JSON.parse(layout_element.textContent)
            layoutRequest.status = STATUS.OK
        }
    }

The dependencies block is similar:

	    if (isEmpty(dependenciesRequest)) {
	        const layout_element = document.getElementById('_dash-dependencies')
	        if ( layout_element ) {
	            dependenciesRequest.content = JSON.parse(layout_element.textContent)
	            dependenciesRequest.status = STATUS.OK
	        }
	    }

This has the proposed changes to APIController.react.js:

To leverage these changes ‘serve_layout()’ and ‘dependencies()’ content needs to be proactively put into the original generated html page rather than waiting for dash-renderer to request it. From the dash server perspective, it is easy to create a custom HTML page that includes the new additional script elements somewhere in its definition:

	'''<script id="_dash-layout">{auto_layout}</script>
	<script id="_dash-dependencies">{auto_deps}</script>'''.format(
	   auto_layout=self.serve_layout().response[0].decode("utf-8"),
	   auto_deps=self.dependencies().response[0].decode("utf-8"))

Finally, for required supporting scripts, the problem is transitive closure over the set of scripts required to support standalone dash application execution. I’m sure there are multiple, better ways to go about this, but what I did was to watch the client/server interactions with dev tools to see the list of requested support .js files being served and make local copies. For my dash usage, this was limited to the same 10 files or so that once I copied I didn’t have to touch again. For the time being, since all I want is self contained behavior, I can make do with copying the required support files locally relative to where I save the html page. When I save the html page, I use curl and sed to do some simple post-processing of the html script elements forcing them to reference the local libraries instead of the dash server versions. Ultimately a single embedded inline solution comparable to what ‘include_plotlyjs=True’ does would be the most convenient but isn’t required for clientside operation.

That’s it. Once started, clientside operation is still a normal running dash app, with nothing else changed. The task as a dash developer is to push the right page behaviors into clientside callbacks for the appropriate offline, dash standalone report experience. In practice, the approach I found most productive was to bulk gather data serverside and put it into into dcc.Store declarations that are included in the initial page layout and can be included in dash Input/Output management.

Examples to follow.

2

Standalone HTML Report Examples

Here’s a zip with the examples I’ll talk about plus all related support files:
staging/stock_demo.zip at main · jwminton/staging · GitHub

I’ll try to highlight the important differences going from a simple save to the proposed dash standalone report. The examples use and cache public stock data. Its a little over the top but it stays the same across all examples and gives a nice rich sample to work from. I’ll go over the important changes from one example to the next ending with the standalone dash report. The main point of the zip are the 4 python file graph saving examples:
pure_plotly_save.py
simple_dash_save.py
interactive_clientside.py
interactive_dash_save.py

Plotly Support

Plotly.py supports the creation of standalone html reports by providing:
Library Access - Support library access in the form of minified plotly.js library that can be embedded into the html file.
HTML Expression - Offline plotly.py libraries for writing plotly figures to html

The plotly approach is to provide an api call that can be used to programmatically produce the standalone html plotly graph report:

fig = go.Figure(base_figure())
fig.write_html('plotly_save.html', include_plotlyjs=True)

The result is a single ~3Mb html file with script elements that embed the plotly library and that have the JSON serliazed plotly plot declarations included.

Dash Support

A Standalone HTML Dash report faces the same challenges.

Library Access - For Library Access, the problem is transitive closure over the set of scripts required to support the standalone dash application execution. Since we have full control over the plotly.py server, watching client/server interactions with dev tools will give the list of required support files. For me, in non-debug mode, this was my list (plus ramda):

async-dropdown.js
async-plotlyjs.js
dash_core_components-shared.js
dash_renderer.min.js
prop-types@15.7.2.min.js
react@16.13.0.min.js
async-graph.js
dash_core_components.min.js
dash_html_components.min.js
polyfill@7.8.7.min.js
react-dom@16.13.0.min.js
ramda.min.js

For the time being, since all I want is self contained behavior, I can make due with local relative file references. (See my post-processing/sed comments below for details.)

HTML Expression - Dash already knows how to write itself to JSON and HTML, in particular it knows how to serialize JSON expressions of clientside Dash application components and how their wired together relationships gets marshalled and unmarshalled. The problem here is the missing feature of being able to process embedded application information from the original page so that subsquent server calls are not required for clientside startup. The proposed changes allow this information to be embedded into the original html page.

Standalone HTML Report Implementation

From plotly to dash Save As…

This follows plotly’s suggested html save advice for a standard plotly figure:

\# Simple Pure Plotly graph save
fig = go.Figure(base_figure())
fig.write_html('pure_plotly_save.html', auto_open=False, include_plotlyjs=True)

Dash managed Save As…

With Dash, its possible to take the standard plotly ‘Save As…’ advice and make something somewhat more managed. Instead of a hardwired save based on server data, with dash you can send the rendered figure back to the plotly.py server and have it do the graph to html writing for you. This creates a server callback for a button label just to have a neutral dash target and sends the graph’s figure data as State to be saved:

app = Dash("Simple Dash Save")
app.layout = Div([
    Graph(id='graph', figure=base_figure()),
    Button(id='save', children='Save'),
])

@app.callback(Output('save', 'children'),
	      Input('save', 'n_clicks'),
	      State('graph', 'figure'),
	      prevent_initial_call=True)
def save_graph(n_clicks, graph):
    html = '''
    <html>
	<head>
	    <script type="text/javascript">{plotlyjs}</script>
	</head>
	<body>
	    <h1>Dash Example Save</h1>
	   {div}
	</body>
    </html>
    '''.format(plotlyjs=get_plotlyjs(),
	       div=plot(graph, output_type='div', include_plotlyjs=False)
	       )
    with open('simple_dash_save.html', 'w') as f:
	f.write(html)
    return 'Save'

Computed Trace

In the next example there is a dropdown that allows for the selection of a computed trace to be added to the current figure. The new trace computation is done with a serverside callback:

@app.callback(Output('graph', 'figure'),
              [Input('dropdown', 'value'),
               Input('graph', 'restyleData')],
              State('graph', 'figure'),
              prevent_initial_call=True)
def computed_trace(selected_operation, rData, figure):
    trace_array = []
    for entry in figure['data']:
        if 'meta' not in entry:
            trace_array.append(entry)

    data_array = []
    for entry in trace_array:
        if 'visible' not in entry or entry['visible'] != 'legendonly':
            data_array.append(entry['y'])

    df = DataFrame(data_array)

    if selected_operation == 'median':
        sum_list = df.median(axis=0)
    elif selected_operation == 'mean':
        sum_list = df.mean(axis=0)
    elif selected_operation == 'min':
        sum_list = df.min(axis=0)
    elif selected_operation == 'max':
        sum_list = df.max(axis=0)
    else:
        sum_list = []

    if len(sum_list) > 0:
        trace_array.append({'x': trace_array[0]['x'],
                            'y': sum_list,
                            'name': selected_operation,
                            'line': {'color': 'purple'},
                            'meta': {'computed': True, }
                            })
    figure['data'] = trace_array
    return figure

Note that the graph that sent back to the server and saved now can also contain this computed trace that wasn’t originally there when the page was initially rendered.

Dash Standalone Report

Lastly, this is the standalone dash report. The save button, save function and computedGraph function have all been removed and a custom MyDash class is used to inject the layout and dependencies content:

class MyDash(Dash):
    # initial layout and dependency script element
    def layout_script_elements(self):
        return '''<script id="_dash-layout" type="application/json">{auto_layout}</script>
                <script id="_dash-dependencies" type="application/json">{auto_deps}</script>'''.format(
            auto_layout=self.serve_layout().response[0].decode("utf-8"),
            auto_deps=self.dependencies().response[0].decode("utf-8"))

    # Called by Dash/Flask during startup for root page definition
    def interpolate_index(self, **kwargs):
        local_scripts = """<script src="/assets/ramda.min.js"></script>"""
        return '''<!DOCTYPE html>
        <html>
            <head>
                <title>Plotly/Dash Demo</title>
            </head>
            <body>
             {app_entry}
                {config}
                {auto_layout}
                {local_scripts}
                {scripts}
                {renderer}
            </body>
        </html>'''.format(app_entry=kwargs.get('app_entry'),
                          auto_layout=self.layout_script_elements(),
                          config=kwargs.get('config'),
                          scripts=kwargs.get('scripts'),
                          local_scripts=local_scripts,
                          renderer=kwargs.get('renderer'),
                          )

A clientside function for the computed trace (using Ramda) is provided to replace the server call:

app.clientside_callback(
    """
    function computed_trace(aggregateOp, rData, fig) {
            
        const traceArray = R.compose(
            R.reject(R.prop('meta')),
        )(fig.data)
        
        const dataArray = R.compose(
            R.map(R.prop('y')),
            R.reject(R.propEq('visible', 'legendonly')),
        )(traceArray)
        
        const transposedData = R.transpose(dataArray)
        var sum_list = null
        switch (aggregateOp) {
            case 'min':
                sum_list = R.map(R.reduce(R.min, Infinity))(transposedData)
                break 
            case 'max':
                sum_list = R.map(R.reduce(R.max, -Infinity))(transposedData)
                break 
            case 'median':
                sum_list = R.map(R.median)(transposedData)
                break
            case 'mean':
                sum_list = R.map(R.mean)(transposedData)
                break
        }
        if (sum_list != null)
            traceArray.push({
                x: fig.data[0].x ,
                y: sum_list,
                line: { color: 'purple' },
                name: aggregateOp,
                meta: { computed: true },
            })
        // clientside, this must be a deep copy to avoid object identity issues. 
        const newFig = JSON.parse(JSON.stringify(fig));
        newFig.data = traceArray
        return newFig
    }
    """,
    Output('graph', "figure"),
    [Input('dropdown', 'value'),
     Input('graph', "restyleData"),
     ],
    State('graph', "figure"),
    prevent_initial_call=True
)

Rather than a save that happens on the server, I can save the html recieved on the client with curl/sed and make my standalone dash report.

The sed script rewrites script src file references to use ./lib equivalents instead and removes some fingerprinting to get back to original file names:

curl localhost:8050 | sed '/<script[ ]\+src\=/ {s#/_dash-component-suites/[^/]*/#./lib/#;s#/assets/#./lib/#;s#\.v[0-9m0-9_]*##;s#\?m=[0-9\.]*##}' > "$DEST_FILE"

I now have a self contained Dash view on my embedded data that can be treated as a unit for archival or sharing with coworkers.