📣 Announcing the Storage component

[Philippe]

Hello Dash Community –

We have a new component coming up for dash core components, the Storage component. It is a wrapper around the WebStorage api and also just to store data in memory on the client without the hidden div hack.

You can try it out right now with:

pip install dash-core-components==0.30.0rc1

Report any issue or comment on the Github PR

Basic Usage:

import dash

from dash.dependencies import Output, Input, State

import dash_html_components as html
import dash_core_components as dcc
from dash.exceptions import PreventUpdate

app = dash.Dash(__name__)
app.scripts.config.serve_locally = True

app.layout = html.Div([
    # Default storage is memory, lost on page reload.
    dcc.Storage(id='memory'),
    # local storage_type is only removed manually.
    dcc.Storage(id='local', storage_type='local'),
    # session storage_type is cleared when the browser exit.
    dcc.Storage(id='session', storage_type='session'),
    html.Button('click me', id='btn'),
    html.Div(id='output')
])


@app.callback(Output('local', 'data'),
              [Input('btn', 'n_clicks')], [State('local', 'data')])
def init_data(n_clicks, data):
    if n_clicks is None:
        raise PreventUpdate

    data = data or {}

    return {'clicked': data.get('clicked', 0) + 1}


@app.callback(Output('output', 'children'),
              [Input('local', 'modified_timestamp')],
              [State('local', 'data')])
def output(ts, data):
    if ts is None:
        raise PreventUpdate

    data = data or {}

    return 'Clicked: {}'.format(data.get('clicked', 0))


if __name__ == '__main__':
    app.run_server()

[Sylvos]

Seems to have affected the DatePickerSingle，is this performance better than a hidden div?

[Philippe]

The performance should be better for large dataset, there is only one de/serialization on the back end instead of having to do it manually a second time. Also on the front end, the data is not inserted in the DOM which can induce some latency. The fastest option would be the memory, the data is kept as is where as with session and local there is a serialization step for putting/retrieving from storage.

I could not reproduce a DatePickerSingle issue, mind sharing code that is Problematic ?

[Philippe]

I plan to release the Storage component early next week, it will be renamed to the result of this poll. Last chance to vote.

[Sylvos]

@Philippe

option would be the memory

What is the size limit for each option?

[Philippe]

Memory would be limited by ram and network usage when used in callbacks.

For the WebStorage api (‘local’, ‘session’), it depends on the browser but on firefox it is 10MB per origin. So the total of all your session and local storage components should not exceed 10MB. They also takes up memory as props.

[amarv]

This would be a huge improvement. Thanks @Philippe!

[Philippe]

The Storage component has been renamed to Store according to the poll results.

Released in dash-core-components==0.32.0

[brad]

1. I’ve started playing around using storage_type='local'. On load, my store object always initializes with data and modified_timestamp as None. Is there a way to access localStorage to populate an object on page load? Maybe I’m not using Store properly. For simplicity, I have added to my layout:

    dcc.Store(
        id='relevant-key-id',
        storage_type='local'),
    dcc.Interval(
        id='check-relevant-key-id-data',
        interval=30000,
        n_intervals=0,
        max_intervals=1)

I am using Input('check-relevant-key-id-data', 'n_intervals') in a callback to access the contents of State('local-store-data', '[data or modified_timestamp property]') and print to logs.

2. What’s best practice for handling non-standard objects? I have been relying on json.dumps(jsonpickle.encode(obj)) and .decode(str) in my hidden divs.

3. Any thoughts regarding dash-based PWA?

   • service workers for application code
   • localStorage for data cache

[brad]

For that matter, is modified_timestamp supposed to update automatically? It doesn’t seem to be updating when I output a value to the data prop.

$ grep dash* requirements.txt
dash==0.28.2
dash-core-components==0.33.0
dash-html-components==0.13.2
dash-renderer==0.14.3

[Philippe]

Getting the initial data with local and session storage_type is a bit tricky because of the None callbacks and the way the dash-renderer handle the initial callback. If you have an Output to the data prop of a dcc.Store, it will block an Input of the same Store data prop. You can circumvent this limitation by using the modified_timestamp prop as Input and use the store data as State.

Here’s an example:

import json
import random
import string

import dash

import dash_html_components as html
import dash_core_components as dcc
from dash.dependencies import Input, Output, State

app = dash.Dash(__name__)


app.layout = html.Div([
    dcc.Store(
        id='store',
        storage_type='local'
    ),
    html.Button(
        id='btn',
        children='click me'
    ),
    html.Div(id='output')
])


@app.callback(
    Output('store', 'data'),
    [Input('btn', 'n_clicks')],
    [State('store', 'data')]
)
def change_that_value(n_clicks, data):
    print('button was clicked')
    if n_clicks is None:
        raise dash.exceptions.PreventUpdate

    rname = (
        ''.join(
            random.choice(string.ascii_uppercase + string.digits)
            for _ in range(6)
        )
    )
    data = data or {}
    data[rname] = random.randint(0, 10)
    return data


@app.callback(
    Output('output', 'children'),
    [Input('store', 'modified_timestamp')],
    [State('store', 'data')]
)
def add_to_storage(timestamp, data):
    if timestamp is None:
        raise dash.exceptions.PreventUpdate

    print('current storage is {}'.format(data))
    print('modified_timestamp is {}'.format(timestamp))

    return json.dumps(data)


if __name__ == '__main__':
    app.run_server(debug=True, threaded=True, port=10888)

[brad]

In your example, after clicking btn the local storage should contain two keys:

store: dict(A1B2C3 = 2)
store-timestamp: [some epochtime]

As far as I can tell, when I start a new session, the following are true:

• output will be empty (modified_timestamp doesn’t get triggered until a button is clicked)
• State('store', 'data') = None
• State('store', 'modified_timestamp') = None

I don’t care as much about output. How do I get State(‘store’, ‘data’) to reflect what was written to storage during the previous session?

In my case, I’m trying something like this with an interval that fires every 30s.

@app.callback(
    Output('store', 'data'),
    [Input('store-update', 'n_intervals')],
    [State('store', 'modified_timestamp'),
     State('store', 'data')]
)
def test_store_object(n, last_updated, data):
    print('last_updated:\t{}'.format(last_updated))
    print('data:\t{}'.format(data))
    raise PreventUpdate

The print statements report both as None, even though data has been saved to the appropriate keys in Local Storage.

[dadv92]

@Philippe this addition is super nice. The hidden div solution was the only thing about dash I could label as “not so good”, but not anymore

I really want to thank you and the rest of the Dash team for making dash huge

[zoohair]

dash.exceptions.InvalidCallbackReturnValue:
                    The callback for property `data`
                    of component `localStore` returned a value
                    which is not JSON serializable.

                    In general, Dash properties can only be
                    dash components, strings, dictionaries, numbers, None,
                    or lists of those.

It’s unfortunate that dcc.Store only supports json serializable variables. I’ve managed to make it work for my purposes by writing my own Object -> string -> Object; that being said, it would be nice if dcc.Store dealt with any object.

Also, it would be useful to specify whether the data is stored client or server side, and what happens when it is used in a callback (is it sent to the server everytime?). What about when the server updates the data; is that sent back to the client?

[Damian]

@zoohair

it would be nice if dcc.Store dealt with any object.

This is a big no no when it comes to Python security. For example let’s say you are using pickle to turn serialize and deserialize an object, when the pickle loads happens any pickable code sent from the client can execute which is effectively any arbitrary code. For example:

import pickle
pickle.loads(b"cos\nsystem\n(S'ls'\ntR.")

Will run ls in whatever directory your program is running in, it would be trivial to get this to execute a bot on your server.

@zoohair

Also, it would be useful to specify whether the data is stored client

On the client, that is the purpose of dcc.store and the “hidden div” method before it. If you want to save data on your server there are lots of options, databases, file store, etc… The Flask-Sessions extension is worth looking in to.

[InsomniacCat]

@Philippe

dash-core-components==0.32.0

Hi, were you able to fix the DatePicker problem? I have the same issue with DatePickerRange.

[Philippe]

I think the DatePicker problem is unrelated to the dcc.Store, might of been affected in a previous release.

There’s a PR related to the DatePicker:

DatePickerSingle prop typos

I don’t see any other issues on the DatePicker, please create one with a minimal reproducible example if the issue persist after upgrading to the latest version of dash-core-components (0.38.0)

[matthewwhite]

I love Dash, the tabs and the interactive tables! I recently saw the new dcc.Store and I trying to use it in place of hidden divs. However, I have not succeeded yet and it causes the app to grind to halt. I have a multi-tab app, the first tab holds the interactive dash table that is fed by the data from the dcc.Store. The user then selects the issue in the table to view the related chart of the data on the next tab.

I had this working using the hidden div but it seems that I needed a separate hidden div for each DF result (the calculation subroutine spits out several DFs) I am only using the first resulting DF for the interactive table in tab 1. Note, that I run the calculation subroutine beforehand and use the DFs to initially populate the rows of the table in tab1, but I need to be able to update the table when I change configurations of the test calculations as well as with new results from the new data feed each day.

When I run the app now using dcc.Store in Jupyter notebook, the interactive table is not filled, and does not fill even after click the “run_tests” button and/or select from one of the dropdowns. I do see the data in the Session Storage of Chrome under the Application section of developer mode. First it is populated with the data from the dataframe itself since the modified_timestamp is None, then that is overridden and populated by the data.to_dict() after button is pressed. However, the modified_timestamp does not show in the Session Storage. So the data are there but my app does not seem to be able to read it back. The app does not list any errors related to this tab in the feedback pane

I appreciate any insight.

Here is the portion of my code that is relevant to this.

app.layout = html.Div([ 
    dcc.Store(
        id='store_data',
        storage_type='session'
    ),
    
	dcc.Tabs(id="tabs", vertical=False, children=[
        dcc.Tab(label='Issues', value='tab1', children=[
            html.Div([
                html.H1('Issues Failing Test Configuration Criteria',style={
            'textAlign': 'left', 'margin': '10px', 'fontFamily': 'Sans-Serif','font-size':16}),
            ]),
            
            html.Div([
                dcc.DatePickerRange(id='issues_date_range',
                    min_date_allowed=DataDF.index.min(),
                    max_date_allowed=DataDF.index.max(),
                    initial_visible_month=DataDF.index.max(),
                    start_date=DataDF.index.max()-timedelta(days=6), #default to the lastest week of data
                    end_date=DataDF.index.max(),                                   
                    updatemode='bothdates')                
            ], style = {'display': 'inline-block', 'width': '30%', 'height': '40', 'vertical-align': 'middle', 'float':'right','position': 'relative'}),
            
            html.Div([
                html.Button('Run All Tests', id='run_tests_button', n_clicks=0)
            ], style = {'display': 'inline-block', 'vertical-align': 'middle', 'float':'right','position': 'relative'}),
            
            html.Div([
                dcc.Dropdown(
                    id='issue_type_dropdown',
                    options=[
                        {'label': 'a', 'value': 1},
                        {'label': 'b', 'value': 2},
                        {'label': 'c', 'value': 3},
                        {'label': 'd', 'value': 4}
                    ],
                    value=1',
                    #placeholder="Select Issue Type",
                    multi=False)
            ], style = {'float': 'left', 'position': 'relative','width': '20%', 'vertical-align': 'middle'}),
            
            html.Div([
                dcc.Dropdown(
                    id='group_dropdown',
                    options=[
                        {'label': 'Group#1', 'value': 1},
                        {'label': 'Group#2', 'value': 2},
                        {'label': 'Group#3', 'value': 3},
                        {'label': 'Group#4', 'value': 4}
                    ],
                    value=1,
                    #placeholder="Select Group",
                    multi=False)
            ], style = {'position': 'relative','width': '20%', 'vertical-align': 'middle'}),
            
            html.Div([
                dt.DataTable(
                    rows=[],
                    columns=[],
                    column_widths=[],
                    editable=False,
                    row_selectable=True,
                    filterable=True,
                    sortable=True,
                    selected_row_indices=[],
                    max_rows_in_viewport=9,
                    id='issues_table'
                )
            ])
        ]),
### more tabs,etc. Here are the related calllbacks ###

        @app.callback(  ### this runs the subroutine and sends the resulting DF to store_data
        Output('store_data', 'data'),
        [Input('run_tests_button', 'n_clicks')],
        [State('issues_date_range','start_date'),
         State('issues_date_range','end_date')])
    def update_issues_table(n_clicks,start,end):
        global Issues
        if n_clicks==0:
            DataDict=Issues.to_dict()
        else:
            RunTests=AllTests(start,end) #calls subroutine from another notebook that returns multiple DFs, for now I am only trying to send the first DF, hence the [0].
            DataDict=RunTests[0].to_dict() 
        return DataDict
    	
    @app.callback(   ### this pulls the DF from the store_data, manipulates the DF to pull the rows that apply to the two dropdowns and sends the rows to the interactive table
        Output('issues_table', 'rows'),
        [Input('issue_type_dropdown', 'value'),
         Input('group_dropdown', 'value')],
        [State('store_data', 'data'),
         State('store_data', 'modified_timestamp')])
    def subset_issues_table(issue_type,bldg_group,data,data_timestamp):
        if data_timestamp is None:
    		raise dash.exceptions.PreventUpdate
        else:
    		subset=pd.DataFrame()
    		StoredIssues=data[0]
    	### slicing of data based on the two dropdowns to produce the "subset" DF  ###
    		subrows=subset.to_dict('records')
    		return subrows

Also, when I stop the app and restart it again with the data still in the Session Storage, it yields, “Error loading dependencies” and states in the Chrome Application area “RangeError: Maximum call stack size exceeded”.

[brad]

Also relating to session and local storage, can someone provide an example of how to retrieve the initial store data at load time? The documentation recommends modified_timestamp input and loading the data as State. However, when I am still seeing the initial state data coming up empty.

Example: the click count example in the documentation will not read the local/session click information on page reload. I also c/p the snippet and tried to read local storage with an interval since I wasn’t sure if the modified_timestamp would get triggered at load time

Retrieving the initial store data

If you use the data prop as an output, you cannot get the initial data on load with the data prop. To counter this, you can use the modified_timestamp as Input and the data as State .

This limitation is due to the initial None callbacks blocking the true data callback in the request queue.

See Filter nil values before triggering initial layout callbacks. by T4rk1n · Pull Request #81 · plotly/dash-renderer · GitHub for further discussion.

Looks like the None callbacks behavior will not change for awhile (dash-renderer v2.x)

[Mike3]

This is an incredibly useful new feature for Dash, and adds a crucial flexibility in allowing developers to store data as they see fit. Does dcc.Store include some functionality for sequentially saving more data, and adding to what was saved before; for example, when a live feed is coming in every few seconds, and you want to append to the data already saved?