Hello,
Some of my apps have stopped working after updating the environment.
On the dash-extensions changelog, they inform us of the following:
The syntax of the ServersideOutputTransform has been changed. Instead of using ServersideOutput in place of the Output, one must now wrap return values in Serverside objects
However, I have been unable to find any updated documentation.
Any examples of the new syntax?
Best,
Ed
OK, I found it:
I have not tested it yet.
It is not working.
I am importing Serverside instead of ServersideOutput.
I have replaced ServersideOutput by Output on the callback.
My callback, returns four objects: three figures and a dictionary:
return fig_featimp, fig_rescore, fig_perf, Serverside(shap_new)
I only want to cache the dictionary on the server because it is quite big.
It complains about the first figure not being JSON serializable:
dash.exceptions.InvalidCallbackReturnValue: The callback for `[<Output `feat-importance-new.figure`>, <Output `new-scores-dist.figure`>, <Output `model-performance.figure`>, <Output `shap-values.data`>]`
returned a value having type `Figure`
which is not JSON serializable.
However, I am not trying to cache any figure. It was working before.
Any ideas?
How did you setup your transforms/backend(s)? Would it be possible for you to post a MWE that demonstrates the issue?
This error basically means that the ServersideOutputTransform was not applied (which could be due to a configuration issue). Hence, the default Dash behaviour of serialising the data is attempted - and that fails (thus yielding the error you see).
EDIT: Could the error be due to one of the figure objects?
These are my relevant imports:
from dash_extensions.enrich import DashProxy, html, dcc, Input, Output, Serverside, ServersideOutputTransform, Dash, dash_table, callback, ctx, State
from dash.exceptions import PreventUpdate
import dash_ag_grid as dag
This is how I define the app:
app = DashProxy(transforms=[ServersideOutputTransform()])
Before it was:
app = DashProxy(__name__, transforms=[ServersideOutputTransform()])
It does not seem to make any difference.
If I remove the Serverside() wrap it works, only that more slowly.
Could it be an issue with having multiple outputs and only one being wrapped?
If I move the server-side output to first position, I receive the following message:
returned a value having type `Serverside`
which is not JSON serializable.
So, yes, it would seem that Dash does not understand Serverside.
However, I am running the latest pip versions of all the libraries.
Are you using the callback or app.callback decorator?
app.callback
I had to downgrade the package as well with the exact same problem. I’ve been storing custom objects in a dcc.store (which has been a godsend), but was getting the exact same error when I upgraded and swapped over the syntax. I’ve been a bit swamped this week, but have been meaning to post it on github.
I was using the callback decorator, not the app.callback decorator. I’ll look back at some code and try and post a simple example.
Well, sure enough, I couldn’t recreate with a MWE. Here is the code I ran (and it works just fine):
from dash_extensions.enrich import DashProxy,ServersideOutputTransform, html,callback,dcc,Serverside,Input,Output
app = DashProxy(
__name__,
transforms=[ServersideOutputTransform()]
)
app.layout = html.Div([
dcc.Location(id='url', refresh="callback-nav"),
dcc.Store(id='test'),
html.Div('BLAH')
])
class Blah:
def __init__(self):
self.blah = '1'
@callback(
Input('url','pathname'),Output('test','data')
)
def testFunc(path):
tt = Blah()
return Serverside(tt)
if __name__ == '__main__':
app.run_server(debug=True,port=8050)
I’m going to swap back over the production code that was giving me problems and see if I just imagined the problem. I think the MWE might be running on python 3.9 and the production code python3.7?
As an aside, really love the package. I’ve been looking for a workaround to save objects to a store for a while (was having to dump them to a dict then recreate them when loading the store). Just anecdotally, I’d say swapping over the dash-extensions sped things up by 10-15%.
Could the issue be caused by having a pre-existing store created with the previous version?
Anyway, I am going to downgrade.
Downgrading fixes the issue.
Finally, I found a few minutes to further research this issue. It seems that the issue only occurs with Pandas dataframes:
The example from the documentation works.
However, if I modify it slightly to store a dataframe it does not (dash-extensions==1.0.0):
import time
import plotly.express as px
from dash_extensions.enrich import DashProxy, Output, Input, State, Serverside, html, dcc, \
ServersideOutputTransform
app = DashProxy(transforms=[ServersideOutputTransform()])
app.layout = html.Div(
[
html.Button("Query data", id="btn"),
dcc.Dropdown(id="dd"),
dcc.Graph(id="graph"),
dcc.Loading(dcc.Store(id="store"), fullscreen=True, type="dot"),
dcc.Store(id="store-df")
]
)
@app.callback(Output("store", "data"), Input("btn", "n_clicks"), prevent_initial_call=True)
def query_data(n_clicks):
time.sleep(3) # emulate slow database operation
return Serverside(px.data.gapminder()) # no JSON serialization here
@app.callback(Output("dd", "options"), Output("dd", "value"), Input("store", "data"), prevent_initial_call=True)
def update_dd(df):
options = [{"label": column, "value": column} for column in df["year"]] # no JSON de-serialization here
return options, options[0]['value']
@app.callback(Output("store-df", "data"), Output("graph", "figure"), [Input("dd", "value"), State("store", "data")], prevent_initial_call=True)
def update_graph(value, df):
df = df.query("year == {}".format(value)) # no JSON de-serialization here
return Serverside(df), px.sunburst(df, path=["continent", "country"], values="pop", color="lifeExp", hover_data=["iso_alpha"])
if __name__ == "__main__":
app.run_server()
But, if I revert to the previous version of the extensions it does work (dash-extensions==0.1.13):
import time
import plotly.express as px
from dash_extensions.enrich import DashProxy, Output, Input, State, ServersideOutput, html, dcc, \
ServersideOutputTransform
app = DashProxy(transforms=[ServersideOutputTransform()])
app.layout = html.Div(
[
html.Button("Query data", id="btn"),
dcc.Dropdown(id="dd"),
dcc.Graph(id="graph"),
dcc.Loading(dcc.Store(id="store"), fullscreen=True, type="dot"),
dcc.Store(id="store-df")
]
)
@app.callback(ServersideOutput("store", "data"), Input("btn", "n_clicks"), prevent_initial_call=True)
def query_data(n_clicks):
time.sleep(3) # emulate slow database operation
return px.data.gapminder() # no JSON serialization here
@app.callback(Output("dd", "options"), Output("dd", "value"), Input("store", "data"), prevent_initial_call=True)
def update_dd(df):
options = [{"label": column, "value": column} for column in df["year"]] # no JSON de-serialization here
return options, options[0]['value']
@app.callback(ServersideOutput("store-df", "data"), Output("graph", "figure"), [Input("dd", "value"), State("store", "data")], prevent_initial_call=True)
def update_graph(value, df):
df = df.query("year == {}".format(value)) # no JSON de-serialization here
return df, px.sunburst(df, path=["continent", "country"], values="pop", color="lifeExp", hover_data=["iso_alpha"])
if __name__ == "__main__":
app.run_server()
Is this a bug, a feature or a subproduct of my ignorance?
No, the problem is not a pandas dataframe per se. It seems that my initial intuition was correct. The issue may be having multiple outputs. If the only serverside output is a dataframe, it works:
import time
import plotly.express as px
from dash_extensions.enrich import DashProxy, Output, Input, State, Serverside, html, dcc, \
ServersideOutputTransform
app = DashProxy(transforms=[ServersideOutputTransform()])
app.layout = html.Div(
[
html.Button("Query data", id="btn"),
dcc.Dropdown(id="dd"),
dcc.Graph(id="graph"),
dcc.Graph(id="graph-2"),
dcc.Loading(dcc.Store(id="store"), fullscreen=True, type="dot"),
dcc.Store(id="store-df")
]
)
@app.callback(Output("store", "data"), Input("btn", "n_clicks"), prevent_initial_call=True)
def query_data(n_clicks):
time.sleep(3) # emulate slow database operation
return Serverside(px.data.gapminder()) # no JSON serialization here
@app.callback(Output("dd", "options"), Output("dd", "value"), Input("store", "data"), prevent_initial_call=True)
def update_dd(df):
options = [{"label": column, "value": column} for column in df["year"]] # no JSON de-serialization here
return options, options[0]['value']
@app.callback(Output("graph", "figure"), [Input("dd", "value"), State("store", "data")], prevent_initial_call=True)
def update_graph(value, df):
df = df.query("year == {}".format(value)) # no JSON de-serialization here
return px.sunburst(df, path=["continent", "country"], values="pop", color="lifeExp", hover_data=["iso_alpha"])
@app.callback(Output("store-df", "data"), Input("store", "data"), prevent_initial_call=True)
def test_df_store(df):
df = df[(df['year'] == 1952)] # no JSON de-serialization here
return Serverside(df)
@app.callback(Output("graph-2", "figure"), Input("store-df", "data"), prevent_initial_call=True)
def update_graph_2(df):
return px.sunburst(df, path=["continent", "country"], values="pop", color="lifeExp", hover_data=["iso_alpha"])
if __name__ == "__main__":
app.run_server()
Thank you for the example @edmoman ! It’s definitly a bug. I missed the case of tuple return type (all my multi output tests were using list outputs). If you return a list instead it should work. I am preparing a new release to fix the issue as we speak.
EDIT: The new 1.0.1 release (available on pypi just now) should fix the issue.