Show and Tell - Server Side Caching

Dash Python
,
12

Hi, I got time to make some testing and benchmarking for the CallbackCache. Here are the results:

Testing CallbackCache

Now I tested with example code of adamschroeder using his example data, and this is what I see in the dev tools:

image
image1652×599 48.4 KB

Each time I pressed the “Run benchmark (with cache)” -button, there would be two HTTP POST requests to http://127.0.0.1:8050/_dash-update-component, (one after another since they are chained). You’ll have one HTTP POST request per each callback that is triggered.

  • The first _dash-update-component is fast and response size is very small, about 400Bytes, content is
    {"response": {"store_wc": {"data": "28de8ed6118d3dc73f39311bf5de1910"}, "time_wc": {"data": "83c5acc6308b231e3e56493fcf3774e1"}}, "multi": true}
    This corresponds the ID’s of the dcc.Store component. With this ID the data is read from the cache, if I understand correctly.

    I think here is the gain: Since only ID is saved to the browser, you do not have to send data from browser (dcc.Store) to server each time a callback is called.

  • The second _dash-update-component is slow (with large TTFB1; about 5-7 seconds). This contains the Graph data as JSON which is drawn to browser. Response size is about 1.8Mb.

As a diagram, this would be something like (3x small JSON, 1x large JSON, df to JSON only once)

image
image725×247 32.4 KB

So, what is taking the time in the second part?

The creation of the figure with px.scatter(df_filtered) takes about 5 seconds. Funny thing that df_filtered.to_json takes only ~0.5 second, so actually 95% of the time used to create a plotly figure is something else than just creating a json object out of it. (some optimization possible in px.scatter(), perhaps? Moreover, it could be possible to memoize calc_wc, too!)

Problem with caching?

I tried the mentioned code + this change

cc = CallbackCache(cache=FileSystemCache(cache_dir="cache"), instant_refresh=True)

@cc.cached_callback([Output("store_wc", "data"), Output("time_wc", "data")],
                    [Trigger("btn_wc", "n_clicks")])
def query_wc():
    print('Calculating')
    import time
    time.sleep(5) # Added sleep
    df = df_org[["passenger_count", "trip_distance", "total_amount"]]
    return df, datetime.datetime.now()

and I was hoping to see the time to be 5 seconds with first callback call and ~0 seconds during the next ones. Unfortunately, it did not work like that. I tried both instant_refresh options (True/False). You can see this yourself from the dev console, looking at the Timing of the first _dash-update-component. It could be a bug, configuration issue, or just how it should work. My best guess is the last; the CallbackCache will not memoize the callback, but just serves as “Store” in the server side. That said, this could be memoized easily also without dash-extensions, (if there is no such functionality?) so I assume there could be additional speed gains, if needed.

Compare to without CallbackCache

The callbacks without CallbackCache would be roughly something like this

@app.callback([Output("store_wc", "data"), Output("time_wc", "data")],
                    [Input("btn_wc", "n_clicks")])
def query_wc(n_clicks):

    df = df_org[["passenger_count", "trip_distance", "total_amount"]]
    return df.to_json(), datetime.datetime.now()


@cc.callback([Output("log_wc", "children"), Output("mygrpah", "figure")],
             [Input("store_wc", "data")], [State("dd_wc",'value'), State("time_wc", "data")])
def calc_wc(df, value, time):
    df = pd.read_json(df)
    toc = datetime.datetime.now()

    df_filtered = df[df["passenger_count"] == value]

    fig = px.scatter(df_filtered, x='trip_distance', y='total_amount')
    return ("ELAPSED "), fig

As a diagram, this would be something like (1x small JSON, 3x large JSON. df to JSON 2 times, JSON to df 1 time)

image
image701×266 36.8 KB

Results

  • First callback takes ~5.3 seconds and response is 6.7Mb! (this is the data as JSON string)
  • Second callback takes about ~10.5 seconds, since now it first sends the data as JSON from browser to server, and then gets the figure as response.
  • Summary: The CallbackCache saves now about 2/3 of time when there is dcc.Store involved with two chained callbacks + a Graph.
  • By caching / using memoization of the callback functions it could be possible to make this even faster.
    image
    image1584×358 35 KB

I hope this makes the gains more clear to everyone!

:bell: Note: The tests were done in localhost without throttling (the network speed is much faster than in normal situation). In real world application, sending the big JSON packages back and forth would be even slower, and the difference between using CallbackCache vs. not using it would be more dramatical.

- Niko

1The TTFB is defined as

The browser is waiting for the first byte of a response. TTFB stands for Time To First Byte. This timing includes 1 round trip of latency and the time the server took to prepare the response

4 Likes