Just a minor addition to your great example: You have to make a deep copy of the figure object. Otherwise you might run into issues when adding elements to an existing list, e.g. such as traces.
I was able to completely decouple any figure modification (e.g. adding descriptive shapes, traces, annotations etc.) while keeping the original (large) figure data in a dcc.Store object and not resend it on every update (of the auxiliary data).