Just to follow up as I used to have this issue with scattermapbox and I must admit I cannot remember how I solved it because it is not happening anymore.
However, if you Ctrl-scroll or other zoom change, window change etc, the figure should be displayed normally without re-computing.