I’m new to Dash. I’m trying to set up an interactive system for analyzing large amount of data. I run into trouble using heatmaps.
Some of the heatmaps I want to display are large (say, ~6000 * 6000 data points). Sure, this is beyond the resolution of the (4K) screen, but it allows zooming into areas of interest. The data is available in a Pandas data frame (data type is float32).
- Low Performance
Trying to simply display the data using Heatmap(x=frame.columns, y=frame.index, z=frame.values) takes a long time. At first, it took >10 minutes, after which I gave up on waiting. I reduced this to “only” around half a minute when I specified zsmooth=False.
Does the backend try to smooth/resample the data to match the displayed size on the browser? That sounds strange, the full data still needs to be sent to the browser to allow for zooming and panning. Also, even if the code does some smoothing, it seems to be extremely inefficient. Applying a convolution to a 6000 * 6000 matrix shouldn’t take even a second on a CPU that can execute a few billions instructions per second.
I am using the built-in “debug” server, but that shouldn’t matter that much. I would expect that it would make a big difference when handling a lot of parallel requests etc., but for a single request single user scenario, using a “debug” server shouldn’t matter… right?
I tried using Heatmapgl and that didn’t solve any of the issues. It did however lose the axis labels. I never had good experience with Heatmapgl even in the offline mode so I gave up on it.
Is a 6000 * 6000 entries heatmap simply a lost cause in Dash+Plotly? If so, what is the alternative for showing large raw matrix data?
- I Can’t trust what I see
Given that regardless of heatmap issues, there will always be some requests for data that may take many seconds to compute, it is impossible to tell whether the currently visible data is/n’t up-to-date. It would be extremely useful to have some sort of visible indicator informing the user whether the backend is still working.
I tried to look for a feature like this in react, it seems they have a “Suspense” component that might do the trick (and it seems Dash uses it when loading the initial page), but I’m not certain on how I can wrap it around my slow-to-compute elements using Python. Does anyone have any idea if this could be done and how, or suggest an alternative?
- Broken Layout (Edit: not solved)
I tried to specify autosize=True in the layout dictionary, and added config=(autosizable=True, responsive=True) to the dcc.Graph element, but all that achieved was making it impossibly slow to even send a 100 * 100 heatmap; the layout remained broken.
Is there some other setting that may fix this?
Edit: Searching further I discovered that for some reason the default size is 450px unless one specifies an explicit height for the graph (div). That works well if you set the height of the div (or its relevant ancestor) to something “absolute”, such as 800px or 100vh (full height). However, I don’t think it is possible to fill the remaining height after some other HTML elements too over some height. The CSS way to do this is to use height=100%, and when that is used, Dash sets the graph height to 100px (!). This seems like a bug, so I have reported it as such in https://github.com/plotly/dash/issues/857
- Aspect ratio
I tried to use xaxis=dict(scaleanchor=‘y’) to force the pixels to stay square, regardless of zooming. This had no effect whatsoever. Are there additional settings required to make this work?