How can I build plotly heatmap with large data and slider faster?

📊 Plotly Python
1

I am trying to make a heatmap plot with slider using python plotly

Here is my sample code.

histogram_meter = np.random.randint(0, 500,(50,100))

# get figures
fig = go.Figure()

# add sliders
cnt_max = int(histogram_meter.max())           

if(len(tmp_histogram) != 0):
    avg = histogram_meter[histogram_meter != 0].mean()
    std = histogram_meter[histogram_meter != 0].std()
else:
    avg = 0
    std = 0
step_max  = int(avg + 2 *std)  # 2-sigma                      
step_list = list(range(0, step_max))
time_check = time.time()
for step in step_list:
    fig.add_trace(
        go.Heatmap(
            visible=False,
            z=histogram_meter,
            zsmooth='fast',
            connectgaps=True,
            colorscale=[[0, 'rgba(0, 0, 255,0)'],
                        [0.1, 'rgba(0, 0, 255,1)'],
                        [1.0, 'rgba(255, 0, 0, 1.0)']],
            zmin=0,
            zmax=step
        )
    )

print("time check for drawing heatmap = ", time.time()-time_check)
time_check = time.time()
fig.data[step_max-1].visible = True

time_check = time.time()
steps = []
for i in range(step_max):  # range(len(fig.data)):
    # time_check2 = time.time()
    step = dict(
        method="update",
        args=[
            {"visible": [False]*len(fig.data)},
            {"title": "Threshold switched to : " + str(i)}],  # layout attribute
        label=" {}".format(i)
    )
    # Toggle i'th trace to "visible"
    step["args"][0]["visible"][i] = True
    steps.append(step)
    # print("time check add 1-step to slider = ", time.time()-time_check2)

print("time check for drawing slider = ", time.time()-time_check)
time_check = time.time()

sliders = [dict(
    active=step_max-1,
    currentvalue={"prefix": "Threshold: "},
    pad={"t": 50},
    steps=steps
)]
fig.update_layout(sliders=sliders)
print("time check for update_layout(slider) = ", time.time()-time_check)
time_check = time.time()

# configure color bar
colorbar = dict(
    title="count",
    thicknessmode="pixels", thickness=50,
    lenmode="pixels", len=400,
    yanchor="top", y=1,
    ticks="outside", ticksuffix="count",
    dtick=5
)
fig.update_layout(coloraxis_colorbar=colorbar)
fig.update_layout(font_size=20)
print("time check for update_layout(color_bar) = ", time.time()-time_check)
time_check = time.time()
fig.show()

print("end")

when number of sample data is 100,

time check for drawing heatmap =  0.15591144561767578
time check for drawing slider =  0.0
time check for update_layout =  0.0879526138305664
time check for update_layout(color_bar) =  0.1019136905670166

and plot is generated in a second.

when number of sample data is 500,

time check for drawing heatmap =  0.4447438716888428
time check for drawing slider =  0.007018089294433594
time check for update_layout(slider) =  0.47070741653442383
time check for update_layout(color_bar) =  0.08794975280761719

In this case, it takes too long for the heatplot to be drawn. (Exactly, the heatmap plot appears to be still loading. )

I think, there are some problem in my code. Is there any way to improve my code?

Can you help me??

2

Hi @hyojoo.cho

If you don’t provide data it is difficult to just look at your code and conclude what makes it slow.

This is an example of many columns heatmap with slider, which is very fast:

import numpy as np
from random import choice
import string
import plotly.graph_objs as go

n= 90  # n =  number of horizontal cells in heatmap
m = 10  #m= number of vertical cells in heatmap

heat = go.Heatmap(z=np.random.randint(10, 25, (m, n)), 
                  colorscale = 'Oranges',
                  colorbar=dict(thickness=20))



y = [[i]*n for i in range(m)] 
y = [item for sublist in y for item in sublist]

#generate random annotations of length 5
text = ["".join([choice(string.ascii_lowercase) for i in range(5)]) for j in range(m*n)]
webgl_text = go.Scattergl(x=list(range(n))*m,
                          y=y,
                          mode='text',
                          text= text,
                          textfont=dict(color='black',
                                        size=12))


fig = go.Figure(data=[heat, webgl_text], 
                layout=dict(height=500, width=1000,
                            xaxis_range=[-0.5, 25.5], 
                            showlegend=False))
                            

steps = [{'args': [{'xaxis.range': [w-0.5, 15.5 + w]}], 
                    'method': 'relayout'} for w in range(n-15)]  #show approx 15 horizontal cells per step
                               
                               
fig.update_layout(sliders=[dict(active = 0,
                                minorticklen = 0,
                                steps = steps)]);
3

Thank you for answering my question.
but when I tested it and did fig.show (), the graph was not drawn on the web page. To be accurate, it seems to be loading continuously.

Actually, I provided sample data at first line. :slight_smile:
I have 2d-histogram data which type is numpy array.
but original source data is x,y pose tuple list
like
[(x0, y0), (x1, y1), (x2, y2), … , (xn, yn)]

and I can change 2d histogram like
… 0 1 2 3 4 5 6 …X
0 1 0 0 0 0 1 0 …0
1 2 0 3 0 0 3 9 …0
2 2 0 3 0 0 3 9 …0
. 2 0 3 0 0 3 9 …0
. 2 0 3 0 0 3 9 …0
. 2 0 3 0 0 3 9 …0
Y 2 0 3 0 0 3 9 …0

4

@hyojoo.cho

Sorry, it seems that I copied your code only starting from fig=go.Figure().

I don’t know why you are saying that my heatmap doesn’t show up. I use Chrome. Here is the corresponding heatmap posted on Plotly cloud: https://chart-studio.plotly.com/~empet/15573