Colors for discrete ranges in heatmaps

[empet]

@heff The heatmap trace selects the color from a colorscale, according to the normalized values in the lists of lists z (or a numpy array of shape (m,n)).

The discrete colorscale should be defined as follows:

• find the max value for z, let us say that it is 125:

• map the interval [0, max_val] to [0,1] by t--> t/max_val

• mapping your thresholds [0, 10, 50, 100, 125] to [0,1], you get
  [0, 0.08, 0.4, 0.8, 1].

• define the discrete colorscale:

  my_colorsc=[[0, 'rgb(255,255,255)'],#white
              [0.08, 'rgb(255,255,255'], 
              [0.08, 'rgb(255,165,0)'],#orange
              [0.4, 'rgb(255,165,0)'],
              [0.4, 'rgb(255,0,0'], #red
              [0.8, 'rgb(255,0,0)'],
              [0.8, 'rgb(0,0,255)'], #blue
              [1, 'rgb(0,0,255)']]
  

Plotly doesn’t recognize cmyk colors.

[jezlax]

@empet - thanks for this, it is a very handy guide. Do you have any inclination if an “absolute” colorscale is possible? My use case is that I’m reading data in from a DB, and showing on a scatter with a discrete colorscale based off a value in the DB. The rub is that I’m querying from the DB and limiting to one client each time. Each client has a different range of values to color based on (one might be from 30-70, another might be from 45-55, 16-62, etc… you get the idea). I’d like the colorscales to take to the absolute range like @heff was originaly trying to code it.

I welcome any thoughts here…thx

[empet]

@jezlax The normalization of range ends, above, was necessary to define the discrete colorscale. The original values are also normalized by plotly.js in order to map them to the coresponding color.

You only have to insert a suitable text as a colorbar ticktext (see the image below).

I detailed all ingredients needed to plot a heatmap with a discrete colorscale in this notebook https://plot.ly/~empet/15229.

[jezlax]

thanks @emp for the work. the only thing i’m running into is that plotly still normalized my Z inputs, when I want them to be treated absolutely. I provided an example here: https://plot.ly/~coryjez/3

Basically the scale is from 0 to 100, and I feed in say values between 34 and 60. It normalizes those values I feed in to be 0-100 now and plots them as such. Basically, I’ve created a percentile score for my client on the backend, and just want to place that percentile score on the range I’ve provided in that notebook. It’s seeming like this won’t be possible in plotly’s current manifestation of this, but wanted to see what you thought. thanks for the help.

[michaelbabyn]

@jezlax, you can force the inputs to be treated absolutely by setting a zmin and zmax value in your heatmap.

heatmap = go.Heatmap(z=z, 
                     colorscale = dcolorsc,
                     zmin=0,
                     zmax=100,
                     colorbar = dict(thickness=25, 
                                     tickvals=tickvals, 
                                     ticktext=ticktext))

[jezlax]

@michaelbabyn - right, those are super helpful! the rub i really run into here is that I can’t use those arguments in a go.Scatter() scenario, trying to plot some points and color by the scale I referenced in my post.

[michaelbabyn]

@jezlax, In that case you would use cmax and cmin like in this example https://plot.ly/python/colorscales/#colorscale-for-scatter-plots

Edited to fix example link.

[empet]

@jezlax

I really don’t understand your complaint. Why did you say that data with range in some subinterval ([34, 60]) of the interval [min(bvals), max(bvals)] (within the function discrete_colorscale() body
bvals[0] is the min value, and bvals[-1] is the max value, after sorting) is normalized with respect to that subinterval? It is normalized with respect to
min and max bvals. If you set some cmin, cmax values in a go.Scatter instance that are different from min(bvals), max(bvals), the associated color can be wrong
because the discrete colorscale was defined such that to map the interval [min (bvals), max(bvals)] to [0,1], [not cmin, cmax].

Running your notebook (with Python 3) and hovering some points in the heatmap led to right displayed values as you can see in these images.

[jezlax]

@empet - not meant to be a complaint at all. i didn’t realize that cmin/cmax arguments were available in go.Scatter() the same way that zmin/zmax are in go.Heatmap(). It looks like this solution scales perfectly, as you mentioned. Without the cmin/cmax being set equal to the range, in my case [0,100]… it did normalize on the subrange, but setting those arguments to 0, 100, respectively seems to work. appreciate the back and forth and the help.

[dewshrs]

Hi @empet, I am using custom colorscale for heatmap, but I am getting different color, shouldn’t the plot be giving limegreen and tomato color?

colorscale= [[0, 'whitesmoke'], [0.33, 'limegreen'], [0.67, 'tomato'], [1, 'teal']]

values = [[0,0,0,0,0,.67,.67,.67,.67,0,0,0,0,0,.33,.33,.33,.33,0,0,0,0],
          [0,0,0,0,0,.67,.67,.67,.67,0,0,0,0,0,.33,.33,.33,.33,0,0,0,0]
         ]

fig_bar = go.Figure(data=go.Heatmap(z=values, colorscale=colorscale,text=values,
                                     hoverinfo ='text',showscale=False)
                    
            )
                    

fig_bar.show()

[empet]

Hi @dewshrs,

To understand why you get a heatmap colored like this you have to know how the colormapping is performed, i.e the definition of a heatmap:

• the initial data are an array of z-values and a Plotly colorscale, consisting in a scale= [0, 0.33, 0.67, 1] (in your example), and the corresponding color names or color codes.

• the z-values are normalized by plotlyjs, via the mapping val -->n_val=(val-z_min)/(z_max-z_min).
  In your case this mapping is: val--> n_val =(val-0)/(0.67-0)

• to each normalized n_val one associates the corresponding color in the colorscale if n_val is an element in the list scale. Otherwise, the corresponding color is deduced by linear interpolation.

I think you expected to get mapped z=0.67, to the color 'tomato'. But by the above algorithm it isn’t because the color assigned to this z-value is derived as follows:

• n_val =(0.67-0)/(0.67-0)=1
• the color corresponding to 1 is teal.

If you want to associate to 0.67 the color tomato, just set in the Heatmap definition, zmin=0, zmax =1
otherwise (by default) plotly.js takes zmin = min(z-values), zmax= max(z-values).

[dewshrs]

thank you so much @empet.

[mycarta]

Hi @empet empet

What would be the proper definition if I wanted to use this with ff.create_annotated_heatmap ?
Creating the tuples without pre-pending rgb is easier but how would you pass the intervals toff.create_annotated_heatmap?

[empet]

Hi @mycarta,

To use such a discrete colorscale for an annotated heatmap, define the colorscale like in the notebook at the link above (https://chart-studio.plotly.com/~empet/15229/heatmap-with-a-discrete-colorscale/#/) , based on your values in the array, z. If dcolorsc is your colorscale, then an annotated heatmap is defined as follows:

import plotly.figure_factory as ff
z1=  np.random.randint(bvals[0],  bvals[-1]+1, size=(8, 8))  #bvals are defined in the notebook 
fig1 = ff.create_annotated_heatmap(z1, colorscale=dcolorsc)
fig1.update_traces(showscale=True, colorbar = dict(thickness=25, 
                                     tickvals=tickvals, 
                                     ticktext=ticktext))
fig1.update_layout(width=500, height=500)

[mycarta]

Outstanding, thank you.
Between what I had and what you showcased at that link, that got me really all the way to 90%. In actual facts I had my own code generating the colorscale, but it created a list with what you call bvals at that link, followed by a tuple with RGB values, like:

[[0.0, (64, 0, 75)],
 [0.03, (96, 18,100)]
...
...

So, all I needed to add was a bit that converted the tuple to hex values:
'#%02x%02x%02x' % (tuple(int(x) for x in(rgb[i])))

[jayeetamukherjee]

Hello ,
I am pretty new to plotly and trying to generate a heatmap , where all positive values will have red color, all negative values will be of green color, zero will have grey. Essentially like “If value >0 color =Green, else if value <0 color= red, else color=grey”. There can be only these three colors in the heatmap. Can this be done in plotly heatmap?
Thanks in advance

[empet]

Hi @jayeetamukherjee,
Your case needs the min&max of the subset of negative, respectively positive values:

import plotly.graph_objects as go
import numpy as np

def three_colorscale(z, colors):
    if len(colors) !=3:
        raise ValueError("")
    neg=z[np.where(z<0)]
    pos=z[np.where(z>0)]    
    a, c = neg.min(), neg.max()
    d, b = pos.min(), pos.max()
    bvals= [a, c/2, d/2, b]
    nvals = [(v-bvals[0])/(bvals[-1]-bvals[0]) for v in bvals] 
    dcolorscale = []
    for k in range(len(colors)):
        dcolorscale.extend([[nvals[k], colors[k]], [nvals[k+1], colors[k]]])
    return dcolorscale     

z= np.random.randint(-5, 7, size=(8,8))
pl_colorscale= three_colorscale(z, ["#19BD1B", "#C0C0C0", "#DC3714"])

fig= go.Figure(go.Heatmap(z=z, colorscale=pl_colorscale, xgap=1, ygap=1, colorbar_thickness=24))
fig.update_layout(width=400, height=400)

[jayeetamukherjee]

Worked like magic. Thank you so much.

[kabure]

Hey @empet

It worked on my side; Thank you very much!!

[miohtama]

If someone is still looking this question 5 years later, here is a little bit related problem and solution.

I had NaNs and zeros as values on the heatmap scale. Having such values in the scale will squeeze the rest of the heatmap scale. E.g. if you have 101, 100.5, 100, 99.5 and then NaN/0 at the last value then the heatmap is unreadable as Plotly scales everything 101…0 instead of 101…99.

The workaround for this is to convert your value items to strings, so Plotly won’t treat them as a number. Each scale value becomes a discrete item.

Below is what I did. [You can find the full heatmap visualisation code here](file:///Users/moo/code/executor/docs/build/html/programming/api/execution/help/tradeexecutor.analysis.grid_search.visualise_heatmap_2d.html#tradeexecutor.analysis.grid_search.visualise_heatmap_2d)

Example Python code:

def visualise_heatmap_2d(
        result: pd.DataFrame,
        parameter_1: str,
        parameter_2: str,
        metric: str,
        color_continuous_scale='Bluered_r',
        continuous_scale: bool | None = None,
) -> Figure:
    """Draw a heatmap square comparing two different parameters.

    Directly shows the resulting matplotlib figure.

    :param parameter_1:
        Y axis

    :param parameter_2:
        X axis

    :param metric:
        Value to examine

    :param result:
        Grid search results as a DataFrame.

        Created by :py:func:`analyse_grid_search_result`.

    :param color_continuous_scale:
        The name of Plotly gradient used for the colour scale.

    :param continuous_scale:
        Are the X and Y scales continuous.

        X and Y scales cannot be continuous if they contain values like None or NaN.
        This will stretch the scale to infinity or zero.

        Set `True` to force continuous, `False` to force discreet steps, `None` to autodetect.

    :return:
        Plotly Figure object
    """

    # Reset multi-index so we can work with parameter 1 and 2 as series
    df = result.reset_index()

    # Detect any non-number values on axes
    if continuous_scale is None:
        continuous_scale = not(df[parameter_1].isna().any() or df[parameter_2].isna().any())

    # setting all column values to string will hint
    # Plotly to make all boxes same size regardless of value
    if not continuous_scale:
        df[parameter_1] = df[parameter_1].astype(str)
        df[parameter_2] = df[parameter_2].astype(str)

    df = df.pivot(index=parameter_1, columns=parameter_2, values=metric)

    # Format percents inside the cells and mouse hovers
    if metric in PERCENT_COLS:
        text = df.applymap(lambda x: f"{x * 100:,.2f}%")
    else:
        text = df.applymap(lambda x: f"{x:,.2f}")

    fig = px.imshow(
        df,
        labels=dict(x=parameter_2, y=parameter_1, color=metric),
        aspect="auto",
        title=metric,
        color_continuous_scale=color_continuous_scale,
    )

    fig.update_traces(text=text, texttemplate="%{text}")

    fig.update_layout(
        title={"text": metric},
        height=600,
    )
    return fig