Plotly & Flask Heatmap Not showing on localhost

Hello all, first post here!

Iā€™m attempting to make a webapp for temperature data. one particular part is to view a temperature probeā€™s temperature by months of all years.

in Jupyter, this snippet works wonderfully:

import plotly.figure_factory as ff

z=np.round(data_test_group_pivot.values[::-1],2)
x=list(data_test_group_pivot.columns)
y=list(data_test_group_pivot.index[::-1])

layout_two = dict(title=ā€˜128 TMR_SUB_18ā€™,
xaxis=dict(title=ā€˜Monthsā€™),
yaxis=dict(title=ā€˜Yearsā€™)
)

fig = go.Figure(data=ff.create_annotated_heatmap(z, x=x, y=y, font_colors=[ā€˜antiquewhiteā€™], hoverinfo=ā€˜y+x+zā€™),
layout=layout_two)

fig.update_layout({ā€œtitleā€: {ā€œtextā€: ā€œ128 Temperature Data: Months over Yearsā€}})

fig.show()

however, when i try to incorporate it into my flask app:

hereā€™s the snippet code, which also works in Jupyter Notebook:

best result in Jupyter

graph_two = []

import plotly.figure_factory as ff

z=np.round(df_all_rules_group_pivot.values[::-1],2)

x=list(df_all_rules_group_pivot.columns)

y=list(df_all_rules_group_pivot.index[::-1])

# fig = ff.create_annotated_heatmap(z, x=x, y=y, annotation_text=z, colorscale='Viridis')

layout_two = dict(title='128 TMR_SUB_18',

                  xaxis=dict(title='Months'),

                  yaxis=dict(title='Years')

                  )

fig_heatmap = go.Figure(data=ff.create_annotated_heatmap(z, 

x=x, 

y=y, 

font_colors=['antiquewhite'], 

hoverinfo='y+x+z'), 

           layout=layout_two)

graph_two.append(fig_heatmap)

figures = []

figures.append(dict(data=graph_two, layout=layout_two))

Figures is then returned which is handled in my routes.py:

from myapp import app

import json, plotly

from flask import render_template

from wrangling_scripts.wrangle_data import return_figures

@app.route('/')

@app.route('/index')

def index():

figures = return_figures()

# plot ids for the html id tag

ids = ['figure-{}'.format(i) for i, _ in enumerate(figures)]

# Convert the plotly figures to JSON for javascript in html template

figuresJSON = json.dumps(figures, cls=plotly.utils.PlotlyJSONEncoder)

return render_template('index.html',

                       ids=ids,

                       figuresJSON=figuresJSON)```

and index.html file has this:

<script type="text/javascript">
    // plots the figure with id
    // id much match the div id above in the html
    var figures = {{figuresJSON | safe}};
    var ids = {{ids | safe}};
    for(var i in figures) {
        Plotly.plot(ids[i],
            figures[i].data,
            figures[i].layout || {});
    }
</script>

iā€™ve used line charts, bar charts, scatters, and tables, but this is the first thing from plotly, iā€™ve had trouble with.

any ideas?

Hey!

Can you share the whole index.html file?

Have you looked at the .js reference pages?

or

hey Dom, sure can!

  • itā€™s from a boilerplate that i made a while back, so thereā€™s extra rows commented out currently. Please ignore (and excuse my mess :grin:)
  • i havenā€™t checked out the .js references much, especially in combining the python based plots with straight .js but that is a very compelling thought.

<html>

<head>

<title>Alaska DOT&PF Temperature Data Dashboard</title>

<!--import script files needed from plotly and bootstrap-->
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/css/bootstrap.min.css" integrity="sha384-Gn5384xqQ1aoWXA+058RXPxPg6fy4IWvTNh0E263XmFcJlSAwiGgFAW/dAiS6JXm" crossorigin="anonymous">
<script src="https://code.jquery.com/jquery-3.3.1.min.js" integrity="sha384-tsQFqpEReu7ZLhBV2VZlAu7zcOV+rXbYlF2cqB8txI/8aZajjp4Bqd+V6D5IgvKT" crossorigin="anonymous"></script> 
<script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.12.9/umd/popper.min.js" integrity="sha384-ApNbgh9B+Y1QKtv3Rn7W3mgPxhU9K/ScQsAP7hUibX39j7fakFPskvXusvfa0b4Q" crossorigin="anonymous"></script>
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/js/bootstrap.min.js" integrity="sha384-JZR6Spejh4U02d8jOt6vLEHfe/JQGiRRSQQxSfFWpi1MquVdAyjUar5+76PVCmYl" crossorigin="anonymous"></script>
<script src="https://cdn.plot.ly/plotly-latest.min.js"></script>

</head>

<body>

<!--navbar links-->     
<nav class="navbar navbar-expand-lg navbar-dark bg-dark sticky-top">
   <a class="navbar-brand" href="#">Data Dashboard</a>
  <button class="navbar-toggler" type="button" data-toggle="collapse" 
  data-target="#navbarTogglerDemo02" 
  aria-controls="navbarTogglerDemo02" aria-expanded="false" 
  aria-label="Toggle navigation">
    <span class="navbar-toggler-icon"></span>
  </button>

  <div class="collapse navbar-collapse" id="navbarTogglerDemo02">
    <ul class="navbar-nav ml-auto mt-2 mt-lg-0">
      <li class="nav-item">
        <a class="nav-link" href="https://alaskatrafficdata.drakewell.com/publicmultinodemap.asp" target=ā€_blankā€>C2 Alaska Public Site</a>
      </li>
      <li class="nav-item">
        <a class="nav-link" href="https://arcg.is/1mD5qu1" target=ā€_blankā€>AKDOT Traffic Links WebApp</a>
      </li>
    </ul>
  </div>
</nav>

<!--middle section-->       
<div class="row">

    <!--social media buttons column-->      
    <div class="col-1 border-right">
        <div id="follow-me" class="mt-3">
            <a href="http://dot.alaska.gov/stwdplng/transdata/tempdataprobes.shtml" target="_blank">
                <img src="/static/img/dot-seal-120.png" alt="akdot_seal" class="img-fluid mb-4 ml-2">
            </a>
            <a href="https://github.com/erikjamesmason/Temperature_Data_Probe_2" target="_blank">
                <img src="/static/img/githublogo.png" alt="github" class="img-fluid ml-2">
            </a>
        </div>
    </div>

    <!--visualizations column-->        
    <div class="col-11">

        <!--chart descriptions-->       
        <div id="middle-info" class="mt-3">

            <h2 id="tag-line">Alaska DOT&PF Temperature Data Dashboard</h2>
            <h4 id="tag-line2" class="text-muted">Sample Temperature Data</h4>
            
        </div>
        
        <!--charts-->       
        <div id="charts" class="container mt-3 text-center">
                    
            <!--top two charts-->       
            <div class="row">
                <div class="col-11">
                    <div id="{{ids[0]}}"></div>
                </div>
               
            </div>

            <!--bottom two charts-->        
           <div class="row mb-6">
               <div class="col-11">
                   <div id="{{ids[1]}}"></div>
               </div>
<!--                <div class="col-6"> -->
<!--                    <div id="chart3">-->
<!--                        <div id="{{ids[2]}}"></div>-->
<!--                    </div>-->
<!--                </div>-->
<!--                <div class="col-6">-->
<!--                    <div id="chart4">-->
<!--                        <div id="{{ids[3]}}"></div>-->
<!--                    </div>-->
<!--                <div>-->
           </div>
        
        </div>
    </div>
</div>

<!--footer section-->               
<div id="footer" class="container"></div>

</body>


<footer>

    <script type="text/javascript">
        // plots the figure with id
        // id much match the div id above in the html
        var figures = {{figuresJSON | safe}};
        var ids = {{ids | safe}};
        for(var i in figures) {
            Plotly.plot(ids[i],
                figures[i].data,
                figures[i].layout || {});
        }
    </script>
    
</footer>


</html>

okay, I seeā€¦

do you have the raw data being inserted into this template by flask for the figures variable? and for the ids variable below?

Why is there a || operator in the third argument of the .plot method?

2021-02-22_08-50-30

yes, the raw data is processed into the figures variables in the wrangling_scripts.wrangle_data.py return_figures() function. This is one of final phases in a much longer pipeline from ā€œrawā€ data to a more usable format. The data in this project is held in a Pickle file.

Granted, thereā€™s no guaranteed method to my madness.

i believe my intentioned process flow was to:

  • take the raw data, use pandas DataFrames to reshape the data according to desired visualization (i.e, z=np.round(df_all_rules_group_pivot.values[::-1],2))

  • then append that graph object to graph_one/graph_two

    • fig_heatmap = go.Figure(data=ff.create_annotated_heatmap(z,x=x, y=y, font_colors=['antiquewhite'], hoverinfo='y+x+z'), layout=layout_two)
    • graph_two.append(fig_heatmap)
  • which then gets appended to figures = []

    • figures.append(dict(data=graph_two, layout=layout_two))

from there you can seehow the routes.py handles it, as far as using the same function (return_figures()) then building up the json.dumps along with the id.

i believe the id is just used for conventionā€™s sake of being able to assign an id to the div.
ids = ['figure-{}'.format(i) for i, _ in enumerate(figures)] - itā€™s being done in a naive-fashion.

as for the last note,

Why is there a || operator in the third argument of the .plot method?

I canā€™t recall a specific reasonā€¦

my best guess - oftentimes, the layout is described in the go.Figure and maybe isnā€™t specified in its own variable (like in the example figures.append(dict(data=graph_two, layout=layout_two)))
So if the plot method went to find the figures[i].layout but it didnā€™t exist, it would look for an empty object/variable and accept that as a faux-predefined layout?

like i said, no guaranteed method to my madness. i will tear this whole project down and rebuild it if i must. :wink:

Well your problem is very likely in the layout and the way that the plotly.JS library handles layout.

Iā€™m looking at this purely from the .js side.

Can you share the raw json data for the layout variable?

And the raw json data for the xyz?

I would also remove that || operator from the .plot method layout parameter, that seems out of place.

Canā€™t say that i know the best way to get the raw json for those individual variables from the python scriptā€¦ the best i could come up with is the figureJSON as raw json - if preferable, i can strip it out to those individual sections manually - iā€™m sending it in a direct message to save space on the thread

scratch that, json is too long. look in your email.

First the output data you provide had backslashes before quotes, probably an escape character of some kind but I donā€™t think javascript likes that.

So I did a find and replace and removed all those back slashes and then removed the quotes at the beginning and end of the json string.

Then copied and pasted that into online json editor for data structure analysisā€¦

looking at the data its basically a two part array that does look broken and poorly structured to feed plotlyā€¦ but the main parts are thereā€¦

so tweaked your html a little, by manually assigning the figures variable and ids variable and changing the ids for two of the divs, I then hit save and one chart rendered.

So youā€™ve got the pieces and youā€™re close. You may have to study the plotly.js library references and look at how your program is passing the json data outā€¦ you might have to tweak some things. The .js plotly library is useful but itā€™s not necessarily better than the python library. I use .js cause thats what works best for what i need to accomplish. The best tool is the tool that gets it done. Iā€™ll send you a pm. Cheers.

Thanks very much for the time, effort, and assistance. It is very much appreciated. I picked up on what you were getting at with the .json and went and produced a bunch of graph objects and the likes and json.dumps() the figs to json to inspect them and you were right, lots of little issues.

ultimately, if anyone else is attempting to use figure_factory with Flask in the same manner that i was (appending multiple graphs to get rendered in a loop), theyā€™ll be pretty challenged and maybe end with disappointment . I was able to successfully create a go.heatmap (after inspecting the .json as you suggested and revising it a bit), but the figure_factory annotated_heatmap could not work at all. I suspect that it is possible, but the way it annotates the heatmap and is outputted in .json is structured in a way that it needs a lot of tailoring.

Iā€™ll try to show you the final result in a bit after i clean up the scripts from my messy dev explorations.

The X axis on the Seaborn side is multiplied by 100, but the plotly side is not multiplied by 100?