Get initial bounding box of a scatter mapbox

After failing with the Holoviews-Datashader-Dash angle for my “display a tiff on a map with tooltip”, I started to implement it without Holoviews. To do that, I need to have the bounding box of the map, and from there I can hopefully create the appropriate image and serve it.

For this, theoretically I should use the RelyoutData. There is even a showcase, see first example here: https://dash.plotly.com/interactive-graphing. I’ve tried, and it works similarly with scatter mapbox too, so that’s great. But the problem, that I only get the detailed information when the user pans the map. In that case I get something like this:

{
  "mapbox.center": {
    "lon": 35.97214172702286,
    "lat": -13.525062504454326
  },
  "mapbox.zoom": 5,
  "mapbox.bearing": 0,
  "mapbox.pitch": 0,
  "mapbox._derived": {
    "coordinates": [
      [
        29.380344852023114,
        -6.388197039267411
      ],
      [
        42.563938602023484,
        -6.388197039267411
      ],
      [
        42.563938602023484,
        -20.454564833837253
      ],
      [
        29.380344852023114,
        -20.454564833837253
      ]
    ]
  }
}

But without any user interactions I just get this:

{
  "autosize": true
}

If the user doesn’t pan just zooms, I get this:

{
  "mapbox.zoom": 5.25
}

Now of course this is extremely useless as we are talking about a responsively sized map in a web application so of course I have no idea of the size of the map, so I have no clues about the initial bounding box.

So my questions:

  • Is there a way to get the same detailed values on page load as on user pan? (I’ve searched for it everywhere with no luck so far.)
  • Is there a way to trigger automatically a pan action on page load? I would be fine with a tiny pan, but I did not find a way to do it. (I’m not talking about changing the “center” of the Figure, as that changes the default position of the map too, and that’s exactly the problem that I don’t get detailed information if the map is on the default values.)
  • If all of this is impossible (would be weird, as why would it have this functionality if there is no way to use it), what would be the best way to calculate an assumed bounding box if we only know the zoom (which is hardcoded at first and got the update value if the user zoomed) and a maximum width/height of the mapbox in px? (I’m thinking that in that case I would be able to generate a bit bigger images than needed to be sure to cover the visible area.)

Thanks!

Is your plan to add/remove images to the map on the fly to simulate that you are viewing the complete dataset?

Yes. I have multiple datasets (currently in tiff format, but it can be Pandas DF, Dask DF, Parquet, whatever) containing gridded data, that I want to be able to display as RGB image overlay on a map and also want to give access to the data itself through a hover tooltip. Imagine a temperature map or population-density map where you want to see the values as color, but also want to have access to the actual values at specific points. They are pretty high resolution, so when zoomed out to country level, I need a low resolution downsample and when zoomed in, I need a higher resolution (based on the zoom level) cut of the visible area. Theoretically I should be able to do the cutting and sampling with Datashader, so at first I’m trying to get the map side to work.

This idea (I have a gridded dataset that I want to display on a map) seems so simple, yet, I’m fascinated by how difficult it proves to be in Dash. :smiley:

Well, it’s pretty easy to display the data directly in Dash. However, if you plot hundreds of megabytes of data, performance will be poor (if it renders at all); it is simply too much for the browser to handle. Hence you need some intermediate piece of software that slices the data and displays only the part that you need.

This kind of software already exists in terms of tile servers, which is what powers the maps that you see on the web every day. It takes some work to get right though, so i wouldn’t recommend diving into this part yourself (unless it is for learning). Instead, use some of the many free tile servers available. I think the easiest options would be,

  1. Use holoviews data shader. It seem to do more-or-less what you want

I haven’t used holoview, so i can’t comment on any specifics, but my guess would be that it is much easier to resolve your issues with holoviews than hacking you own tile loading functionality into Dash. Even if you succeed, I doubt that performance will be as you intended it.

  1. Use a map component (either the native Plotly component or Dash Leafet) to display a tile layer (displaying you data), which you serve from a local tile server such as Terracotta

I previously encountered the same issue that you did (before holoviews was a thing), and i solved it using Dash Leaflet + a custom tile server (I used Terracotta, but pick whatever your heart desires). The only thing about the solution that i used that seems to be not-good-enough for your use case is the speed of the hover tooltip update.

You could solve this by doing a reverse color map lookup client side, which would yield instant tooltip updates. Here is an example using this approach,

https://johngravois.com/lerc-leaflet/

I didn’t choose this solution when i did my implementation for three main reasons,

  • It doesn’t allow index-lookup. Say that you display temperature on the map, but you would also like to show wind speed (stored in a different geotiff but with the same grid) in the tooltip (I needed that). With my current implementation you can do that (since you get the index in the raster), but with the client side reverse color map lookup, you can’t (since you don’t get the index, just the value)
  • Accuracy of displayed values is limited to the accuracy of the colormap
  • It only works if there is a 1:1 mapping between color and values, which puts some restrictions on color scale (less important)

Yeah, the last thing I would like to do is to reinvent some enterprise level functionality people usually write phd of.

But the thing is, as I linked my previous post at the beginning of this post, that I started with the Holoviews approach and could immediately do what I wanted in the Bokeh backend, but as soon as I switched to the Plotly backend, it swore to me that it is impossible to overlay an Image to a Mapbox in Plotly. “Image” here meaning “gridded raster data with visual reprezentation”, not just “visual image”. (The beauty of Holoviews is that you do the coding in HV and just by picking a backend, it generates the same plot in Bokeh, Plotly, matplotlib etc if possible.) To be honest, I’m very happy I found this HV+Bokeh solution, at least I know I’m not mad, this whole “displaying raster image as visual image AND data on a map” concept really should be easy to implement. I’ve made a small notebook, you can check it out, in less then 40 line it works, displays the image on a mapbox map and makes the data available through hover tooltip:
https://colab.research.google.com/drive/1x5pqiAfblwaqIiJ6xJVRBAHfVV_rRCN8?usp=sharing

(This doesn’t do the automatic resizing and clipping with Datashader, but that part is pretty well documented, so I’m sure it would be easy to update with this.)

And after days of research, I start to believe that Plotly itself is missing this functionality. By now I tried every figure type I could think of (px.imshow, go.image, go.heatmap etc) just to be able to overlay meaningful raster data with visualization to a scatter mapbox, and I could make any of them work. It’s very easy to combine traces in plotly, but it seems that the mapbox traces do not play together with any of the non-mapbox type figures. Even when I could force them together, the mapbox always fully covered the non-mapbox figure, and the two of them were obviously using two different containers with two different axis-set, with no options to merge them together. By now I think I’ve seen every plotly mapbox and raster example available on the net, and zero of them contained both types in the same figure (like my Bokeh example), so I think it is impossible.

So my question above is irrelevant and obsolete. Even if I could do the slicing in Datashader based on some bounding box information, there is no way to display a raster image with its data on a mapbox map and access its data in the same time. Which is surprising as this is such a basic functionality, no wonder it didn’t cause any problem for Bokeh, but it is what it is. So right now I’m pretty stuck. I have to put this functionality into an existing Dash app, but it is impossible. There is a plotly/dash-alternative-viz proof-of-concept module showcasing how you can supposedly embed Bokeh figures in Dash, but even their example script is not willing to do that and it just simply doesn’t show the Bokeh figure. Argh.

And thanks for your ideas, but as you mentioned it, the main problem with the tileset approach, that you at first convert all the data to a “visual-representation-only” image (as a tileset) and serve that to the client, so the client has to dark-magic the RGB values back to the original data values on the fly and yeah, that’s an interesting fringe concept, but definitely not something I would build a whole app around. So if I would go with the tileset approach, I would have to accept the turnaround time for tooltip-on-click functionality. Which is pretty good in itself and as I already mentioned, I’m pretty impressed with the whole terracotta-dash-leaflet project. It’s just that after seeing the ideal solution to work seamlessly in Bokeh, it would be pretty hard to abandon that goal.

I really loved Dash when I was just demoing and experimenting, it seems to be such a cool concept and implementation. But since I started to actually implement a specific task in it, time after time I ran into a ton of official examples that flat out didn’t work; a lot of features that should have worked but the last information about them were some bug reports on Github from 2018; and simply missing basic functionalities like this one. At this point I’m actually considering abandoning Dash and start my project from scratch in Bokeh, which pains me as I already poured a lot of effort into building an app with some pretty advanced functionalities. But this seems like a dead end, and even if I would find a workaround, if there is a week delay to find the workaround for every half-day task (which is kinda my experience so far), that’s not a workflow I’d like to live my life in.

That’s really interesting. In my team we are currently doing the same switch - just the other way around, from Bokeh to Dash. Can you give some examples (besides the holoviews case, which sounds like a bug) of where the Bokeh setup was a lot simpler?

Hehh, that’s funny, but I guess depending on the use case, different systems could work better.

I don’t know Bokeh at all (back when I tried to pick one system to start with I was considering it, but I decided for Dash), so I just started to read their docs to see how that works. Tbh, at first glance I really like it, feels much less opionated than Dash with more accessible hooks to the JS side of things (I like being able to directly tweak the frontend, so that’s a plus for me). But that’s really just at first glance, we’ll see what happens when I start to recreate the same app. What were you negative experiences that made you move from Bokeh to Dash?

It definitely depends on the use case, and probably also a lot on personal preferences. I am by no means a Bokeh expert. I only touched it briefly during the demo phase where we were deciding on the framework to use going forward. I think the main things that I liked were,

  • The Dash syntax being more expressive, i.e. it is easier for new people to understand what is going on
  • It seemed to me (at least at the time) that it was easy to build some things in Bokeh, but if your use case didn’t “fit the template”, it required more work to adopt (as compared to Dash)
  • The Dash server is stateless, which makes it less complicated to scale applications
  • With the Dash frontend being React, it is relatively simple to port React components to Dash. That opens up a huge ecosystem that you can access at low cost. I am not sure how easy it is to create custom components in Bokeh

That being said, there are some thing that are much harder to do in Dash than they should be. But Dash is still pretty young, and as many issues have been solved while I have been working with Dash, I am optimistic about future developments.

1 Like