Datashader image on Mapbox offset

Hi All,

I am already working with plotly for about a year right now and I really love the package. But right now I have a issue for about a month already that I can’t solve so maybe one of you can help me. I am trying to make a heatmap with datashader and plot it on a map (Plotly and Datashader | Python | Plotly). For the most part this works but the image does not overlay right with the mapbox in my case. With the code I pasted bellow this mapbox is given:


This is not right and it’s weird because I have never had this kind of projection issues when just working with Lat Lon’s.

print("Opening .csv")
DF_COMBINED = dd.read_csv('output/' + '***/' + '***/' + 'AREA_DATA/' + 'TotalCSV_Data' + '.csv')
      
print("Opened .csv")
import datashader as ds
cvs = ds.Canvas(plot_width=4000, plot_height=4000)
# conversion to web mercator
from datashader.utils import lnglat_to_meters
# create new columns 'x' (longitude) and 'y' (latitude)
DF_COMBINED['x'], DF_COMBINED['y'] = DF_COMBINED['longitude_deg'], DF_COMBINED['latitude_deg']
# keep a clean dataframe
# DF_COMBINED = DF_COMBINED.drop(['LONGITUDE', 'LATITUDE'], axis=1)
agg = cvs.points(DF_COMBINED, x='x', y='y')
# agg is an xarray object, see http://xarray.pydata.org/en/stable/ for more details
coords_lat, coords_lon = agg.coords['y'].values, agg.coords['x'].values
# Corners of the image, which need to be passed to mapbox
coordinates = [[coords_lon[0], coords_lat[0]],
                       [coords_lon[-1], coords_lat[0]],
                       [coords_lon[-1], coords_lat[-1]],
                       [coords_lon[0], coords_lat[-1]]]
from colorcet import fire
import datashader.transfer_functions as tf
img = tf.shade(agg, cmap=fire)[::-1].to_pil()
from PIL import Image
j = Image.fromarray(np.asarray(img))
j.save("output/test.png")
import plotly.express as px
# Trick to create rapidly a figure with mapbox axes
fig = px.scatter_mapbox(DF_COMBINED.tail(1), lat='latitude_deg', lon='longitude_deg', zoom=1)
# Add the datashader image as a mapbox layer image
fig.update_layout(mapbox_style="carto-darkmatter",
                         mapbox_layers = [
                        {
                            "sourcetype": "image",
                            "source": img,
                            "coordinates": coordinates
                        }]
        )
fig.write_html('output/' + '***/' + '***/' + 'AREA_DATA/' + 'TestCustomHeatmap2' + '.html', auto_open=True)

I found some tips where I could convert the coordinates to a epsg:3857 format before passing them into datashader but this also does not work for me. (This converts the Lat Lon’s to meters) Personally I have always just used Lat Lon coordinates which never had a issue on the Mapbox.

I have also tried the following code which generates a lot of grids in a Dict and appends all the images to a Mapbox but there is still a (smaller) offset in the generated image. Also the total image is not looking good.

df = pd.DataFrame()
df['Lat'] = dfCSV['latitude_deg']
df['Lon'] = dfCSV['longitude_deg']
LastLonChecker = 0
LastLatChecker = 0
print('Done')
LatLonTileDict = {}
LatLonTilesCounter = 0
for LonCounter in range(-180, 180, 5):
    for LatCounter in range(-90, 90, 5):

        LatCounterPlus = LatCounter + 5
        LonCounterPlus = LonCounter + 5
        dff = df.query('Lat > @LatCounter').query('Lat < @LatCounterPlus').query('Lon > @LonCounter').query('Lon < @LonCounterPlus')
        # print(LatCounter, LatCounterPlus, LonCounter, LonCounterPlus)
        # dff = df.query('Lat > 0').query('Lat < 90').query('Lon > 0').query('Lon < 180')
        if len(dff['Lat']) > 0:
            # print(dff)
            cvs = ds.Canvas(plot_width=1000, plot_height=1000)
            agg = cvs.points(dff, x='Lon', y='Lat')
            img = tf.shade(agg, cmap=fire, how='log')[::-1].to_pil()
            # agg is an xarray object, see http://xarray.pydata.org/en/stable/ for more details
            coords_lat, coords_lon = agg.coords['Lat'].values, agg.coords['Lon'].values
            # Corners of the image, which need to be passed to mapbox
            coordinates = [[coords_lon[0], coords_lat[0]],
                           [coords_lon[-1], coords_lat[0]],
                           [coords_lon[-1], coords_lat[-1]],
                           [coords_lon[0], coords_lat[-1]]]

            LatLonTileDict.update({LatLonTilesCounter: {'Coordinates': coordinates, 'IMG': img}})
            from PIL import Image
            j = Image.fromarray(np.asarray(img))
            j.save("output/HeatmapImages/" + str(LatLonTilesCounter) + "test.png")
            LatLonTilesCounter += 1

print(LatLonTileDict)

# Trick to create rapidly a figure with mapbox axes
fig = px.scatter_mapbox(df[:1], lat='Lat', lon='Lon', zoom=12)

def MakeLayers():
    MapboxLayersList = []
    for key in LatLonTileDict.keys():
        MapboxLayersList.append({
                            "sourcetype": "image",
                            "source": LatLonTileDict[key]['IMG'],
                            "coordinates": LatLonTileDict[key]['Coordinates']
                        })
    return MapboxLayersList

print(MakeLayers())
# Add the datashader image as a mapbox layer image
fig.update_layout(mapbox_style="carto-darkmatter",
                  mapbox_layers = MakeLayers())


fig.write_html('output/HeatmapImages/' + 'TestCustom' + '.html')

Results in:

Does anybody have a idea how I can fix this?

Is it maybe possible to plot the image over a ScatterMap instead of a Mapbox to fix this?