I am using Dash-Leaflet to filter a large geojson via a callback. Currently, the callback calls a function that filters a dataframe which in turn creates a geojson and then returns the geojson to the dl.GeoJSON constructor. This is extremely slow to say the least. How can I improve this process? I am filtering the dataframe on date range(start date and end date) as well as a categorical list. Can I just write the geojson to file and filter it directly? This will avoid using json.loads which is extremely slow.
How would I filter geojson directly?
window.myNamespace = Object.assign({}, window.myNamespace, {
mySubNamespace: {
filterFeatures: function(feature, startDate, endDate, category) {
# there is one field in the geojson for date and one for category, but the category argument would hold a list of three values
}
}
});
There are a few things you can do to improve the efficiency,
Save the GeoJSON data to a (static) hosted file (e.g. inside the assets folder) so that you can pass it via the url property
Strip out any information that you don’t need (e.g. unused columns of data, etc.)
Do the filtering client side rather than server side
Point 1 & 2 are general recommendations that will reduce the amount of data to be transferred the first time (+ enable browser caching), i.e. they will improve the speed of the initial load of the map.
Point 3 is the most important for your use case as it will eliminate the need to transfer data multiple times, i.e. after the map has loaded the first time data will not be transferred again. Hence subsequent changes in the filter will be fast.
Thanks Emil! I will look into savings the GeoJSON as a static file and constructing a client-side callback. I might have more questions regarding callback construction but I will do some research on how to do that.
I have been thinking about making it easier to use small js snippets inside a Dash app for some time. Your use case fits perfectly with the idea, so i decided to try it out. You can get my initial implementation from pip,
pip install dash-extensions==0.0.54rc1
The main new feature is the assign function that let’s you write javascript code directly in the Python script. Behind the scenes, the function is then written to a .js function in the assets directory. Here is a small demo,
import os
import urllib.request
import dash_html_components as html
import dash_leaflet as dl
import dash_core_components as dcc
import json
from dash import Dash
from dash.dependencies import Output, Input
from dash_extensions.javascript import assign
us_states_path = "assets/us-states.json"
# Download asset if it doesn't exists.
os.makedirs("assets", exist_ok=True)
if not os.path.isfile(us_states_path):
urllib.request.urlretrieve("http://dash-leaflet.herokuapp.com/assets/us-states.json", us_states_path)
# Load us states.
with open(us_states_path) as f:
states = [f["properties"]["name"] for f in json.load(f)["features"]]
# Create javascript function that filters on feature name.
geojson_filter = assign("function(feature, context){return context.props.hideout == feature.properties.name;}")
# Create example app.
app = Dash()
app.layout = html.Div([
dl.Map(children=[dl.TileLayer(), dl.GeoJSON(url=f"/{us_states_path}", options=dict(filter=geojson_filter),
hideout="Texas", id="geojson", zoomToBounds=True)],
style={'width': '100%', 'height': '50vh', 'margin': "auto", "display": "block"}, id="map"),
dcc.Dropdown(id="dd", value="Texas", options=[dict(value=s, label=s) for s in states], clearable=False)
])
# Link drop down to geojson hideout prop.
app.clientside_callback("function(x){return x;}", Output("geojson", "hideout"), Input("dd", "value"))
if __name__ == '__main__':
app.run_server()
I am having issues downloading the newest pip install of the library. But this is basically what I need to do below, but I am unsure on how to filter the geojson directly when it is passed directly to url.
def geojsonFilter(startDate,endDate,status):
# how do I filter the geojson passed from assets--- iterate and return a geojson then construct it via clientside?
return ((feature['properties']['Date'] >= str(startDate)) and (feature['properties']['Date'] <= str(endDate))) and (feature['properties']['status'] in category)
html.Div([
dl.Map(zoom=3, children=[
dl.TileLayer(),
dl.GeoJSON(url ="assets/test.geojson", id="map1", options=dict(style=ns('style'))),
], style={'width': '100%', 'height': '60vh', 'margin': "auto", "display": "block"},id = 'map'),
], className = "nine columns fix_charts bg_color"),
],className = "row")
#callback contains a date range and a checklist which are categories
@app.callback(Output('map1','data'),
[Input('checklist_items,'value'),
Input('date_picker_range','start_date'),
Input('date_picker_range','end_date')])
def update(startDate,endDate,status):
data = geojsonFilter(startDate,endDate,status)
return data
