Show and Tell - Slapdash: Boilerplate for bootstrapping scalable multi-page Dash apps

Hi everyone!

Slapdash is a project that provides a boilerplate project structure for bootstrapping scalable multi-page Dash applications. The idea is that you can clone/copy the project structure and then use that to bootstrap construction of your own app, without having to spend time on the initial scaffolding, which will often be the same across projects.

I have tried my best to use Dash and Flask best-practice and to make it easy to take advantage of some of the various neat affordances Dash provides, but itā€™s very much an ongoing project. Iā€™m sure Iā€™ll come across many ways in which the boilerplate can be improved as I use it to build out more apps.

Iā€™ve been working on this on-and-off for a while, but I feel like itā€™s now in pretty good state to be ā€œannouncingā€. Would love to get feedback and pull requests!

6 Likes

Hi nedned,

This looks very neat indeed! I like the idea of building such a boilerplate.

I was trying to run the app though and the following occurred:

python run-flask.py --debug
Traceback (most recent call last):
  File "run-flask.py", line 5, in <module>
    from slapdash.app import app
  File "/home/VICOMTECH/jbruse/Documents/Slapdash/slapdash-master/src/slapdash/__init__.py", line 5
    def create_flask(config_object=f'{__package__}.settings'):
                                                           ^
SyntaxError: invalid syntax

there were some other prepended "f"s in component.py (line 50) and utils.py (lines 8, 28). Not entirely sure whatā€™s going on there.

Thanks!
Jan

Iā€™m guessing youā€™re running Python 2? Slapdash requires 3.6+. Iā€™ve just now added a note making this explicit. Sorry!

Those are f-strings, string literals that allow you to format strings using the same interface as the str.format() method. They were added in Python 3.6.

You can safely replace all f-strings with the equivalent str.format(). I think thatā€™s probably the only Python 3 specific thing. I donā€™t plan to add Python 2 support as itā€™s end of life at the end of this year.

Hi nedned,

Many thanks for your quick reply! In fact, I was using Python 3.5.2 - so all my fault. Sorry, didnā€™t know about those f-strings, interesting :slight_smile:
Thanks!

Hi all, just an update with a couple of new features Slapdash now offers:

  • run-slapdash-dev a Click-based command line script for running your app in dev environments
  • run-slapdash-prod a bash script for running your app in production environments using mod_wsgi-express, which is tuned to run efficiently by performing compression with Apache not within the Python process and by serving the contents of the assets directly rather than having them go through the Python process (which is waaay slower)
  • a new DashRouter class for managing the URLs/pages of your app, which support passing GET style query parameters into callables that return your page layouts.
3 Likes

Hi nedned,

This is amazing!

One question: How would I add flask_login to this framework?

Edit: Iā€™ve been thinking about this and hereā€™s what Iā€™ve come up with so far. I havenā€™t implemented anything yet so this approach could be wrong.

  • Initialize the SQL Alchemy user DB and Flask_Login in the create_flask() function
  • Import a User model into the create_flask() function
  • Pass dash_url_pathname = '/<insertnamehere>/' to the dash instance
  • Add a @server.route('/login') to handle user logins. This callback should be put into app.py file after the app = create_dash(server) call
  • Add the @login_required to the server.route which routes to the dash app (ā€™/<insertnamehere/ā€™)

Would this work?

Thanks @mdylan2!

I havenā€™t actually used Flask login before, so I canā€™t give suggestions based on my direct experience. I think you are on the right track though. Hereā€™s a couple of suggestions:

  • You could either add the extensions to the Flask instance within create_flask, or on the resultant Flask instance returned from that function in app.py. This is just personal preference.
  • there is a slight preference towards using the more explicit routes_pathname_prefix than url_base_pathname (which I assume is what you meant by dash_url_pathname)
  • youā€™re right about needing to add the login_required decorator to the various Dash routes. This I think is the tricky bit as in addition to protecting the index you also need to protect the API endpoints. From memory, a few people posted their approach to doing that in this issue: https://github.com/plotly/dash/issues/214

Thanks for the help @nedned!

Iā€™ve included my code below in case anyone needs to implement a login system for nedā€™s Slapdash framework. This method does not include any information on editing, deleting, or updating users in the database - it simply reflects existing users in the user table of the DB and checks their passwords (assuming the passwords are hashed using bcrypt).

This method protects all dash views as well so people canā€™t access the dash pages without logging in. To log out, simply access the logout URL (i.e. localhost/logout).

models.py (defined in the slapdash folder)

from slapdash import db
import bcrypt
from flask_login import UserMixin
from slapdash import login

class User(UserMixin, db.Model):
    __table__ = db.Model.metadata.tables['user']

    def __repr__(self):
        return '<User {}>'.format(self.username)

    def check_password(self, password):
        return bcrypt.checkpw(password.encode('utf8'), self.password.encode('utf8'))

@login.user_loader
def load_user(id):
    return User.query.get(int(id))       

init.py

...
from flask_sqlalchemy import SQLAlchemy
from flask_login import LoginManager

db = SQLAlchemy()
login = LoginManager()
login.login_view = 'login'
bootstrap = Bootstrap()

def create_flask
....

    server.config.from_envvar("SLAPDASH_SETTINGS", silent=True)

    with server.app_context():
        db.init_app(server)
        db.Model.metadata.reflect(db.engine) #This part is where I reflect all the tables from my database
        from slapdash import models
        
    login.init_app(server)
    bootstrap.init_app(server)
    return server

settings.py

ā€¦
SQLALCHEMY_DATABASE_URI =
SQLALCHEMY_TRACK_MODIFICATIONS = False
SECRET_KEY =

login.html (in slapdash/templates)

{% import 'bootstrap/wtf.html' as wtf %}
<html>
<body>
    <div class="login-page">
        <div class = 'row'>
            <div class ='col-3 offset-1 login-holder'>
                    <div class="jumbotron logon container">
                            {{ wtf.quick_form(form, button_map = {'submit': 'danger btn-lg login'}) }}
                        </div>
            </div>
        </div>
    </div>
</body>
</html>

app.py

from . import create_flask, create_dash
from .layouts import main_layout_header, main_layout_sidebar
from flask import redirect, render_template, url_for, redirect, request, flash
from flask_login import current_user, login_user, logout_user, login_required
from werkzeug.urls import url_parse
from .auth.form import LoginForm

def protect_views(app):
    for view_func in app.server.view_functions:
        if view_func.startswith('/'):
            app.server.view_functions[view_func] = login_required(app.server.view_functions[view_func])
    return app

# The Flask instance
server = create_flask()

# The Dash instance
app = create_dash(server)

app = protect_views(app)

from .models import User

@server.route('/')
@server.route('/login', methods = ['GET','POST'])
def login():
    if current_user.is_authenticated:
        return redirect('/')
    form = LoginForm()
    if form.validate_on_submit():
        user = User.query.filter_by(username = form.username.data).first()
        if user is None or not user.check_password(form.password.data):
            flash('Invalid username or password')
            return redirect(url_for('login'))
        login_user(user, remember=form.remember_me.data)
        next_page = request.args.get('next')
        if not next_page or url_parse(next_page).netloc != '':
            next_page = '/'
        return redirect(next_page) 
    return render_template('login.html', title='Sign In', form=form)



@server.route('/logout')
@login_required
def logout():
    logout_user()
    return redirect(url_for('login'))
...

The method could definitely be cleaned up but you guys can probably figure that out on your own.

Please let me know if you have issues!

1 Like

Hi @nedned,

How would we update parameters in the URL? Would we need to write a callback that updates the search attribute of the location_component_id?

If so, wouldnā€™t we only be able to assign one callback to update the URLā€™s search parameter?

Or does the slapdash framework allow me to add unique dcc.Location components to each page and still preserve DashRouter's routing?

Edit:
I think it may be easier for you to help if I described what I was doing.

Hereā€™s the flow I had:

  • Select data from a few dropdowns, click a ā€œLets Goā€ button after selecting that data
  • The letā€™s go button changes the search parameter of the URL. The search parameters update the layout
  • After the search parameter updates, this runs a callback that queries my database (using the parameters) and processes the data. This processed data is stored in a local memory store.
  • All the graph elements display data from the store and update according to other filters

The only issue I had with this method is that the search parameter for the serverā€™s dcc.Location component can only have one callback associated with it. Hence, I could not have the same callback flow for all pages. How can I work around this?

As far as I know, Dash requires a single Location component. So I think you do want to use a single URL callback that is triggered by changes to the pathname or search props ā€“ just as the DashRouter uses. URL search params can then be fed into your different page layout functions.

But then we bump into a current limitation of Dash, which is that it does not support synced/bidirectional component updating. Currently, if you were to update the search component of the URL through a callback in order to keep it synced with the UI, it would then trigger the Location callback, when it should not. This limitation is described in this issue.

I have seen some workarounds for this specific context presented in this other issue, but havenā€™t tried them myself.

I havenā€™t heard of any work on progress to address this limitation. @chriddyp, do you know if this is something on the roadmap?

Iā€™ve continued working on this and hereā€™s a workaround for the above issue. The only problem with my workaround is that it is limited by the Store component.

Hereā€™s the idea:

  • I update the href of a button based on the value of the dcc.Dropdown, ā€œinputā€. This works around the limitation of updating the URL search attribute of the global dcc.Location component (CALLBACK 1)
  • Clicking the button tacks on value1 to the path
  • The html.Div, ā€œdivā€, should read the search parameter from the URL and update accordingly (CALLBACK 2)
  • The dcc.Store, ā€œstoreā€, should then update after the div, and store the [1,2,3] array (CALLBACK 3)

The issue with this is that my div doesnā€™t update when I click the letsgobutton. After clicking the button, the URL changes and the page refreshes (due to the Slapdash DashRouter). Then, CALLBACK 3 gets fired, printing None; I want CALLBACK 2 to get fired first.

However, if I remove the dcc.Store callback (CALLBACK 3), this method works. Clicking the button fires CALLBACK 2, which can then be used as a hidden div. However, Iā€™d prefer avoiding the hidden div solution. Do you know why this is happening?

def layout(**kwargs):
    value1 = int(kwargs.get("value1", 0))
    return html.Div([
           html.Div(id = "div"),
           dcc.Store(id ="store"),
           dbc.Button(id = "letsgobutton"),
           dcc.Dropdown(id = "input"),
           html.Div(id = "layout_from_kwarg", children = [value1])
           ])
# CALLBACK 1
@app.callback(
    Output("letsgobutton","href"),
    [Input("input","value")]
)
def update_button_link(input1):
    if input1 is None:
        raise PreventUpdate
    code = urllib.parse.urlencode(*args go here*)
    return '?' + code

# CALLBACK 2
@app.callback(
    Output("div","children"),
    [Input(app.server.config["LOCATION_COMPONENT_ID"],"search")]
)
def function(path):
    kwargs = get_search_params(path)
    value1 = int(kwargs.get("value1", 0))
    return value1

# CALLBACK 3
@app.callback(
    Output("store","data"),
    [Input("div","children")]
)
def function2(child):
    print(child)
    return '[1,2,3]'

@mdylan2, havenā€™t looked into your issue yet, but will try and have a ponder soon!

For anyone following along this thread, Slapdash is now a Cookiecutter template. This means that after answering a couple of questions, itā€™s possible to create your own customised project based off of Slapdash with project metadata (such as project name, author name) already filled in as needed across the codebase. This makes spinning up a new project much quicker, as you donā€™t have to around manually changing things across the codebase.

2 Likes

@nedned - been building a new app using Slapdash and loving it so far. A couple challenges Iā€™m running into, and wanted to see if you have seen ways to solve it:

  1. I have some graphs that I want to use selectedData to jump me to a new page with a query string ?var1=blah&var2=blah. I have the selectedData triggering a callback to update dcc.Location, which is working, but it causes a total refresh of the page versus a seemless transition of layouts like your DashRouter handles. Is there a better way to jump to a new page that doesnā€™t involve using an href?

  2. If I wanted to dump a data store and reference it from all the pages, where would I put that dcc.Store in your templates? Would I put that somewhere in your make navbar layout method that has access to the other pages?

Thanks!
Scott

Hey @Scrawford, thatā€™s great to hear!

For the first issue, do you have refresh=False in your Location component?

Regarding placement of the dcc.Store component, thereā€™s a range of things you could do, but my intuition would be to start along the lines as you describe, perhaps by putting it in the top level layout-making function that youā€™re using, from layouts.py.

Second issue is solved exactly as you mentioned, all of my data stores went on the same page/context in index.py.

As for the first issue, if I set refresh to False the url changes but now the app wonā€™t go to that page. Iā€™m not sure how to have a selectedData callback fire like a NavLink does. Do I somehow need to tie into the DashRouter somehow?

Edit: I guess my real question is, how do you navigate between pages, with search params, in the app without using a navlink button?

@nedned I know this is a dumb question but I am unable to get my app to run using the {{cookiecutter.project_slug}}. Every time I go to run it I am getting the following error. Could you let me know what I am missing?

(base) C:\Users\Derek.sweet\Desktop\asm_util\slapdash_test\dash>run-dash-dev
Traceback (most recent call last):
File ā€œC:\ProgramData\Anaconda4\Scripts\run-dash-dev-script.pyā€, line 11, in
load_entry_point(ā€˜slapdash-testā€™, ā€˜console_scriptsā€™, ā€˜run-dash-devā€™)()
File ā€œC:\ProgramData\Anaconda4\lib\site-packages\pkg_resources_init_.pyā€, line 489, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File ā€œC:\ProgramData\Anaconda4\lib\site-packages\pkg_resources_init_.pyā€, line 2852, in load_entry_point
return ep.load()
File ā€œC:\ProgramData\Anaconda4\lib\site-packages\pkg_resources_init_.pyā€, line 2443, in load
return self.resolve()
File ā€œC:\ProgramData\Anaconda4\lib\site-packages\pkg_resources_init_.pyā€, line 2449, in resolve
module = import(self.module_name, fromlist=[ā€˜nameā€™], level=0)
ModuleNotFoundError: No module named ā€˜dash.dev_cliā€™

What is the best way to update the dataframe used to produce a graph using this boiler plate?

I want the dataframe to update everyhour or so. I have tried the following in app.py:

from concurrent.futures import ThreadPoolExecutor
from . import create_flask, create_dash
from .layouts import main_layout_header, main_layout_sidebar
from .get_data import initial_df_creation, hourly_df_update


# The Flask instance
server = create_flask()

# The Dash instance
app = create_dash(server)

# Get dataset from DB
df_rules_performance, df_rules_overview, df_users_lookup = initial_df_creation()

# # Run the function to renew dataframes every 24 hours in another thread
executor = ThreadPoolExecutor(max_workers=1)
executor.submit(hourly_df_update)


# Push an application context so we can use Flask's 'current_app'
with server.app_context():

    # load the rest of our Dash app
    from . import index

    # configure the Dash instance's layout
    app.layout = main_layout_sidebar()

and import the dataframes in my layouts. Is there a better way to do this?