Logging in Dash - LogTransform

Dash Python
,
21

Thanks! As you have already found out, messages are only propagated to the UI when the callback execution ends. That’s due to technical limitations (essentially because the logging mechanism builds on top of existing Dash callback architecture).

If you want to circumvent this limitation, you would need a different logging architecture altogether; one option would be to write logs somewhere (a file, a cache, …), and then pull the logs to the UI using an Interval component or a Websocket component. I guess it could be an option to automate this pattern in dash-extensions, but I haven’t done that yet.

1 Like
22

Thanks for your prompt answer, so I stop looking in that direction and will implement your suggestion.
I hope it will not be too heavy to keep a smooth execution.

23

Hi @yann_laurant

You could try the background callbacks in dash >=2.6.1 . You can pass dash components the running prop to show the progress.

24

mmh, looks nice but I don’t see how to manage dmc notification that way (progress bar will not be suitable for me because the long run is an external function).

Do you have an example of Notification using Interval ? I am stuck with an error at execution because enrich seems to do some modification in input/output, and i’m not able to return a dictionnary in data_store…

from dash import dcc, callback_context as ctx, no_update, callback
from dash.exceptions import PreventUpdate
from dash_extensions.enrich import Output, Input, State, html, DashProxy, LogTransform, DashLogger, MultiplexerTransform
import dash_mantine_components as dmc
from dash_iconify import DashIconify
import time

app = DashProxy(transforms=[MultiplexerTransform(), LogTransform()])
app.layout = html.Div([
    html.Button("Run", id="btn"),
    html.Button("Long", id="long"),
    html.Div(id="txt"),
    dcc.Store(id='notif_store'),
    dcc.Interval(
        id='interval-component',
        interval=1 * 1000,  # in milliseconds
        n_intervals=0,
        disabled=False
    )
])

@app.callback(Output("txt", "children"), Input("btn", "n_clicks"), log=True)
def do_stuff(n_clicks, dash_logger: DashLogger):
    if not n_clicks:
        raise PreventUpdate()
    dash_logger.info("Here goes some info")
    dash_logger.warning("This is a warning")
    dash_logger.error("Some error occurred")
    return f"Run number {n_clicks} completed"

@app.callback(Output("txt", "children"), Input("long", "n_clicks"), log=True)
def do_stuff(n_clicks, dash_logger: DashLogger):
    if not n_clicks:
        raise PreventUpdate()

    write_notification("work in progress", level='info', loading=True)
    time.sleep(10)
    write_notification("done", level='info', loading=False)
    return f"Long Run number {n_clicks} completed"


@app.callback(
    Output("notif_store", "data"),
    [Input('interval-component', 'n_intervals')],
    [State('notif_store','data')],
    log=True,
)
def update_notif(n,store,dash_logger: DashLogger):
    if not ctx.triggered:
        return no_update

    level, loading, message = read_notification()
    icomp = 0
    print(store)
    if store is not None:
        if 'level' in store:
            if store['level'] == level:
                icomp = icomp+1
        if 'loading' in store:
            if store['loading'] == loading:
                icomp = icomp + 1
        if 'message' in store:
            if store['message'] == message:
                icomp = icomp + 1
    else:
        store=dict()

    if icomp < 3:
        if level == "info":
            dash_logger.info(message, loading=loading)

    store['level']=level
    store['loading'] = loading
    store['message'] = message

    print(store)

    return store




def write_notification(message,level='info',loading=False):
    with open('notif_log.txt', 'w') as f:
        f.write('{};{};{}'.format(level,loading,message))

def read_notification():
    filename = 'notif_log.txt'
    with open(filename, 'r') as f:
        lines = f.readlines()
    last_notif = lines[-1].split(';')

    return last_notif[0], last_notif[1], last_notif[2]


if __name__ == '__main__':
    app.run_server(debug=True)
25

I’m getting closer to what I want to do… I thik what i’m looking for is in fact the Websocket ability of dash-extension…
But i’m not able to make the example here : Dash, probably because i have no idea of what is a websocket endpoint. Maybe I need to launch another process somewhere to fill the url : “ws://127.0.0.1:5000/random_data”

@Emil I have found an example here https://community.plotly.com/t/dash-extensions-websockets-working-example/50907/2 for someone who fight with the same issue, but the file does not exit anymore… can you help ?

Thanks in advance

Yann

26

The docs include a small example. Did you try that out? :slight_smile:

1 Like
27

yes I did, but I don’t understand here how the random data are generated (how is controlled “ws://127.0.0.1:5000/random_data”) - I miss some competency I guess… What is the port 5000 ? should it be the same as the Dash client ?

28

The example in the docs holds two pieces of code, the client app (i.e. the Dash part) and the websocket server (i.e. the part that emits the data),

import asyncio
import json
import random
from quart import websocket, Quart

app = Quart(__name__)

@app.websocket("/random_data")
async def random_data():
    while True:
        output = json.dumps([random.random() for _ in range(10)])
        await websocket.send(output)
        await asyncio.sleep(1)

if __name__ == "__main__":
    app.run(port=5000)

which runs as a separate process on a different port. The websocket server used in the example is Quart, but you can use any one you like.

29

Thanks @Emil ! By any chance, is there a documentary that explains available parameters as well as format ofLogConfig?

30

Currently, the docs only go through the basic,

I would recommend taking a look at the source code, if you need additional details. Could you elaborate on what you would like to achieve?

1 Like
31

Has anyone managed to get LogTransform working in an app that uses background callbacks?

In my app, LogTransform works and I really like how it plays out with the mantine notification component. However, I am struggling with getting my background callback registered by Celery. I am using Celery + Redis for the job queue.

Thanks @Emil for your work on dash-extensions!

32

@sislvacl Thanks! The LogTransform does not support background callbacks. Background are running in a separate process, so it would require an extensions of this existing implementation to support this usecase.

33

I’m sorry for confusion. I wasn’t clear enough. I don’t need to use dash logger in the background callback. I just need my background callbacks work while using dash logger in normal callbacks. I can’t get celery to register my background callbacks when using DashProxy.

34

Hi!
I created my own Logger class using Logging library and Dash Bootstrap Components (DBC). I call the useLogger() function when I initialize my Dash application, so when I use “logger.info(“message”)”, my message is displayed in a dbc.Toast(). The problem is that I deployed my application to GCP, and now these logger messages are shared between all sessions. How can I avoid this?

class DashLogger(logging.StreamHandler):
def __init__(self, stream=None):
        super().__init__(stream=stream)
        self.logs = list()
        self.levels = list()`

    def emit(self, record):
        try:
            msg = self.format(record)
            self.logs.append(msg)
            self.logs = self.logs[-1000:]
            self.levels.append(record.levelname)
            self.levels = self.levels[-1000:]
            self.flush()
        except Exception:
            self.handleError(record)

def useLogger():
    logger = logging.getLogger(__name__)
    logging.basicConfig(level=logging.INFO)
    dash_logger = DashLogger(stream=sys.stdout)
    logger.addHandler(dash_logger)

    return logger, dash_logger

Thanks in advance