Watch logfile and draw it online

Dash Python
1

Hi.

I want constantly read data from file at the same time as this file is written by other service (something like tail -f file.log) and draw results. How draw it, I undestand: dash_core_components.Interval

I wanna do it without using queries brokers like redis etc, Is it possible?

2

Hi! I’m doing a similar thing with .csv files. For me it’s not being a problem using .Interval as an input in a callback, pandas.readcsv inside the callback function, and the figure of the chart in the output.

Cheers!

3

Hey! Did you manage to do it? I can’t find a way to auto-scroll to last line added (in order to print new logs)

4

One approach might be to Store the latest file position after each update. Then, you could do something like the following within the callback:

latest_position = my_tail_store

new_data = pd.DataFrame()
with open(filename, 'r') as fin:
    #skip to the latest read position before beginning processing remaining lines
    fin.seek(0, latest_position)

    # process line by line until eof
    for line in fin:
        # do something with each line that will prepare it for visualization in dash
        row = myparser.parse_logmessage(line.strip())
        # ... aggregate it to use once the file has been closed
        new_data.append(row)

    # record the current file.EOF
    latest_position = fin.tell()

# let's assume you have dash 0.38 rc1 installed, supporting multiple outputs in callbacks
return latest_position, new_data

If you’re just printing log message into a pre/div you could skip the read-by-line parsing stuff.

1 Like
5

this fit perfectly with the new multi output.

Thanks :slight_smile: