I cannot make plotly dash work for my live update dashboard

I have the below code that suppose to be doing the below:

1- Capture traffic form a NIC, but this is a separate file which will run continuously outside this section

2- The main_process function will read the files captured by point 1 above and convert them into df, but I changed this to static df, which should be ok for testing. then the same function calculates something and then generates the px plotting, then pass the result outside the function using multiprocessing pipe()

3- The plotting function will receive the drawing details from the main_process function via the MPing pipe() then will start the layout section, it runs for 1 time within the if statement

Now, the MPing will control the starting of each function, I added delays to test controlling the sequence of function

I have few issues with this code:

1- When the server starts, it does not recognize the layout, even though I am controlling what starts first, but probably because it is inside the function

2- If I move the app.layout outside the function, I will not be able to get the data for plotting, as it was not created yet first and second it is inside a function

3- If I move the run_server outside the if block, again it will not recognize the layout

I think I need to use the callback, but not sure how to use it, none of the online examples made me understand how to use it in my case, to be honest, and the reason why I am thinking about the callback is because the data will change every 30 seconds, so the dashboard should be updated accordingly

Any help will be appreciated

import pandas as pd 
from dash import html, Dash, dcc
import plotly.express as px
import os, glob, time, sys
from multiprocessing
import Pipe
import multiprocessing as mp

app = Dash(__name__)
 
def main_process(conn2):
     print('main process started')
     print('The process ID for the main_process is', os.getpid())
     i = 0
     while True:
         time.sleep(5)
         data = [['192.168.0.105','192.168.0.103','6'],
                 ['192.168.0.103','192.168.0.105','6'],
                 ['192.168.0.105','192.168.0.103','6'],
                 ['52.98.151.66','192.168.0.105','6'],
                 ['192.168.0.105','52.98.151.66','6'],
                 ['52.98.228.226','192.168.0.105','6'],
                 ['52.98.228.226','192.168.0.105','6'],
                 ['192.168.0.103','224.0.0.251','17'],
                 ['192.168.0.105','52.98.228.226','6']]
         column_names = ['Source', 'Destination','Protocol']
         for i in range(6):
             df = pd.DataFrame (data)
             df.columns = column_names
         # Ploting top 10 Inbound traffic IP addresses
         inbound_ip_count = df.groupby('Source').size().nlargest(n=10)
         top_in_IP = px.bar(df['Source'],
                             x=inbound_ip_count.index, y= inbound_ip_count.values,
                             labels={
                               'x': 'IP Addresses',
                               'y': 'Packets',
                           },
                                 title='Top 10 incoming traffic IP addresses')
         
         data_out = top_in_IP
         print('data_out generated')
         conn2.send(data_out)
         print('data_out sent')
 
def plotting(conn1):
     n = 0
     print('The process ID for the plotting is', os.getpid())
     while True:
#        time.sleep(2)
         data_out = conn1.recv()
         print('data_out received')
         fig0 = data_out
         n +=1
         print('n is ', n)
         if n == 1:
             print('starting the layout')
             app.layout = html.Div(className="row", children =[
                 html.H1('Network Traffic charts', style={'text-align':'center'}),
                 html.Div(children = [
                     dcc.Graph(figure=fig0, style={'width': '95vh', 'height': '90vh','display': 'inline-block'}),
                 ])
             ])
             print('app.layout loaded')
 
l = 0 if __name__ == "__main__":
    conn1, conn2 = Pipe(duplex=False)
    p1 = mp.Process(target=main_process, args=(conn2,))
    p2 = mp.Process(target=plotting, args=(conn1,))
    p1.start()
    p2.start()
    time.sleep(10)
    l +=1
    if l == 1:
         print('Before server running', l)
         print('The process ID for the server is', os.getpid())
         time.sleep(18)
         app.run_server(port=2020, debug=False, use_reloader=False)

Hello @arafeek,

Welcome to the community!

You should take a look here:

That being said, your app should not be what is updating the data, but only reading the updated file and/or database. To read the data, you can use an interval to do so.

Thanks for the reply.

The live update seems to be happening when a user selects something, like dropdown list, button, etc, (correct me if I am wrong), my situation is a bit different, my plotting variables are continuously changing, every 30 seconds, this change is controlled by the first if statement in my code,( and another engine in the background that I did not include here), the actual df creation code is not included in the example, because it involves reading files from the local drive and generating the df, so I substitute it with the data in the code.

Anyway, as I mentioned, my issue is that the input to the layout being updated local (by reading from another function) and the sync with the server run

Again, maybe I am not understanding the live updates and callback correctly to understand your repky

Thanks again

As I understand, your variable changing for every 30 second and you want to update graphs based on new variable (dataframe). So as jinnyzor’s mention. I think you could use dcc.Interval for this. Something as below:

import ccxt
from datetime import datetime
import plotly.graph_objects as go
import dash
import dash_bootstrap_components as dbc
from dash import Input, Output, State, dcc, html

# connect dahs page to main app
app = dash.Dash(__name__, title='Dashboard', external_stylesheets=[dbc.themes.BOOTSTRAP])
app.layout = html.Div([
    dcc.Graph(id='candlestick', figure={}),
    dcc.Interval(
            id='interval-component',
            interval=5*1000, # in milliseconds
            n_intervals=0
        )
])

@app.callback(Output('candlestick','figure'), 
              Input('interval-component','n_intervals'))
def update_graph(n):

    binance = ccxt.binanceusdm()
    trading_pair = 'BTC/USDT'
    candles = binance.fetch_ohlcv(trading_pair, '1m')
    dates = []
    open_data = []
    high_data = []
    low_data = []
    close_data = []

    # format the data to match the charting library
    for candle in candles:
        dates.append(datetime.fromtimestamp(candle[0] / 1000.0).strftime('%Y-%m-%d %H:%M:%S.%f'))
        open_data.append(candle[1])
        high_data.append(candle[2])
        low_data.append(candle[3])
        close_data.append(candle[4])

    # The latest status of the bar
    Candleـupdate = [dates[-1], open_data[-1], high_data[-1], low_data[-1], close_data[-1]]

    fig = go.Figure(data=[go.Candlestick(x=dates,
                open=open_data, high=high_data,
                low=low_data, close=close_data)])

    return fig
if __name__ == "__main__":
    app.run_server(debug=False)

So for each 5 second, graphs will be updated with new data.

This worked, thanks a lot, I actually have seen this example, but could not figure out how to tweak it to my case, I guess because I just woke up fresh :slight_smile:

Thanks for pointing it out to me

1 Like

One more question, if you don’t mind, now this code is presenting 1 chart, however I have 4, so is there a way to have the 4 displayed? two per row.

I tried adding the second figure as fig2, and have it in the return

return fig, fig2

but that did not work, I am not sure how fig is called to be displayed

Did you add more dcc.Graph in your layout and define it in your callback Output?

I added in the dcc.Graph but not the callback, now I tried the callback, but did work, I am not sure how to do it, here is what I did:

     @app.callback(Output('bar_chart','figure'),
          Output('bar_chart','figure2'), 
          Input('interval-component','n_intervals'),
          Input('interval-component','n_intervals'))

and here is the dcc.Graph section:

fig2 = go.Figure(data = px.bar(df['Destination'],
                    x=outbound_ip_count.index, y=outbound_ip_count.values,
                    labels={
                      'x': 'IP Addresses',
                      'y': 'Packets',
                  },
                    title='Top 10 Outbound traffic IP addresses')
                 )

First of all you need to set id name of dcc.Graph be different. In the Output you have to change from:

 @app.callback(Output('bar_chart','figure'),
      Output('bar_chart','figure2'), 
      Input('interval-component','n_intervals'),
      Input('interval-component','n_intervals'))

to:

 @app.callback(Output('bar_chart','figure'),
      Output('bar_chart_2','figure'), 
      Input('interval-component','n_intervals'))

With my old sample, you can revise as below:

import ccxt
from datetime import datetime
import plotly.graph_objects as go
import dash
import dash_bootstrap_components as dbc
from dash import Input, Output, State, dcc, html

# connect dahs page to main app
app = dash.Dash(__name__, title='Dashboard', external_stylesheets=[dbc.themes.BOOTSTRAP])
app.layout = html.Div([
    dbc.Row([
        dbc.Col([
            dcc.Graph(id='candlestick', figure={})
        ]),
        dbc.Col([
            dcc.Graph(id='candlestick2', figure={})
        ]),
        dcc.Interval(
            id='interval-component',
            interval=5*1000, # in milliseconds
            n_intervals=0
        )
    ])
])

@app.callback(Output('candlestick','figure'), 
              Output('candlestick2','figure'),
              Input('interval-component','n_intervals'))
def update_graph(n):

    binance = ccxt.binanceusdm()
    trading_pair = 'BTC/USDT'
    candles = binance.fetch_ohlcv(trading_pair, '1m')
    dates = []
    open_data = []
    high_data = []
    low_data = []
    close_data = []

    # format the data to match the charting library
    for candle in candles:
        dates.append(datetime.fromtimestamp(candle[0] / 1000.0).strftime('%Y-%m-%d %H:%M:%S.%f'))
        open_data.append(candle[1])
        high_data.append(candle[2])
        low_data.append(candle[3])
        close_data.append(candle[4])

    # The latest status of the bar
    Candleـupdate = [dates[-1], open_data[-1], high_data[-1], low_data[-1], close_data[-1]]

    fig = go.Figure(data=[go.Candlestick(x=dates,
                open=open_data, high=high_data,
                low=low_data, close=close_data)])

    return fig, fig
if __name__ == "__main__":
    app.run_server(debug=False)

Excellent, thanks, in your example you re displayed the same graph, but I have a different, do created a new fig1, and

return fig, fig2

Thanks again

1 Like

Yep, it’s just example. You can change name of fig and return it with callback.

1 Like

One more Q, and I hope will be the last for this topic, now I have the four charts that I want, but they are all in one row, how can separate them and make them in two rows 2x2?

You can separate it in 2 Row by using something like below:

app.layout = html.Div([
    dbc.Row([
        dbc.Col([
            dcc.Graph(id='candlestick', figure={})
        ], width={'size': 6}),
        dbc.Col([
            dcc.Graph(id='candlestick2', figure={})
        ], width={'size': 6})
    ]),
    dbc.Row([
        dbc.Col([
            dcc.Graph(id='candlestick3', figure={})
        ], width={'size': 6}),
        dbc.Col([
            dcc.Graph(id='candlestick4', figure={})
        ], width={'size': 6})
    ]),
    dcc.Interval(
        id='interval-component',
        interval=5*1000, # in milliseconds
        n_intervals=0
    )
])
1 Like

bro got my ip :skull: