My app parse the JSON data from a message bus (e.g. Apache Kafka) and display the data on the web browser. How do I update the graph only when I receive a new data? (Instead of constantly calling callback function every 1 second)
hi @tud7
You might have to use Async-Dash for this or similar components in Dash-extensions. @snehilvj would you be able to give @tud7 more guidance on this please.
One option would be to create a small Quart server (or similar) that listens to the relevant Kafka topic(s) and pipes each message into a websocket endpoint that you then listen to in your Dash app using a WebSocket component.
Thanks guys for the response. I’m working on production system, so Async-Dash might not be an option for now.
I will look into WebSocket solution.
Can Dash listen to an object or variable?
I’m thinking another solution is maybe the Kafka Consumer will listens to the Kafka topic and update an internal object which will trigger the Dash GUI to update the graph.
No, that is not possible. You need something to initiate the update from the client. The typical components used to do this are the Interval component and the WebSocket component (or the EventSource component). The former updates at regular intervals (which you seem to not like), while the latter allows for pushing updates from the server (which seems to be what you are looking for).
If you want to avoid the websocket (or event source) layer, you could create a Dash component that consumes Kafka messages directly. I imagine an implementation based on KafkaJS should be possible, but it would take a lot more work (as compared to the initial proposal of a simple websocket proxy).
Got it thanks. Seems like either with EventSource or WebSocket, I still need a small web server to listens to Kafka topics and pipes to an URL endpoint.
Is Interval a good practice in dynamic GUI design? The data might come at different rate, so I’m thinking more like a event-driven approach, but if Interval is how people usually handle it, I can try this approach too. Then I can just retrieve data from Kafka when it’s time to update.
I would say it depends on how close to real time you need the data to be. The Interval component is by far the simplest approach, in particular in terms of the infrastructure needed. So if you can live with the slight delay introduced by polling updates (e.g. once per minute), it might not be worth the trouble to use a websocket.
what is the polling update that you mentioned?
Sorry not being explicit, by polling I mean using the Interval component (to trigger a poll).
“ One option would be to create a small Quart server (or similar) that listens to the relevant Kafka topic(s) and pipes each message into a websocket endpoint”
Do we have an example of how the quart server looks like? Maybe example code?