i’m build a webapp on Dash. My work is based on the Wind-Streaming example, but instead of querying a SQLite DB i will query a MongoDB database. I will be pushing data to my database and later to my pandas dataframe about every 5 seconds. So basically my chart will be updated with real time data constantly.
My concern is that at a certain point i will have a lot of data (about 1M entries on my DB table), will charting all that data crash my project or make it extremely heavy? I was thinking of updating my data instead of just adding it, but the problem is that if i overwrite data, that data will also vanish from the chart, so i won’t have historical data on my chart. How can i avoid this?