I would like to use plotly+Dash to inspect a large dataset of vectors describing some spectral data, but it does not seem like
px.line is suitable for such plotting moderately large datasets. Is there anything that can be done to speed up examining large datesets to the point where callbacks will be functional, or do I need to learn a new tool? Below a toy example that generates a dataset large enough to make plotly unusable, you can set
n_rows to something small to verify the script is working.
import plotly.express as px import numpy as np import pandas as pd n_vars = 4000 x_range = np.linspace(0, 5*np.math.pi, n_vars) n_rows = 2500 total_x = np.empty_like(x_range, shape = (n_rows, n_vars)) classes = np.ones_like('hello_world', shape = (n_rows)) ids = np.ones_like('hello_world', shape = (n_rows)) #could vectorize the loop below by generating all of the `choice` at once, then using `np.where` #generating the data is plenty fast without this though for i in range(n_rows): choice = np.random.randint(low = 0 , high = 2, size = 1) if choice == 1: response = (np.random.randint(1,10) * np.random.rand()) + np.sin(x_range) c = 'sin' else: response = (np.random.randint(1,10) * np.random.rand()) + np.cos(x_range) c = 'cos' total_x[i, :] = response classes[i] = c ids[i] = 'sample' + "_" + str(i) df = pd.DataFrame(total_x) df['class'] = classes df['ids'] = ids df_m = df.melt(id_vars = ['class', 'ids'], var_name = 'index', value_name = 'response' ) px.line(data_frame = df_m, x = 'index', y = 'response', line_group = 'ids', color = 'class')
Any help is appreciated.