Compress Timeseries Plotly Json Export

📊 Plotly Python
1

I am using plotly in conjunction with weights and biases to log some rather hefty timeseries data (about 24 x 30,000 point charts per plot). Each plot is simply a line, with an x-axis of timestamps.

Each plot somehow takes up 50MB on disk, which is huge. My suspicion is that this is because each line is saved with unique x data, consisting of timestamp strings (see pic below).

Is there any way to force plotly to store unix timestamps instead of iso strings, and to make it share the x-axis data between plots with the same x-axis instead of storing a copy for each line? (ideally, it would simply store something like daterange(start, end, frequency) for the x-axis…).

Thanks for any help!

Screenshot 2022-09-26 at 18.48.41
Screenshot 2022-09-26 at 18.48.41782×884 97.4 KB

2

What you can do is pass in x0 and dx values to go.Scatter (docs), which instead of storing the whole list of timestamp, will only store your step size and initial date.

For example if your time-series data has a period of 60s (60*1000 milliseconds), you can do:

go.Scatter(x0=start_datetime, dx=60*1000, y=your_data)