Major Releases of Plotly and Dash: Narwhals, Dash Hooks, and More

Major Releases of Plotly and Dash: Narwhals, Dash Hooks, and More

Written by: Nathan Drezner and Greg Wilson

We’re heading into another icy Canadian winter at the Plotly headquarters in Montreal. What better way to embrace the cold than with a warm batch of open-source updates?

Four of Plotly’s open-source libraries are getting major releases. Spoiler: lightning-fast server-side performance with Narwhals in Plotly.py, a new hooks system for Dash, a new design for Kaleido, and more. These improvements will make building, deploying, and sharing data apps faster than ever before.

Read on to learn what’s new and see how to break through frustrating obstacles freezing your development.

Plotly.py 6.0 rewrites dataframe performance with Narwhals

First up: we are completely rewriting how our Plotly.py library talks to dataframes in the 6.0 release. Instead of relying on the Pandas API, we are using Narwhals which provides an abstraction layer over several kinds of tabular data. This means faster, more efficient handling of tabular data and serious performance gains for data apps at scale.

You’ll notice with this change that Plotly.py no longer has to do in-memory copying when you hand it something like a Polars dataframe. The change directly addresses server-side bottlenecks, making data processing faster and web apps more responsive. This work was contributed by a community member, Francesco Bruzzesi, with lots of input from Marco Gorelli, and is discussed in detail in an upcoming blog post.

Plotly t-shirt

There’s an old joke that a programmer is someone who will do something for a t-shirt that they wouldn’t do for $100. Switching to Narwhals cost us two t-shirts; we think it was a great investment.

In Plotly.py 6.0, we are dropping support for Jupyter Notebook versions before 7.0, which allows us to simplify our build and release process and take advantage of lighter-weight packaging and new features in the latest versions of Jupyter. Again, the work is large enough that we will devote an additional blog post to it. There are minor changes in Plotly.py 6.0 as well, mostly to reflect changes in Plotly.js: please see the release notes or download the release candidate from PyPi or install it directly with:

pip install --upgrade plotly --pre

Again, if you spot any problems, please file an issue on the Plotly.py GitHub page.

Narwhals, Jupyter, and Polars

Plotly.js 3.0 introduces improved integration and faster builds

Plotly.js is the core plotting library powering our visualizations. In its 3.0 release, we’ve modernized it to use JavaScript ES6 imports and switched from webpack to esbuild for building releases.

Moving to ES6 lets us remove our dependence on require.js and jQuery, making it far easier for third parties to integrate Plotly.js into their systems. Meanwhile, the shift to esbuild cuts build times by over 50%, saving developers time and making iteration smoother than ever.

Plotly.js data visualization

We are also using this release as an opportunity to remove features that were deprecated years ago. From the opacity of error bars to 3D camera positions, we are finally taking out attributes that have more modern equivalents: see the release notes for details, or try out the release candidate at npm. If you spot any problems, please report them by filing an issue at the Plotly.js GitHub page.

In between: faster communication between JavaScript and Python libraries

The final major improvement to our core plotting libraries affects them both: we are now using base64 encoding to make communication between the JavaScript and Python libraries faster. We will dive into the details and share some performance data in yet another upcoming post.

core Plotly plotting libraries

Dash 3.0 offers a new hooks system

Dash 3.0 offers developers a hooks system: a way to tell Dash to run functions at certain points in a dashboard’s lifecycle. Using these hooks allows developers to extend Dash without having to fork the project or dig into its source code. As we will describe in an upcoming blog post, Dash offers:

  • Setup hooks that run during application initialization;
  • Layout, routing, and callback hooks that simplify the creation of custom components;
  • Error hooks that allow applications to do things like log messages to enterprise infrastructure;
  • Index hooks that make it easier to connect to things like third-party analytics services;
  • Client-side hooks for building those custom components; and
  • A pair of API functions for adding custom CSS and JavaScript modules to applications.

Bug reports in the Dash GitHub page are very welcome, but please do take a moment to check existing issues before filing yours.

Major overhaul of Kaleido

As previously announced, the last of our major releases is a complete overhaul of Kaleido, a library that converts charts to static images in various formats programmatically. Previous versions of Kaleido have included a custom build of the Chrome browser wrapped in Python. While this approach allowed us to take advantage of Chrome’s SVG-to-image conversion, it had stability problems on Windows and had proved impossible to maintain (not least because it took several hours to compile).

Thanks to work from Andrew Pikul, the re-architected version has two parts: a new package called Choreographer that enables Python programs to launch and control a browser, and a greatly simplified Kaleido that uses that to turn charts into images. The combination is literally thousands of times smaller than the original, and as a result, will be much easier to maintain. It is also slower, but we have ideas for addressing that: if you’d like to help out, please have a look at the GitHub page for open issues.

Kaleido diagram

Thank you to the Plotly contributor community — our Plotly champions!

These releases would not have been possible without the work of Plotly’s contributor community. You can join the Plotly community by contributing to any of our open-source projects or by starting one that builds on what others have created; to find out how, visit our community contribution guide.

Stay tuned for follow-up articles on our releases, and to learn more, sign up for our upcoming product spotlight event.

14 Likes

Where can we read more about Dash 3.0? Is there draft docs of the new hooks system mentioned?

1 Like

Blog post will be coming out soon. Hopefully, this week.

2 Likes

I am not convinced how an API wrapper can make anything faster. In Narwhals’ own words (the emphasis is mine):

So, what’s the overhead of running pandas vs pandas via Narwhals?
Based on experiments we’ve done, the answer is: it’s negligible. Here are timings from the TPC-H queries, comparing running pandas directly vs running pandas via Narwhals:

On some runs, the Narwhals code makes things marginally faster, on others marginally slower. The overall picture is clear: with Narwhals, you can support both Polars and pandas APIs with little to no impact on either.

Hey @avm23 – The performance improvements are for developers working with non-Pandas dataframes, like Polars, PyArrow, Modin, and cuDF – we’ve seen pretty significant performance improvements (2-3x) for those data types which we’ll be deep-diving in a blog post this week!

Our target performance overhead for Pandas dataframes was 5% but in most cases in most cases we saw a 1-2% overhead in performance, and the average case was below 5% overhead – we’ll be publishing a detailed blog on this later this week, but if you want a sneak peek at the results, check out the profiling results we were using during development of this feature :wink:: Visualise timings | Kaggle and Narwhals Plotly.py Performance Test · GitHub.

2 Likes

Will Dash release an RC preview version before the official 3.0 release to assist with some preliminary testing?

The hooks sound like an amazing new feature! When will Dash 3.0 be available? :smiley:

1 Like

Yes we will! We’re targeting this week for the RC.

1 Like

We’re planning to issue the release candidate this week with GA TBD as we hear feedback from the community on the RC :hugs:

3 Likes

Any updates on this? really looking forward! :slight_smile:

1 Like

That sounds great! Where can we find the RC? (:

https://pypi.org/project/plotly/6.0.0rc0/

To install with pip:
pip install plotly==6.0.0rc0

The Dash RC was delayed as we wrap up Default to react 18 by T4rk1n · Pull Request #3093 · plotly/dash · GitHub : Should land early in the new year as the team returns from the holidays! I’ll keep this thread updated :raised_hands:

1 Like