Help international users of Plotly, non-Latin encoding problem

[Veronika]

I am trying to display a pie chart with Plotly express in a Django app. The chart displays OK, but all data, labels, title are in Unicode (e.g. “\u031f\u0435...” I assume it is Unicode). My data is in a non-Latin language. Everywhere else in Django templates my non-Latin data is displayed OK. The issue is only when using Plotly.

Following the discussion in this thread: RTL and bidi text support - #4 by adamschroeder , I added local, but to no avail.

import local 
import plotly.express as px

def pie_chart(themes_data):
    locale.setlocale(locale.LC_ALL, "sk_SK.utf8")  # also tried "sk_SK"
    fig = px.pie(
        themes_data,
        values="count", 
        names="name",
        title="My non-latin title",
    )
    chart = fig.to_html()
    context = {'chart': chart}
    return render(request, 'chart.html', context)

How to make plotly use utf-8 and correctly display non-Latin characters?

(Django==5.1.4, plotly==6.0.1)

[adamschroeder]

@PipInstallPython if I recall correctly you’ve worked with Plotly in Django. Did this kind of thing ever happen to you?

@Veronika when you do this only with Plotly, outside of Django, do you get the same error?

[PipInstallPython]

I havent seen this bug, and i also dont currently use the package django plotly dash in my slack. Sorry

[Veronika]

Thank you for responding!
I tried Plotly in Jupyter lab, foreign words display fine.
But this does not exactly reproduce a web app in Django.
What is different in Django:

• data comes form a database (but it is utf-8 encoded by default)
• instead of fig.show() I use fig = fig.to_html()

New developments in the issue:

• not just data appears in unicode, but also <br> in hover text appears as unicode
• (somewhat off topic) When I give up and use English labels, get my chart perfectly fine on a locale machine in Django. I stage on a server and get MemoryError pointing to the line with fig = fig.to_html(). I do use a light grade server, but not to the point as to run out of memory on making a simple chart.

I am open to any advice on how to use plotly in an actual web app. (Display non-Latin characters and not to run out of memory with fig.to_html()).

My thoughts to try: generate a static chart, or to try to complement Plotly with Bokeh.

[PipInstallPython]

You might find this useful:

Settings | Django documentation

My projects architecture looks like:

I’m not using the django-plotly-dash package, but instead I have 3 applications hosted in my vps within sub domains on a docker swarm. So 2 of the applications I’m running are django and one is a dash application.

The idea of the stack is I broke the application into django backend & dash as the frontend app with an extra django application built with oscar to act as an e-commerce site.

Not sure how much more help I can be with this as my setup is much difrent than what you are working within but In my django application I have a locale folder for changing the languages of my django application.

This README.md provides some context into how I set this up and use it, but I’m not entirely sure if this would fix your problem or not.. it might as it could translate everything prior to it being sent to the plotly graph.

Translations

Start by configuring the LANGUAGES settings in base.py, by uncommenting languages you are willing to support. Then, translations strings will be placed in this folder when running:

docker compose -f local.yml run --rm django python manage.py makemessages -all --no-location

This should generate django.po (stands for Portable Object) files under each locale <locale name>/LC_MESSAGES/django.po. Each translatable string in the codebase is collected with its msgid and need to be translated as msgstr, for example:

msgid "users"
msgstr "utilisateurs"

Once all translations are done, they need to be compiled into .mo files (stands for Machine Object), which are the actual binary files used by the application:

docker compose -f local.yml run --rm django python manage.py compilemessages

Note that the .po files are NOT used by the application directly, so if the .mo files are out of dates, the content won’t appear as translated even if the .po files are up-to-date.

Production

The production image runs compilemessages automatically at build time, so as long as your translated source files (PO) are up-to-date, you’re good to go.

Add a new language

1. Update the LANGUAGES setting to your project’s base settings.
2. Create the locale folder for the language next to this file, e.g. fr_FR for French. Make sure the case is correct.
3. Run makemessages (as instructed above) to generate the PO files for the new language.

[Veronika]

Interesting stack, thank you, should look into it