How to work with pandas NAType in DataTable

Dash Python
1

Hi.
I’ve converted several of my columns to Int16DType to reduce the amount of memory consumed.
When this happens all NaN are then converted to be pandas.NA type which seems not to be correctly handled when I try to present a DataTable.
The error I got seems to be JSON related:

Traceback (most recent call last):
  File "/Users/goncalo.valverde/Library/Python/3.8/lib/python/site-packages/dash/dash.py", line 1025, in add_context
    jsonResponse = json.dumps(
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/json/__init__.py", line 234, in dumps
    return cls(
  File "/Users/goncalo.valverde/Library/Python/3.8/lib/python/site-packages/_plotly_utils/utils.py", line 45, in encode
    encoded_o = super(PlotlyJSONEncoder, self).encode(o)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/Users/goncalo.valverde/Library/Python/3.8/lib/python/site-packages/_plotly_utils/utils.py", line 115, in default
    return _json.JSONEncoder.default(self, obj)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type NAType is not JSON serializable

This is the code I’m using to generate the DataTable:

dash_table.DataTable(
                id="table",
                columns=[{"name": i, "id": i} for i in df.columns],
                data=df.to_dict("records"),
)

Any ideas how to solve this or to work around?

1 Like