Large numeric values returned from callback are rounded

Dash Python
1

Hello,
I’m currently working on an application where very large values are displayed in ag-grid table. However I’ve noticed that if the values exceed 16 digits they become rounded. Here’s an example:

    numbers = [
        1111111111111111111,
        2222222222222222222,
        3333333333333333333,
    ]
    test_df = pd.DataFrame()
    test_df["test"] = numbers
    grid = dag.AgGrid(
        rowData=test_df.to_dict("records"),
        defaultColDef={
        "sortable": True,
        "resizable": True,
        "editable": False,
        "filter": "agNumberColumnFilter",
    },
        columnDefs=[{"field": i} for i in test_df.columns],
        dashGridOptions=dash_grid_options,
        style={"height": "600px", "width": "100%", "border": "none"},
    )
    return grid

Produces this result:
image

As you can see the numbers become rounded started from the 17th digit. Unfortunately for my type of application this is unacceptable. I know for sure that it is not an issue related ag-grid because returning the above mentioned numbers from a callback to a simple dmc.NumberInput produces exactly the same results.

One workaround I’m considering would be to convert all numbers longer than 16 digits to strings, however I would loose some ag-grid functionalities that I care for, like numerical sorting and filtering.

Hence my question to You. Has anyone every encountered a similar issue and managed to solve or work around it?

Thanks!

2 Likes
2

Hello @Oliwer,

Welcome to the community!

You are probably encountering this issue in both places because they are both being interpreted as numbers, the dmc.NumberInput is obviously a number and then the grid, assuming you are using v31, is also inferring a number. The below is a basic number input from html. HTML input type="number"

image

image

I’m not sure there is a way around this…

You could create your own renderer or something.

Are these fields going to be editable? If not, you could just treat them as strings. Unless you are wanting to utilize some of the number filters?

3

Hey @jinnyzor

thanks for the answer. Unfortunately I don’t think it’s a problem with the renderer. I can already see that the values are rounded in the response:
image

So it looks like the rounding happens before the response even reaches the target renderer.
Same for the number input:
image

4

Here is a basic example to print out that the grid is definitely receiving the values as you present. This rowData gets updated directly from the grid after it comes in.

from dash import *
import dash_ag_grid as dag
import pandas as pd

app = Dash(__name__)

numbers = [
    1111111111111111111,
    2222222222222222222,
    3333333333333333333,
]
test_df = pd.DataFrame()
test_df["test"] = numbers
grid = dag.AgGrid(
    rowData=test_df.to_dict("records"),
    defaultColDef={
        "sortable": True,
        "resizable": True,
        "editable": False,
        "filter": "agNumberColumnFilter",
    },
    columnDefs=[{"field": i} for i in test_df.columns],
    style={"height": "600px", "width": "100%", "border": "none"},
)

print(grid.to_plotly_json()['props']['rowData'])

app.layout = grid

app.run(debug=True)

image

image
image924×442 27.1 KB

To confirm the data is received as such.

Please note, in future requests, please make a mre (minimal reproducible app) for us to check.

1 Like
5

It looks like this is a problem with HTML in general. Even in a string form, to work with filters, you need to convert it into an int, which makes it round again.

You can keep it in string form to allow the callbacks to work still.

6

I must admit I’m a bit confused, on your sample I can still see a network request _dash-layout, which presents with the rounded values in it:

image
image885×520 96.1 KB

I modified your MRE a little bit to add a callback. Here we can see that after pressing the button the values get updated and in the request we can notice that the values are rounded again, before reaching the ag-grid component.

image
image885×520 59.5 KB

Here’s the modified MRE:

from dash import *
import dash_ag_grid as dag
import pandas as pd
import dash_mantine_components as dmc


numbers = [
    1111111111111111111,
    2222222222222222222,
    3333333333333333333,
]
test_df = pd.DataFrame()
test_df["test"] = numbers
app = Dash(__name__)
app.layout = html.Div([
dag.AgGrid(
    id="grid",
    rowData=test_df.to_dict("records"),
    defaultColDef={
        "sortable": True,
        "resizable": True,
        "editable": False,
        "filter": "agNumberColumnFilter",
    },
    columnDefs=[{"field": i} for i in test_df.columns],
    style={"height": "600px", "width": "100%", "border": "none"},
),
    dmc.Button("Callback", id="button")
])


@callback(
    Output('grid', 'rowData'),
    Input('button', 'n_clicks'),
    prevent_initial_call=True,
)
def update_number(clicks):
    updated_df = pd.DataFrame()
    updated_df["test"] = [
    4444444444444444444,
    5555555555555555555,
    6666666666666666666,
    ]
    row_data = updated_df.to_dict("records")
    print(row_data)
    return row_data


app.run(debug=True)

Thanks for the help!

7

Unfortunately, I care about using filters, hence my cry for help. I was hoping that somebody managed to circumvent this limitation.

8

Even when punching in the number directly into the interface for filtering, it still runs into the same issue.

One possibility is to actually convert this to be an infinite rowModel and then use the server to perform all your other stuff. :slight_smile:

9

Here is a basic example:

from dash import *
import dash_ag_grid as dag
import pandas as pd
import dash_mantine_components as dmc

operators = {
    "greaterThanOrEqual": "ge",
    "lessThanOrEqual": "le",
    "lessThan": "lt",
    "greaterThan": "gt",
    "notEqual": "ne",
    "equals": "eq",
}


def filter_df(dff, filter_model, col):
    if "filter" in filter_model:
        if filter_model["filterType"] == "date":
            crit1 = filter_model["dateFrom"]
            crit1 = pd.Series(crit1).astype(dff[col].dtype)[0]
            if "dateTo" in filter_model:
                crit2 = filter_model["dateTo"]
                crit2 = pd.Series(crit2).astype(dff[col].dtype)[0]
        else:
            crit1 = filter_model["filter"]
            crit1 = pd.Series(crit1).astype(dff[col].dtype)[0]
            if "filterTo" in filter_model:
                crit2 = filter_model["filterTo"]
                crit2 = pd.Series(crit2).astype(dff[col].dtype)[0]
    if "type" in filter_model:
        if filter_model["type"] == "contains":
            dff = dff.loc[dff[col].str.contains(crit1)]
        elif filter_model["type"] == "notContains":
            dff = dff.loc[~dff[col].str.contains(crit1)]
        elif filter_model["type"] == "startsWith":
            dff = dff.loc[dff[col].str.startswith(crit1)]
        elif filter_model["type"] == "notStartsWith":
            dff = dff.loc[~dff[col].str.startswith(crit1)]
        elif filter_model["type"] == "endsWith":
            dff = dff.loc[dff[col].str.endswith(crit1)]
        elif filter_model["type"] == "notEndsWith":
            dff = dff.loc[~dff[col].str.endswith(crit1)]
        elif filter_model["type"] == "inRange":
            if filter_model["filterType"] == "date":
                dff = dff.loc[
                    dff[col].astype("datetime64[ns]").between_time(crit1, crit2)
                ]
            else:
                dff = dff.loc[dff[col].between(crit1, crit2)]
        elif filter_model["type"] == "blank":
            dff = dff.loc[dff[col].isnull()]
        elif filter_model["type"] == "notBlank":
            dff = dff.loc[~dff[col].isnull()]
        else:
            dff = dff.loc[getattr(dff[col], operators[filter_model["type"]])(crit1)]
    elif filter_model["filterType"] == "set":
        dff = dff.loc[dff[col].astype("string").isin(filter_model["values"])]
    return dff

numbers = [
    '1111111111111111111',
    '2222222222222222222',
    '3333333333333333333',
]
test_df = pd.DataFrame()
test_df["test"] = numbers
app = Dash(__name__)
app.layout = html.Div([
dag.AgGrid(
    id="grid",
    rowData=test_df.to_dict("records"),
    defaultColDef={
        "sortable": True,
        "resizable": True,
        "editable": False,
        "filter": "agNumberColumnFilter",
    },
    columnDefs=[{"field": i} for i in test_df.columns],
    style={"height": "600px", "width": "100%", "border": "none"},
),
    dmc.Button("Callback", id="button")
])

@callback(
    Output("grid", "getRowsResponse"),
    Input("grid", "getRowsRequest"),
)
def infinite_scroll(request):
    if request:
        dff = test_df.copy()
        ### format dff to be numbers
        dff['test'] = pd.to_numeric(dff['test'])
        if request["filterModel"]:
            filters = request["filterModel"]
            for f in filters:
                try:
                    if "operator" in filters[f]:
                        if filters[f]["operator"] == "AND":
                            dff = filter_df(dff, filters[f]["condition1"], f)
                            dff = filter_df(dff, filters[f]["condition2"], f)
                        else:
                            dff1 = filter_df(dff, filters[f]["condition1"], f)
                            dff2 = filter_df(dff, filters[f]["condition2"], f)
                            dff = pd.concat([dff1, dff2])
                    else:
                        dff = filter_df(dff, filters[f], f)
                except:
                    pass

        if request["sortModel"]:
            sorting = []
            asc = []
            for sort in request["sortModel"]:
                sorting.append(sort["colId"])
                if sort["sort"] == "asc":
                    asc.append(True)
                else:
                    asc.append(False)
            dff = dff.sort_values(by=sorting, ascending=asc)

        lines = len(dff.index)
        if lines == 0:
            lines = 1

        partial = dff.iloc[request["startRow"]: request["endRow"]]
        return {"rowData": partial.to_dict("records"), "rowCount": lines}
    return no_update


app.run(debug=True)

Obviously, if you want to add more data, you’ll have to get creative in how to do that.

10

Thanks for the example!! I’m going to need some time to analyze this and think about how it suits my needs :wink:

11

Sorry, here it is:

from dash import *
import dash_ag_grid as dag
import pandas as pd
import dash_mantine_components as dmc

operators = {
    "greaterThanOrEqual": "ge",
    "lessThanOrEqual": "le",
    "lessThan": "lt",
    "greaterThan": "gt",
    "notEqual": "ne",
    "equals": "eq",
}


def filter_df(dff, filter_model, col):
    if "filter" in filter_model:
        if filter_model["filterType"] == "date":
            crit1 = filter_model["dateFrom"]
            crit1 = pd.Series(crit1).astype(dff[col].dtype)[0]
            if "dateTo" in filter_model:
                crit2 = filter_model["dateTo"]
                crit2 = pd.Series(crit2).astype(dff[col].dtype)[0]
        else:
            crit1 = filter_model["filter"]
            crit1 = pd.Series(crit1).astype(dff[col].dtype)[0]
            if "filterTo" in filter_model:
                crit2 = filter_model["filterTo"]
                crit2 = pd.Series(crit2).astype(dff[col].dtype)[0]
    if "type" in filter_model:
        if filter_model["type"] == "contains":
            dff = dff.loc[dff[col].str.contains(crit1)]
        elif filter_model["type"] == "notContains":
            dff = dff.loc[~dff[col].str.contains(crit1)]
        elif filter_model["type"] == "startsWith":
            dff = dff.loc[dff[col].str.startswith(crit1)]
        elif filter_model["type"] == "notStartsWith":
            dff = dff.loc[~dff[col].str.startswith(crit1)]
        elif filter_model["type"] == "endsWith":
            dff = dff.loc[dff[col].str.endswith(crit1)]
        elif filter_model["type"] == "notEndsWith":
            dff = dff.loc[~dff[col].str.endswith(crit1)]
        elif filter_model["type"] == "inRange":
            if filter_model["filterType"] == "date":
                dff = dff.loc[
                    dff[col].astype("datetime64[ns]").between_time(crit1, crit2)
                ]
            else:
                dff = dff.loc[dff[col].between(crit1, crit2)]
        elif filter_model["type"] == "blank":
            dff = dff.loc[dff[col].isnull()]
        elif filter_model["type"] == "notBlank":
            dff = dff.loc[~dff[col].isnull()]
        else:
            dff = dff.loc[getattr(dff[col], operators[filter_model["type"]])(crit1)]
    elif filter_model["filterType"] == "set":
        dff = dff.loc[dff[col].astype("string").isin(filter_model["values"])]
    return dff

numbers = [
    '1111111111111111111',
    '2222222222222222222',
    '3333333333333333333',
]
test_df = pd.DataFrame()
test_df["test"] = numbers
app = Dash(__name__)
app.layout = html.Div([
dag.AgGrid(
    id="grid",
    defaultColDef={
        "sortable": True,
        "resizable": True,
        "editable": False,
        "filter": "agNumberColumnFilter",
    },
    rowModelType='infinite',
    columnDefs=[{"field": i} for i in test_df.columns],
    style={"height": "600px", "width": "100%", "border": "none"},
),
    dmc.Button("Callback", id="button")
])

@callback(
    Output("grid", "getRowsResponse"),
    Input("grid", "getRowsRequest"),
)
def infinite_scroll(request):
    if request:
        dff = test_df.copy()
        ### format dff to be numbers
        dff['test'] = pd.to_numeric(dff['test'])
        if request["filterModel"]:
            filters = request["filterModel"]
            for f in filters:
                try:
                    if "operator" in filters[f]:
                        if filters[f]["operator"] == "AND":
                            dff = filter_df(dff, filters[f]["condition1"], f)
                            dff = filter_df(dff, filters[f]["condition2"], f)
                        else:
                            dff1 = filter_df(dff, filters[f]["condition1"], f)
                            dff2 = filter_df(dff, filters[f]["condition2"], f)
                            dff = pd.concat([dff1, dff2])
                    else:
                        dff = filter_df(dff, filters[f], f)
                except:
                    pass

        if request["sortModel"]:
            sorting = []
            asc = []
            for sort in request["sortModel"]:
                sorting.append(sort["colId"])
                if sort["sort"] == "asc":
                    asc.append(True)
                else:
                    asc.append(False)
            dff = dff.sort_values(by=sorting, ascending=asc)

        lines = len(dff.index)
        if lines == 0:
            lines = 1

        partial = dff.iloc[request["startRow"]: request["endRow"]]
        partial['test'] = partial['test'].astype(str)
        return {"rowData": partial.to_dict("records"), "rowCount": lines}
    return no_update


app.run(debug=True)

The above one wasnt using the rowModel.

12

Thanks again, I’ll get back to You once I manage to dig into it!

1 Like
13

Looks like the solution proposed by @jinnyzor works. I marked it as a solution to the problem. Thanks a lot for all the help!

1 Like