Retrieving Information for All Modified Cells in dash_ag_grid

wowwwn · December 23, 2023, 12:15pm

Hi,
I’m currently using dash_ag_grid to construct a dash webapp with features similar to Excel.
I’m dealing with big CSV data, hence I’m using ‘infinite’ rowModel (extually its ‘serverside’).
I’ve activated the EnterpriseModules and using gridOptions like below

dashGridOptions={
    "undoRedoCellEditing": True,
    "enableRangeSelection": True,
    "undoRedoCellEditing": True,
}

I’ve made it editable and when a cell is modified, I’ve synchronized it to modify the corresponding cell in the dataframe through ‘cellValueChanged’ in the callback.
(referenced from Editing and Callbacks | Dash for Python Documentation | Plotly)

The problem arises when I select a range of cells, copy and then paste it elsewhere, as in Excel. Even though multiple cells get modified at once, in the callback, ‘cellValueChanged’ shows information for only one cell (the last one among the modified cells).

Since the rowModel is ‘infinite’, if the modifications are not reflected in the dataframe, the changes will all be lost when the grid is refreshed.

How can I retrieve information for all the modified cells?

I’m sharing the code I’m using for your reference.

import dash_ag_grid as dag
from dash import Dash, Input, Output, html, no_update, callback, State, Patch
import pandas as pd
import dash_bootstrap_components as dbc
app = Dash(__name__)


df = pd.read_csv(
    "https://raw.githubusercontent.com/plotly/datasets/master/ag-grid/olympic-winners.csv"
)

columnDefs = [
    {
        "headerName": "ID",
        "maxWidth": 100,
        "valueGetter": {"function": "params.node.id"},
    },
    {"field": "athlete", "minWidth": 150},
    {"field": "country", "minWidth": 150},
    {"field": "year"},
    {"field": "sport", "minWidth": 150},
    {"field": "total"},
]

defaultColDef = {
    "flex": 1,
    "minWidth": 150,
    "sortable": False,
    "resizable": True,
    'editable': True
}


adding_newColumn = dbc.Col(
    [
        html.Div("New Column:"),
        dbc.InputGroup(
            [
                dbc.Input(
                    id="new-column-header-input",
                    placeholder="header",
                    type="text",
                    size="sm",
                    className="smaller-input",
                ),
                dbc.Input(
                    id="new-column-value-input",
                    placeholder="value",
                    type="text",
                    size="sm",
                    className="smaller-input",
                ),
                dbc.Button(
                    "Add",
                    id='add-new-column-btn',
                    color="secondary",
                    size="sm",
                    className="me-1",
                    # disabled=True,
                ),
            ],
            size="sm",
        ),
    ],
    width=2,
)

app.layout = html.Div(
    [
        adding_newColumn,
        dag.AgGrid(
            id="infinite-row-grid",
            columnDefs=columnDefs,
            defaultColDef=defaultColDef,
            rowModelType="infinite",
            enableEnterpriseModules=True,
            dashGridOptions={
                "undoRedoCellEditing": True,
                "enableRangeSelection": True,
                "undoRedoCellEditing": True,
            },

            style={"height": "70vh"},
        ),
        html.Div(id="editing-grid-output"),
    ],
)


@callback(
    Output("infinite-row-grid", "getRowsResponse"),
    Input("infinite-row-grid", "getRowsRequest"),
    Input("infinite-row-grid", "columnDefs"),
)
def infinite_scroll(request, columnDefs):
    if request is None:
        return no_update
    partial = df.iloc[request["startRow"]: request["endRow"]]
    return {"rowData": partial.to_dict("records"), "rowCount": len(df.index)}


@callback(
    Output("editing-grid-output", "children"), Input("infinite-row-grid", "cellValueChanged")
)
def update(cell_changed):
    return f"{cell_changed}"


@callback(
    Output("infinite-row-grid", "columnDefs"),
    Input("add-new-column-btn", "n_clicks"),
    State("new-column-header-input", "value"),
    State("new-column-value-input", "value"),
    prevent_initial_call=True,
)
def add_column(n, header, value):
    global df
    df[header] = value
    patched_columnDefs = Patch()
    patched_columnDefs.append({"field": header, 'editable': True})
    print(df.head())

    return patched_columnDefs


if __name__ == "__main__":
    app.run(debug=True)

Thankyou

jinnyzor · December 23, 2023, 1:30pm

Hello @wowwwn,

What happens in the console if you add this event listener? Do you see all the events?

dash_ag_grid.getApiAsync(id).then((grid) => grid.addEventListener(‘cellValueChanged’, (e) => console.log(e)))

Put this in a clientside_callback on the grid’s id and then return window.dash_clientside.no_update

wowwwn · December 23, 2023, 2:23pm

Hello @jinnyzor,

I am always grateful for your prompt responses.

I am not very familiar with JavaScript, but I tried adding the following code as you suggested:

app.clientside_callback(
    """async function (id) {
        
        dash_ag_grid.getApiAsync(id).then((grid) => grid.addEventListener('cellValueChanged', (e) => console.log(e)))
        
        return window.dash_clientside.no_update
    }""",
    Output("infinite-row-grid", "id"),
    Input("infinite-row-grid", "id"),
)

When I pressed ‘F12’ to look at the console in the chrome browser, I confirmed that it outputs the information I wanted (cellValueChanged for all modified cells).

As I’m familiar with Python and less so with JavaScript or React, I wonder if there’s a way to handle this information in a Python callback?

Also, I’m very interested in the advanced features of ag_grid that are not supported in dash_ag_grid. Do you have any recommended resources for JavaScript or React I could refer to?

I truly appreciate your help every time. I’m not sure how I can ever repay your kindness.

Merry Christmas

Emil · December 23, 2023, 2:39pm

Do you get all events, if you use a clientside callback that just prints to console?

wowwwn · December 23, 2023, 2:40pm

Yes, but only last one in the callback

jinnyzor · December 23, 2023, 3:22pm

Ok.

So then you’ll need to make your own handle for the events, there is probably something that keeps it from triggering too quickly.

You can always check your network tab to see if the callbacks are being sent to the server.

Infinite scrolling should have a cache though, so as you scroll back and forth and up and down you should be pulling from the cache. The issue is syncing the cellValueChanged to the server.

What you could do is build your own store on the clientside that can collect the data and then pass it in chunks periodically.

The thing I am wondering, is this info shared among other users? If they need these changes realtime, then you might need to use something that can send the fetch requests at a much higher rate and the server that can handle a higher frequency of updates.

If these are only local to your user, then I think there isn’t an issue with just allowing the grid to cache the data and then when you leave, have it push the store of changes back to the df.

As an aside, typically I shy away from realtime changes and push changes on a button push by the user. You can run into issues with a db being locked or people stepping on each others toes, etc.

Emil · December 23, 2023, 6:22pm

What about attaching a custom event handler for the cell changed event that collects all events with some delay (say, 100 ms or so), and then fires a new event to trigger a Dash callback with the resulting list of events, which event are no longer arriving?

EDIT: Maybe this could actually be implemented as part of dash agreed itself (

jinnyzor · December 23, 2023, 6:35pm

Hmm. That might work.

Would change this to a list though. We could potentially do it in the open PR to the next version.

Emil · December 23, 2023, 7:29pm

We could change type to union of list/single element, set default delay to -1, in which case a single element is returned, thus preserving backwards compatibility.

jinnyzor · December 23, 2023, 7:36pm

But, this could be confusing as far as writing codes where a single element is still returned when multiple is expected.

The lift for cellValueChanged would be a find and replace with adding on a [0], where the alternative would be to check isinstance list or dict.

Emil · December 23, 2023, 7:39pm

That’s true. So I guess the question is if we value backwards compatibility versus returning the same type (list) always. I can see the benefits of each choice - but I tend to lean towards backwards compatibility for an enterprise component like ag-grid😊

jinnyzor · December 23, 2023, 7:49pm

There’s already breaking changes in the next version, so one more is ok. Plus it makes sense to not have to deal with it again. Haha.

At least that’s my thinking.

wowwwn · December 26, 2023, 7:34am

Thank you. I was able to solve it with the hint you gave me.

I used the fact that in the clientside_callback, we can check the information we want in the cellValueChanged EventListener. In the clientside_callback, I implemented a CellValueChange in Javascript and passed the necessary information to the server side for processing using the server function of Flask’s server by posting it to the CellValueChange function, and stored all the cell information that needs to be updated in a global set variable called updated_cell_cache.

app.py:

@server.route("/api/CellValueChange", methods=["POST"])
def CellValueChange():
	data = flask.request.json
	col_id = data['colId']
	new_value = data['value']
	unique_id = data['unique_id']
	global updated_cell_cache
	updated_cell_cache.add((unique_id, col_id, new_value))
	return "ok"



app.clientside_callback(
	"""async function (id) {
		const delay = ms => new Promise(res => setTimeout(res, ms));
		const updateData = (grid) => {
			var datasource = createServerSideDatasource();
			grid.setServerSideDatasource(datasource);
		};

		const onCellValueChanged = async (event) => {
			if (event.type === 'cellValueChanged'){
				try {
					await CellValueChange(event);
				} catch (error) {
					console.error("Error sending cell value change:", error)
				}
				return window.dash_clientside.no_update
			}
		};

		const getGrid = async (id) => {
			let grid;
			let count = 0;
			while (!grid) {
				await delay(200);
				try {
					grid = dash_ag_grid.getApi(id);
				} catch {}
				count++;
				if (count > 20) {
					break;
				}
			}
			return grid;
		};
		
		dash_ag_grid.getApiAsync(id).then((grid) => grid.addEventListener('cellValueChanged', onCellValueChanged));

		const grid = await getGrid(id);
		if (grid) {
			await updateData(grid);
		}
	
		return window.dash_clientside.no_update
	}""",
	Output({"type": "aggrid-table", "index": MATCH}, "id"),
	Input({"type": "aggrid-table", "index": MATCH}, "id"),
)

assets/dag.js

async function CellValueChange(event) {
    try{
        const response = await fetch('./api/CellValueChange', {
            method: 'POST',
            body: JSON.stringify({colId: event['column']['colId'], value: event['value'], unique_id: event['data']['unique_id']}),
            headers: { 'content-type': 'application/json' }
        });
        if (!response.ok) {
            throw new Error("Network response was not ok.")
        }
    } catch (error){
        console.error("Failed to send cell value changed:", error);
    }
}

Then, in the dash_callback called cellrange_edit_update, I was able to fully update my dataframe by retrieving all the updated cell information from the updated_cell_cache.

@app.callback(
	Output({"type": "editable-grid", "index": MATCH}, "value"),
	Input({"type": "aggrid-table", "index": MATCH},
		  "cellValueChanged"),
	State({"type": "aggrid-table", "index": MATCH}, "id"),
	prevent_initial_call=True,
)
def cellrange_edit_update(cell_changed, idx):
	while updated_cell_cache:
		unique_id, col_id, dfId, new_value = updated_cell_cache.pop()

		global df
		df.loc[df['unique_id'] == unique_id, col_id] = new_value
		
	raise exceptions.PreventUpdate()

wowwwn · December 26, 2023, 7:40am

Do you know when the next version of dash_ag_grid will be released?
I’m looking forward to seeing many important features, such as advanced filters, added in the update.

jinnyzor · December 26, 2023, 12:25pm

Good job, that’s what I was thinking of.

And as far as when the next version, depends on documentation.

AnnMarieW · December 27, 2023, 3:53pm

@wowwwn

Be sure to check this section - how updating global variables in a callback can break your app:

wowwwn · December 28, 2023, 1:55am

Thank you for advice. The link you provided was very helpful.
(I am also enjoying “The Book of Dash”)

Since there are many things to consider for multiuser,
considering my situation, I’ll make users to run webserver in there local and use it for there own with 1 worker.
So, what I will do after finish developing my dash app, I will make it to exe binary file(like QT gui app, Im not sure if it possible though…)

Emil · December 29, 2023, 10:45am

I have created a PR to enable the requested functionality. If everything goes well, I hope it will be part of the next release

github.com/plotly/dash-ag-grid

Propagate muti-cell paste event to Dash

plotly:main ← emilhe:cellValueChanged

opened 10:43AM - 29 Dec 23 UTC

emilhe

+100 -62

Currently, the `cellValueChanged` events are propagated to Dash one-by-one. This… becomes a problem in cases, where events are fires faster than Dash can handle them, e.g. when pasting multiple cells into the grid. For these cases, only the first event triggers a Dash callback, with all other changes lost. This PR changes the event collection mechanism to include all events fired within a specific time window (with the length currently defined by `CELL_VALUE_CHANGED_DEBOUNCE_MS = 1`). By making the window large enough that all events in e.g. a multi cell paste operation can be processed (initial tests indicate that 1 ms is sufficient), all changes can be propagted to Dash. Hence the callback will now receieve a _list_ of change events rather than a _single_ change event. Here is a small example that demonstrates the usage, ```python import pandas as pd import dash_ag_grid as dag from dash import Dash, Input, Output, html, callback df = pd.read_csv( "https://raw.githubusercontent.com/plotly/datasets/master/ag-grid/olympic-winners.csv" ) app = Dash(__name__) app.layout = html.Div( [ dag.AgGrid( id="grid", rowData=df.to_dict("records"), columnDefs=[{"field": i} for i in df.columns], defaultColDef={"resizable": True, "sortable": True, "filter": True, "editable": True}, dashGridOptions={ "enableRangeSelection": True, }, enableEnterpriseModules=True, columnSize="sizeToFit", ), html.Div(id="dummy"), ], ) @callback( Output("dummy", "children"), Input("grid", "cellValueChanged") ) def update(cell_changed): return str(cell_changed) if __name__ == "__main__": app.run(debug=True) ``` Copy-pasting (the whole) row 0 to row 4 will yield the following output with the main branch, ```bash {'rowIndex': 4, 'rowId': '4', 'data': {'athlete': 'Michael Phelps', 'age': '23', 'country': 'United States', 'year': '2008', 'date': '24/08/2008', 'sport': 'Swimming', 'gold': '8', 'silver': '0', 'bronze': '0', 'total': '8'}, 'oldValue': 6, 'value': '8', 'colId': 'total', 'timestamp': 1703846436733} ``` With the updates propose in the PR, the output will be, ```bash [{'rowIndex': 4, 'rowId': '4', 'data': {'athlete': 'Michael Phelps', 'age': '23', 'country': 'United States', 'year': '2008', 'date': '24/08/2008', 'sport': 'Swimming', 'gold': '8', 'silver': '0', 'bronze': '0', 'total': '8'}, 'oldValue': 'Aleksey Nemov', 'value': 'Michael Phelps', 'colId': 'athlete', 'timestamp': 1703846303490}, {'rowIndex': 4, 'rowId': '4', 'data': {'athlete': 'Michael Phelps', 'age': '23', 'country': 'United States', 'year': '2008', 'date': '24/08/2008', 'sport': 'Swimming', 'gold': '8', 'silver': '0', 'bronze': '0', 'total': '8'}, 'oldValue': 24, 'value': '23', 'colId': 'age', 'timestamp': 1703846303491}, {'rowIndex': 4, 'rowId': '4', 'data': {'athlete': 'Michael Phelps', 'age': '23', 'country': 'United States', 'year': '2008', 'date': '24/08/2008', 'sport': 'Swimming', 'gold': '8', 'silver': '0', 'bronze': '0', 'total': '8'}, 'oldValue': 'Russia', 'value': 'United States', 'colId': 'country', 'timestamp': 1703846303491}, {'rowIndex': 4, 'rowId': '4', 'data': {'athlete': 'Michael Phelps', 'age': '23', 'country': 'United States', 'year': '2008', 'date': '24/08/2008', 'sport': 'Swimming', 'gold': '8', 'silver': '0', 'bronze': '0', 'total': '8'}, 'oldValue': 2000, 'value': '2008', 'colId': 'year', 'timestamp': 1703846303491}, {'rowIndex': 4, 'rowId': '4', 'data': {'athlete': 'Michael Phelps', 'age': '23', 'country': 'United States', 'year': '2008', 'date': '24/08/2008', 'sport': 'Swimming', 'gold': '8', 'silver': '0', 'bronze': '0', 'total': '8'}, 'oldValue': '1/10/2000', 'value': '24/08/2008', 'colId': 'date', 'timestamp': 1703846303491}, {'rowIndex': 4, 'rowId': '4', 'data': {'athlete': 'Michael Phelps', 'age': '23', 'country': 'United States', 'year': '2008', 'date': '24/08/2008', 'sport': 'Swimming', 'gold': '8', 'silver': '0', 'bronze': '0', 'total': '8'}, 'oldValue': 'Gymnastics', 'value': 'Swimming', 'colId': 'sport', 'timestamp': 1703846303491}, {'rowIndex': 4, 'rowId': '4', 'data': {'athlete': 'Michael Phelps', 'age': '23', 'country': 'United States', 'year': '2008', 'date': '24/08/2008', 'sport': 'Swimming', 'gold': '8', 'silver': '0', 'bronze': '0', 'total': '8'}, 'oldValue': 2, 'value': '8', 'colId': 'gold', 'timestamp': 1703846303491}, {'rowIndex': 4, 'rowId': '4', 'data': {'athlete': 'Michael Phelps', 'age': '23', 'country': 'United States', 'year': '2008', 'date': '24/08/2008', 'sport': 'Swimming', 'gold': '8', 'silver': '0', 'bronze': '0', 'total': '8'}, 'oldValue': 1, 'value': '0', 'colId': 'silver', 'timestamp': 1703846303493}, {'rowIndex': 4, 'rowId': '4', 'data': {'athlete': 'Michael Phelps', 'age': '23', 'country': 'United States', 'year': '2008', 'date': '24/08/2008', 'sport': 'Swimming', 'gold': '8', 'silver': '0', 'bronze': '0', 'total': '8'}, 'oldValue': 3, 'value': '0', 'colId': 'bronze', 'timestamp': 1703846303493}, {'rowIndex': 4, 'rowId': '4', 'data': {'athlete': 'Michael Phelps', 'age': '23', 'country': 'United States', 'year': '2008', 'date': '24/08/2008', 'sport': 'Swimming', 'gold': '8', 'silver': '0', 'bronze': '0', 'total': '8'}, 'oldValue': 6, 'value': '8', 'colId': 'total', 'timestamp': 1703846303493}] ``` Currently, the time window size is hardcoded, but it could also be changed to be a property of the grid so that users get more fine-grained control. Please let me know, if this approach is preferred or not.

Topic		Replies	Views
[AG-Grid] Get cellValueChanged and effected calculated column? Dash Python question	2	117	June 7, 2024
Dash AG Grid: "undoRedoCellEditing" not working after changing rowData in callback Dash Python	14	559	February 22, 2024
Dash AG Grid single cell styling after cell edit Dash Python question	10	2500	April 28, 2024
Dash ag grid cellValueChanged updates one step behind when used with clientside callback Dash Python question	5	800	August 11, 2023
How to get cell value after update Dash Python	2	229	September 15, 2023

Retrieving Information for All Modified Cells in dash_ag_grid

Related topics