Table Date filtering not working

Hello,

I am defining a table from a data frame. I have a date field in that df that I need to show it and be able to filter it (equals, in range, etc.)

in my column definition, I coded the below:
{
“field”: “Timestamp”,
“headerName”: “Timestamp”,
“filter”: “agDateColumnFilter”,
“valueFormatter”: {“function”: f"d3.timeParse(‘%d/%m/%Y)(params.data[“Timestamp”])"},
“valueGetter”: {“function”: f"d3.timeParse(’%d/%m/%Y)(params.data[“Timestamp”])"},
}

The dates are listed as expected. but when filtering, if I put equals, it doesn’t filter; if I put a range, the dates disappear.

Your help highly appreciated.

Thanks

1 Like

Hi @gmouawad and welcome to the Dash community :slight_smile:

Note that for the date filter to work, it must be a date only and if there is a time portion, it must be zero. For more information see:

And the last 2 examples on this page:

Check the syntax of the function - it looks like you are missing a quote:

{“function”: f"d3.timeParse(‘%d/%m/%Y)(params.data[“Timestamp”])"},

should be:

{“function”:  "d3.timeParse('%d/%m/%Y')(params.data.Timestamp)"},

Hello, thanks a lot @AnnMarieW.

in the below example, even with time portion including hours, date filter is working:

from dash import Dash, dcc, html
import dash_ag_grid as dag
from datetime import datetime

rowData = [
    {"date": datetime(2023, 1, 1, 1, 0, 0)},
    {"date": datetime(2023, 1, 2, 2, 0, 0)},
    {"date": datetime(2023, 1, 3, 3, 0, 0)},
    {"date": datetime(2023, 1, 4, 18, 0, 0)},
    {"date": datetime(2023, 1, 5, 22, 0, 0)},
]

# function to create a date object from  a date string "YYYY-MM-DD"
date_obj = "d3.timeParse('%Y-%m-%dT%H:%M:%S')(params.data.date)"

# if the time is in utc:
# date_obj = "d3.utcParse('%Y-%m-%dT%H:%M:%S')(params.data.date)"


columnDefs = [
    {
        "headerName": "Datetime string",
        "field": "date",
        "filter": False,
    },
    {
        "headerName": "MM/DD/YYYY",
        "valueGetter": {"function": date_obj},
        "valueFormatter": {"function": f"d3.timeFormat('%m/%d/%Y')({date_obj})"},
    },
    {
        "headerName": "Mon DD, YYYY",
        "valueGetter": {"function": date_obj},
        "valueFormatter": {"function": f"d3.timeFormat('%b %d, %Y')({date_obj})"},
    },
    {
        "headerName": "day, Mon DD, YYYY",
        "valueGetter": {"function": date_obj},
        "valueFormatter": {"function": f"d3.timeFormat('%a %b %d, %Y')({date_obj})"},
    },
    {
        "headerName": "yyyy-mm-dd HH:MM:SS tt",
        "valueGetter": {"function": date_obj},
        "valueFormatter": {
            "function": f"d3.timeFormat('%Y-%m-%d %I:%M:%S %p')({date_obj})"
        },
    },
    {
        "headerName": "yyyy-mm-dd hh:mm:ss",
        "valueGetter": {"function": date_obj},
        "valueFormatter": {
            "function": f"d3.timeFormat('%Y-%m-%d %H:%M:%S')({date_obj})"
        },
    },
]

defaultColDef = {
    "filter": "agDateColumnFilter",
    "filterParams": {
        "buttons": ["clear", "apply"],
    },
    "sortable": True,
}


app = Dash(__name__)

app.layout = html.Div(
    [
        dcc.Markdown("Datetime formatting example.  Note - The date filter will not work with datetime.  Time must be zero (i.e. 2023-01-01T00:00:00)"),
        dag.AgGrid(id="date-times-example-2", columnDefs=columnDefs, rowData=rowData, defaultColDef=defaultColDef),
    ],
    style={"margin": 20},
)


if __name__ == "__main__":
    app.run(debug=True)

Reference: D3 Value Formatters | Dash for Python Documentation | Plotly

Changing my code to:

{“function”: “d3.timeParse(‘%d/%m/%Y’)(params.data.Timestamp)”},

the table shows:
image

The example you posted has a time included with the date so the date filter won’t work. Try running it with this data:



rowData = [
    {"date": datetime(2023, 1, 1, 0, 0, 0)},
    {"date": datetime(2023, 1, 2, 0, 0, 0)},
    {"date": datetime(2023, 1, 3, 0, 0, 0)},
    {"date": datetime(2023, 1, 4, 0, 0, 0)},
    {"date": datetime(2023, 1, 5, 0, 0, 0)},
]

Note - to format the code click the </> icon in the editor or enclose it in three backticks

dear, with the time included, it is working. it is weird

dear, the time can be included as long as it’s zero

understood but in the above set, time is not zero and filtering is working.

in my code data frame (not the above) I set time to zero and still it is not filtering.
Note that I am using it under a mercury app, if that can make things clearer.

Thanks

That is odd, because the filter with a time doesn’t work in the docs and it doesn’t work when I run it locally. :woman_shrugging:

Here it is in the docs - if you filter for 01/02/2023 it does not work:

date-error


Here’s the app I posted above where the time is set to zero. You can see that the date filter now works

date-error2

What is a mercury app?

Can you post a complete minimal example with some sample data that reproduces the filter not working?

Hello @AnnMarieW and @gmouawad,

When using the date filter, it automatically selects midnight in the time aspect, therefore, when you use equals, it will only show responses for the exact timestamp of 01/02/2023 or 2023-01-02T00:00:00 as these points dont exist, then you’ll need to adjust this.

Example:

If order to allow for only filtering of the date, I’d recommend creating your own date filter, something like this:

dagfuncs.cleanDateColumn = (cellValue) => {
    if (cellValue === 'NaT') {
        returnValue = ''
    } else {
        returnValue = cellValue.includes('T') ? cellValue.split('T')[0] : cellValue
    }
    return returnValue;
}

dagfuncs.DateComparator = (filterLocalDateAtMidnight, cellValue) => {
    // converts timestamp to just date for easier user filtering
     const dateAsString = cellValue ? dagfuncs.cleanDateColumn(cellValue): null;
     if (dateAsString == null) {
         return 0;
     }

     // convert date string to date
     var tempDate = new Date(dateAsString);

     // convert dates to disregard timezones
     var userTimezoneOffset = tempDate.getTimezoneOffset() * 60000;
     const cellDate = new Date(tempDate.getTime() + userTimezoneOffset);

     // Now that both parameters are Date objects, we can compare
     if (cellDate < filterLocalDateAtMidnight) {
         return -1;
     } else if (cellDate > filterLocalDateAtMidnight) {
         return 1;
     }
     return 0;
 }

To place in your code, you’d add these:

'filter': 'agDateColumnFilter', 'filterParams': {"comparator": {"function": "DateComparator"}}


You also dont need to convert the value getter to the date object as we take care of that inside the cleanDateColumn function:

from dash import Dash, dcc, html
import dash_ag_grid as dag
from datetime import datetime

rowData = [
    {"date": datetime(2023, 1, 1, 1, 0, 0)},
    {"date": datetime(2023, 1, 2, 2, 0, 0)},
    {"date": datetime(2023, 1, 3, 3, 0, 0)},
    {"date": datetime(2023, 1, 4, 18, 0, 0)},
    {"date": datetime(2023, 1, 5, 22, 0, 0)},
]

# function to create a date object from  a date string "YYYY-MM-DD"
date_obj = "d3.timeParse('%Y-%m-%dT%H:%M:%S')(params.data.date)"

# if the time is in utc:
# date_obj = "d3.utcParse('%Y-%m-%dT%H:%M:%S')(params.data.date)"


columnDefs = [
    {
        "headerName": "Datetime string",
        "field": "date",
        "filter": False,
    },
    {
        "headerName": "MM/DD/YYYY",
        "valueGetter": {"function": 'params.data.date'},
        "valueFormatter": {"function": f"d3.timeFormat('%m/%d/%Y')({date_obj})"},
    },
    {
        "headerName": "Mon DD, YYYY",
        "valueGetter": {"function": 'params.data.date'},
        "valueFormatter": {"function": f"d3.timeFormat('%b %d, %Y')({date_obj})"},
    },
    {
        "headerName": "day, Mon DD, YYYY",
        "valueGetter": {"function": 'params.data.date'},
        "valueFormatter": {"function": f"d3.timeFormat('%a %b %d, %Y')({date_obj})"},
    },
    {
        "headerName": "yyyy-mm-dd HH:MM:SS tt",
        "valueGetter": {"function": 'params.data.date'},
        "valueFormatter": {
            "function": f"d3.timeFormat('%Y-%m-%d %H:%M:%S %p')({date_obj})"
        },
    },
    {
        "headerName": "yyyy-mm-dd hh:mm:ss",
        "valueGetter": {"function": 'params.data.date'},
        "valueFormatter": {
            "function": f"d3.timeFormat('%Y-%m-%d %H:%M:%S')({date_obj})"
        },
    },
]

defaultColDef = {
    "filter": "agDateColumnFilter",
    "filterParams": {
        "buttons": ["clear", "apply"],
        "comparator": {"function": "DateComparator"}
    },
    "sortable": True,
}


app = Dash(__name__)

app.layout = html.Div(
    [
        dcc.Markdown("Datetime formatting example.  Note - The date filter will not work with datetime.  Time must be zero (i.e. 2023-01-01T00:00:00)"),
        dag.AgGrid(id="date-times-example-2", columnDefs=columnDefs, rowData=rowData, defaultColDef=defaultColDef),
    ],
    style={"margin": 20},
)


if __name__ == "__main__":
    app.run(debug=True)

Hello,

You can check mercury on the following link: https://runmercury.com/

Regarding the code example:

#history list definition
historyList = []

#in a for loop, create an auditItem and add timestamp. property and value to it from an api response. Then add the auditItem to the historyList
auditItem = {}
auditItem["Timestamp"] = datetime.datetime(realDateTime.year, realDateTime.month, realDateTime.day, 0, 0, 0)
auditItem["Property"] = str(auditName)
auditItem["Value"] = str(value)
historyList.append(auditItem)

# dash section
historydf = pd.DataFrame(historyList)

app = Dash(__name__)

# basic columns definition with column defaults
columnDefs = [
    {
        "field": "Timestamp",
        "headerName": "Timestamp",
        "filter": "agDateColumnFilter",
        "valueFormatter": {"function": 'params.data.Timestamp'},
        "valueGetter": {"function": 'params.data.Timestamp'},
    },
    {"field": "Property"},
    {"field": "Value"},
]

app.layout = html.Div(
    [
        dag.AgGrid(
            id="history_aggrid",
            columnDefs=columnDefs,
            rowData=historydf.to_dict("records"),
            columnSize="sizeToFit",
            defaultColDef={"resizable": True, "sortable": True, "filter": True},
        ),
    ],
    style={"margin": 0,
        "width": 750}
)

if __name__ == "__main__":
    app.run(debug=True)

With this code, the filter is not working.

Hello @gmouawad,

Please try my comparator and adjust your code accordingly.

Assuming your timestamp has the split of T, it should work.

Hello, in which language your comparators are written? I don’t think a python code, right?

It is in JavaScript, that’s what the filter works in.

You need to have it in an assets folder file, check out some of the examples with custom Js functions in the grid documents.

1 Like

Thank you for the hint. It is working now

1 Like

Hi,

I have tried many tricks and the example code you gave. Still the filter doesnt work, it filters nothing if I set it to equal of anything and it filters the whole table empty, if I use Less than or Greater than filters. The javascript file is in the assets folder of the root directory of the python file.

I altered the python file by setting
rowData = [
{“date”: datetime(2023, 1, 1, 0, 0, 0)},
{“date”: datetime(2023, 1, 2, 0, 0, 0)},
{“date”: datetime(2023, 1, 3, 0, 0, 0)},
{“date”: datetime(2023, 1, 4, 0, 0, 0)},
{“date”: datetime(2023, 1, 5, 0, 0, 0)},
]
And adding dagfuncs to DateComparator:

defaultColDef = {
“filter”: “agDateColumnFilter”,
“filterParams”: {
“buttons”: [“clear”, “apply”],
“comparator”: {“function”: “dagfuncs.DateComparator”}
},
“sortable”: True,
}

Also added var dagfuncs = dagfuncs || {};
To the start of the JS file.

Even with these modifications I cant get it to work properly.

I am out of ideas. Please help.

Hello @PlotlyDude,

It is just DateComparator, not dagfuncs.DateComparator, this is in the Python code.

Hi and thanks for reply.

Im a noob and dont understand sorry. The javascript you gave has dagfuncs before the function, which implies that it has as a namespace that needs to be referenced to? Anyways, I tried DateComparator aswel, but it doesnt work. and I thought the python code was missing the dagfuncs.

@PlotlyDude,

This is true, but the namespace doesn’t need to be referenced in the Python. Can you please post more of how you are trying to do it?

Im just trying to get your example to work that you provided. So here is the python code you gave:

from dash import Dash, dcc, html
import dash_ag_grid as dag
from datetime import datetime

rowData = [
    {"date": datetime(2023, 1, 1, 0, 0, 0)}, ## Changed hours to 0
    {"date": datetime(2023, 1, 2, 0, 0, 0)},
    {"date": datetime(2023, 1, 3, 0, 0, 0)},
    {"date": datetime(2023, 1, 4, 0, 0, 0)},
    {"date": datetime(2023, 1, 5, 0, 0, 0)},
]

# function to create a date object from  a date string "YYYY-MM-DD"
date_obj = "d3.timeParse('%Y-%m-%dT%H:%M:%S')(params.data.date)"

# if the time is in utc:
#date_obj = "d3.utcParse('%Y-%m-%dT%H:%M:%S')(params.data.date)"


columnDefs = [
    {
        "headerName": "Datetime string",
        "field": "date",
        "filter": False,
    },
    {
        "headerName": "MM/DD/YYYY",
        "valueGetter": {"function": 'params.data.date'},
        "valueFormatter": {"function": f"d3.timeFormat('%m/%d/%Y')({date_obj})"},
    },
    {
        "headerName": "Mon DD, YYYY",
        "valueGetter": {"function": 'params.data.date'},
        "valueFormatter": {"function": f"d3.timeFormat('%b %d, %Y')({date_obj})"},
    },
    {
        "headerName": "day, Mon DD, YYYY",
        "valueGetter": {"function": 'params.data.date'},
        "valueFormatter": {"function": f"d3.timeFormat('%a %b %d, %Y')({date_obj})"},
    },
    {
        "headerName": "yyyy-mm-dd HH:MM:SS tt",
        "valueGetter": {"function": 'params.data.date'},
        "valueFormatter": {
            "function": f"d3.timeFormat('%Y-%m-%d %H:%M:%S %p')({date_obj})"
        },
    },
    {
        "headerName": "yyyy-mm-dd hh:mm:ss",
        "valueGetter": {"function": 'params.data.date'},
        "valueFormatter": {
            "function": f"d3.timeFormat('%Y-%m-%d %H:%M:%S')({date_obj})"
        },
    },
]

defaultColDef = {
    "filter": "agDateColumnFilter",
    "filterParams": {
        "buttons": ["clear", "apply"],
        "comparator": {"function": "DateComparator"} ## Here I tried to add dagfuncs. to no success
    },
    "sortable": True,
}


app = Dash(__name__)

app.layout = html.Div(
    [
        dcc.Markdown("Datetime formatting example.  Note - The date filter will not work with datetime.  Time must be zero (i.e. 2023-01-01T00:00:00)"),
        dag.AgGrid(id="date-times-example-2", columnDefs=columnDefs, rowData=rowData, defaultColDef=defaultColDef),
    ],
    style={"margin": 20},
)


if __name__ == "__main__":
    app.run(debug=True)

Here is the JS you provided. Its in the assets folder and is loaded correctly to the app. I added var dagfuncs = dagfuncs ||{}; to the beginning so it would stop giving errors in console.


dagfuncs.cleanDateColumn = (cellValue) => {
    if (cellValue === 'NaT') {
        returnValue = ''
    } else {
        returnValue = cellValue.includes('T') ? cellValue.split('T')[0] : cellValue
    }
    return returnValue;
}

dagfuncs.DateComparator = (filterLocalDateAtMidnight, cellValue) => {
    // converts timestamp to just date for easier user filtering
     const dateAsString = cellValue ? dagfuncs.cleanDateColumn(cellValue): null;
     if (dateAsString == null) {
         return 0;
     }

     // convert date string to date
     var tempDate = new Date(dateAsString);

     // convert dates to disregard timezones
     var userTimezoneOffset = tempDate.getTimezoneOffset() * 60000;
     const cellDate = new Date(tempDate.getTime() + userTimezoneOffset);

     // Now that both parameters are Date objects, we can compare
     if (cellDate < filterLocalDateAtMidnight) {
         return -1;
     } else if (cellDate > filterLocalDateAtMidnight) {
         return 1;
     }
     return 0;
 }

Here in the image what it looks like when I filter Less than 14/01/2024:

@PlotlyDude,

It needs to be this:

var dagfuncs = window.dashAgGridFunctions = window.dashAgGridFunctions || {};