Issue with px.timeline() datetime formats

bphillip · August 10, 2020, 7:42pm

I am currently at a loss as to how to solve this date input problem with px.timeline() gantt charts. I input two dt64 objects for x_start and x_stop but start shows up in iso format and the other as normal formatted datetime. They also seem to overlap as if they are plotting based on a different timezone. I have tried stripping timezone to no avail and when I throw them through strfmt, the end date converts to epochtime and there is nothing I can do to prevent that. I am not sure if I am doing something incorrectly or is there is a bug.

Screen Shot 2020-08-10 at 3.40.28 PM
Screen Shot 2020-08-10 at 3.20.04 PM

PRINT SERIES OUTPUT

[605 rows x 7 columns]
0     2020-04-06 22:17:00
1     2020-04-06 22:17:00
2     2020-04-07 02:15:00
3     2020-04-07 02:15:00
4     2020-04-07 02:15:00
              ...        
602   2020-04-24 12:27:02
601   2020-04-24 12:27:02
604   2020-04-27 01:00:00
603   2020-04-27 01:00:00
533   2020-04-22 17:49:00
Name: u_impact_start, Length: 629, dtype: datetime64[ns]
0     2020-04-07 01:29:00
1     2020-04-07 01:29:00
2     2020-04-07 04:00:00
3     2020-04-07 04:00:00
4     2020-04-07 04:00:00
              ...        
602   2020-04-26 10:17:00
601   2020-04-26 10:17:00
604   2020-04-27 02:00:00
603   2020-04-27 02:00:00
533   2020-05-06 19:57:00
Name: u_impact_stop, Length: 629, dtype: datetime64[ns]

nicolaskruchten · August 10, 2020, 8:53pm

Can you provide a small running example we can reproduce please? I’d be glad to try to help

bphillip · August 10, 2020, 9:39pm

Sure thing. It might not be ready until tomorrow though as it it currently bedtime routine for the kids! Thanks

bphillip · August 11, 2020, 3:45pm

Here is a small test case that shows what I am experiencing. The below example shows the start and stop times in hover as two different formats even though they are the same format inside the dataframe.

data = {'u_impact_start': ['2020-04-06 22:17:00', ' 2020-04-06 22:17:00', '2020-04-07 02:15:00', '2020-04-07 02:15:00', '2020-04-07 02:15:00'],
            'u_impact_stop': ['2020-04-07 01:29:00', '2020-04-07 01:29:00', '2020-04-07 04:00:00', '2020-04-07 04:00:00', '2020-04-07 04:00:00'],
            'ci_item': ['test_1', 'test_2', 'test_3', 'test_4', 'test_5'],
            'failed_out': [True, False, False, True, False]
            }
    test_df = pd.DataFrame(data)

    fig = px.timeline(test_df,
                      title='Impact Timeline',
                      x_start='u_impact_start',
                      x_end='u_impact_stop',
                      y='ci_item',
                      color='failed_out',
                      color_discrete_map={True: 'mediumseagreen', False: 'FireBrick'},
                      )

If I convert the column to a datetime object, it shows up as ISO format on hover (like the original code).

    test_df['u_impact_start'] = pd.to_datetime(test_df['u_impact_start'])

If I then try to convert either string date or datetime to a format I would like to see with strfmt, it converts the stop times to epoch.

    test_df['u_impact_start'] = pd.DatetimeIndex(
        test_df['u_impact_start']).strftime("%b %d %H:%M:%S %Y")

or

test_df['u_impact_start'] = [datetime.fromisoformat(
        str(y)) for y in test_df['u_impact_start']]

If I convert both start and stop to datetime, start time is ISO format and stop time is not. Converting both to datetime and then running strfmt doesn’t help either. I have tried several ways to strip out iso formatting because it is plotting based on UTC which I don’t want to no avail. I am at a loss at this point and any help is appreciated.

nicolaskruchten · August 11, 2020, 5:57pm

Thanks!

It’s a bit awkward at the moment but you can format both the start and end date in the hoverlabel like this (ideally you’d be able to do both from within hover_data but that’s not possible at the moment unfortunately):

data = {'u_impact_start': ['2020-04-06 22:17:00', ' 2020-04-06 22:17:00', '2020-04-07 02:15:00', '2020-04-07 02:15:00', '2020-04-07 02:15:00'],
            'u_impact_stop': ['2020-04-07 01:29:00', '2020-04-07 01:29:00', '2020-04-07 04:00:00', '2020-04-07 04:00:00', '2020-04-07 04:00:00'],
            'ci_item': ['test_1', 'test_2', 'test_3', 'test_4', 'test_5'],
            'failed_out': [True, False, False, True, False]
            }
test_df = pd.DataFrame(data)

desired_format = "%m %d %H:%M"

fig = px.timeline(test_df,
                  title='Impact Timeline',
                  x_start='u_impact_start',
                  x_end='u_impact_stop',
                  y='ci_item',
                  color='failed_out',
                  color_discrete_map={True: 'mediumseagreen', False: 'FireBrick'},
                  hover_data=dict(
                      u_impact_start="|" + desired_format
                  )
                  )
fig.update_xaxes(tickformat=desired_format)
fig.show()

I’m not sure what issues you’re hitting with respect to the timezone, but there is a known problem at the moment with bars less than 10 seconds long, which should be resolved in the next version which should come out in the next 10 days or so. If you can provide an example of “They also seem to overlap as if they are plotting based on a different timezone” I’d be happy to take a look!

bphillip · August 11, 2020, 7:13pm

Sure, the example I provided in my previous post shows what is happening. The x_start is actually being plotted after the x_stop because it seems to be taking the x_start as UTC. I suppose I could modify x_stop dates to account for timezones to account for this. It leaves the question though, why does it handle x_start this way but not x_stop? Thanks again!

[edit] After closer inspection, it seems to be converting the x_stop according to my timezone (-5) but not x_start.

test_df:
         u_impact_start        u_impact_stop ci_item  failed_out
0   2020-04-06 22:17:00  2020-04-07 01:29:00  test_1        True
1   2020-04-06 22:17:00  2020-04-07 01:29:00  test_2       False
2   2020-04-07 02:15:00  2020-04-07 04:00:00  test_3       False
3   2020-04-07 02:15:00  2020-04-07 04:00:00  test_4        True
4   2020-04-07 02:15:00  2020-04-07 04:00:00  test_5       False

nicolaskruchten · August 11, 2020, 7:30pm

Ah, I hadn’t noticed that, I was looking at the hour portion of the date and not the date portion. That is disconcerting indeed and smells like a bug to me. I’ll dig into it.

nicolaskruchten · August 11, 2020, 7:33pm

Interestingly, this doesn’t happen locally for me. One way to get into this situation would be to be using plotly version 4.9 in JupyterLab and to have a version of the jupyterlab_plotly extension that’s older than 4.9.0… Does that match your setup? If you’re working within a Dash context, it’s also possible to get into this situation if you have an older version of dash_core_components installed.

bphillip · August 11, 2020, 7:34pm

I will try upgrading them now but here is my config.

Dash version 1.12.0
dash_table version 4.7.0
Plotly version 4.9.0
dcc version 1.10.0
html version 1.0.3
dbc version 0.10.2

bphillip · August 11, 2020, 7:40pm

Ha, sometimes the simplest solution is the correct one. Thank you very much sir.

nicolaskruchten · August 11, 2020, 7:43pm

Yep, that’s the explanation: dcc 1.10.0 doesn’t include the bugfix which makes px.timeline() work in a browser/locale-independent way. Sorry it took so many iterations for me to recognize this and recommend that as a fix.

I’ll see what I can do about enabling the formatting of both x_start and x_end via hover_data… the asymmetry is frustrating

nicolaskruchten · August 11, 2020, 7:51pm

Follow-up issue: https://github.com/plotly/plotly.js/issues/5066

joel · September 18, 2020, 9:03pm

Thanks to both of you!

I am migrating from create_gantt to timeline and this discussion fixed my issue!

Lulu · January 6, 2021, 2:41pm

Hello Nicolas

i wish you the best for 2021,

I am trying to resolve the date format issue on the Hover; in fact the “Finish” date format displays a series of numbers .
The columns in my dataframe are both formatted the same way.

<class 'pandas.core.frame.DataFrame'>
Int64Index: 18164 entries, 3106 to 353
Data columns (total 13 columns):
 #   Column                    Non-Null Count  Dtype         
---  ------                    --------------  -----         
 0   Num_PTP                   18071 non-null  object        
 1   Nom_Technicien            5144 non-null   object        
 2   Nom_Chef_de_Projet        12125 non-null  object        
 3   Num_Echantillon/Nb_bouts  18041 non-null  object        
 4   Nom_Opérations            17758 non-null  object        
 5   Outils                    18164 non-null  object        
 6   Done                      18164 non-null  float64       
 7   Date_Début                18164 non-null  datetime64[ns]
 8   Date_Fin                  18164 non-null  datetime64[ns]
 9   Commentaires              18164 non-null  object        
 10  Op_Pise                   18100 non-null  object        
 11  Durée                     18164 non-null  float64       
 12  Opérations                18119 non-null  object        
dtypes: datetime64[ns](2), float64(2), object(9)
memory usage: 1.9+ MB

formatdatefinish

the following version are installed
dash 1.17.0
dash-core-components 1.13.0
plotly 4.13.0

I am not sure if I am doing something incorrectly or is there is a bug.

nicolaskruchten · January 11, 2021, 9:56pm

Hello @Lulu and happy new year to you also!

Can you please share the px.timeline() call you’re making that’s giving you this output?

Lulu · January 12, 2021, 8:22am

Hello Nicolas please find below the px.timeline

dfopgantt = df1op[['Nom_Opérations', 'Date_Début', 'Date_Fin',
                                      'Opérations', 'Outils', 'Num_Echantillon/Nb_bouts', 'Commentaires', 'Durée']]


            dfopgantt.columns = ['Task', 'Start', 'Finish',
                                 'Resource', 'Tools', 'QtyRéf', 'Comments', 'Duration']

                #try:
            Gantt=px.timeline ( dfopgantt ,  x_start = "Start" ,  x_end = "Finish" ,y='Task',    color = 'Resource' , 
            hover_data={'Task','Start','Finish','Resource','Tools','QtyRéf','Comments','Duration'},color_discrete_map=colors)                       

            Gantt.update_yaxes(autorange="reversed")
            Gantt.update_layout(height=700,yaxis_title="",paper_bgcolor='rgba(23,41,131,0.0)')#,plot_bgcolor='rgba(255,255,255,1)')

            return Gantt

nicolaskruchten · January 12, 2021, 1:05pm

Thanks @Lulu, I’m pretty sure your issue is well-captured by px.timeline loses type of x_end values · Issue #2934 · plotly/plotly.py · GitHub, which I will try to fix soon.

nicolaskruchten · January 12, 2021, 5:02pm

OK, try it again with version 4.14.3 … it should behave better now

Lulu · January 14, 2021, 2:02pm

Thanks a lot for your support @nicolaskruchten ; that works well now !!

see ya