df[col].dtype==object reverts back to df[col].dtype==datetime upon df.replace({np.nan: None}) · Issue #44498 · pandas-dev/pandas (original) (raw)

I explicitly coerce the dtype==datetime of a column containing NaT and datetimes (call it: DateTimeCol) to be of type object so that I can convert NaT to None (because Django complains about it). Then later during the workflow, when I have a need to replace all other NaN to None (because Django also compalins about it) using df.replace({np.nan: None}), the DateTimeCol is retransformed back into a type: datetime automatically without it explicit input and the NaT are present again which forces django to complain. Surely this is not a wanted behaviour.

`

pre replace

0 None
1 None
2 None
3 None
4 None
...
163 2013-12-19 16:30:00+01:00
164 2013-12-19 16:30:00+01:00
165 2013-12-19 16:30:00+01:00
166 2013-12-19 16:30:00+01:00
167 2013-12-19 16:30:00+01:00
Name: Tempering DateTime, Length: 168, dtype: object

df.replace({np.nan: None})

post replace

0 NaT
1 NaT
2 NaT
3 NaT
4 NaT
...
163 2013-12-19 16:30:00+01:00
164 2013-12-19 16:30:00+01:00
165 2013-12-19 16:30:00+01:00
166 2013-12-19 16:30:00+01:00
167 2013-12-19 16:30:00+01:00
Name: Tempering DateTime, Length: 168, dtype: datetime64[ns, Europe/Berlin]
`