BUG: Pandas 1.4.0 - pd.NaT can not be replaced. · Issue #45836 · pandas-dev/pandas (original) (raw)

Pandas version checks

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of pandas.
• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

pd.DataFrame([pd.NaT, pd.NaT]).replace({pd.NaT: None, pd.np.NaN: None})

Either pd.NaT or pd.np.NaN work in 1.3.5

Issue Description

In Pandas 1.3.5, the above example returns

  pd.DataFrame([pd.NaT, pd.NaT]).replace({pd.NaT: None, pd.np.NaN: None})
Out[1]: 
      0
0  None
1  None

in pandas 1.4.0, this no longer works

  pd.DataFrame([pd.NaT, pd.NaT]).replace({pd.NaT: None, pd.np.NaN: None})
Out[5]: 
    0
0 NaT
1 NaT

Expected Behavior

.replace() should correctly replace pd.NaT to whatever is specified.

even if we assume pd.NaT is no longer np.NaN - this should still work as pd.NaT is explicitly given.

Installed Versions

Version 1.4.0:

INSTALLED VERSIONS

commit : bb1f651
python : 3.9.7.final.0
python-bits : 64
OS : Linux
OS-release : 5.16.5-arch1-1
Version : #1 SMP PREEMPT Tue, 01 Feb 2022 21:42:50 +0000
machine : x86_64
processor :
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8
pandas : 1.4.0
numpy : 1.22.1
pytz : 2021.3
dateutil : 2.8.2
pip : 22.0.3
setuptools : 57.4.0
Cython : None
pytest : 6.2.5
hypothesis : None
sphinx : None
blosc : 1.10.6
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : 1.0.2
psycopg2 : 2.9.3
jinja2 : 3.0.3
IPython : 7.29.0
pandas_datareader: None
bs4 : None
bottleneck : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : None
numba : None
numexpr : 2.7.3
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : 1.7.3
sqlalchemy : 1.4.31
tables : 3.7.0
tabulate : 0.8.9
xarray : None
xlrd : None
xlwt : None
zstandard : None

Version 1.3.5 (where it's still working):

INSTALLED VERSIONS

commit : 66e3805
python : 3.9.7.final.0
python-bits : 64
OS : Linux
OS-release : 5.16.5-arch1-1
Version : #1 SMP PREEMPT Tue, 01 Feb 2022 21:42:50 +0000
machine : x86_64
processor :
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.3.5
numpy : 1.22.1
pytz : 2021.3
dateutil : 2.8.2
pip : 22.0.3
setuptools : 57.4.0
Cython : None
pytest : 6.2.5
hypothesis : None
sphinx : None
blosc : 1.10.6
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : 1.0.2
psycopg2 : 2.9.3 (dt dec pq3 ext lo64)
jinja2 : 3.0.3
IPython : 7.29.0
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : None
numexpr : 2.7.3
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyxlsb : None
s3fs : None
scipy : 1.7.3
sqlalchemy : 1.4.31
tables : 3.7.0
tabulate : 0.8.9
xarray : None
xlrd : None
xlwt : None
numba : None