pd.Series.sum() returns nan for Int64 with pd.NA · Issue #30958 · pandas-dev/pandas (original) (raw)

Code Sample, a copy-pastable example if possible

I'm not sure this is a bug or intended behavior. If intended, I'm glad to be pointed to how API design was discussed and decided.

On 1.0.0 rc0, pd.Series.sum()returns nan when using skipna=False for 'Int64' with pd.NA.

pd.Series([1, 2, pd.NA]).sum(skipna=False) pd.Series([1, 2, pd.NA], dtype='Int64').sum(skipna=False) nan

pd.Series([1, 2, np.nan]).sum(skipna=False) nan pd.Series([1, 2, np.nan], dtype='Int64').sum(skipna=False) nan

pd.Series([1, 2, None]).sum(skipna=False) nan pd.Series([1, 2, None], dtype='Int64').sum(skipna=False) nan

Problem description

I understand the second and third example returns nan for backward compatibility, but the first example of dtype='Int64' seems natural to return pd.NA.

Expected Output

pd.Series([1, 2, pd.NA], dtype='Int64').sum(skipna=False)

When breaking backward compatibility is allowed, this is also expected (on and after pandas 2.0.0?)

pd.Series([1, 2, np.nan], dtype='Int64').sum(skipna=False) pd.Series([1, 2, None], dtype='Int64').sum(skipna=False)

Output of pd.show_versions()

>>> pd.show_versions()

INSTALLED VERSIONS

commit : None
python : 3.7.6.final.0
python-bits : 64
OS : Linux
OS-release : 4.9.184-linuxkit
machine : x86_64
processor :
byteorder : little
LC_ALL : None
LANG : C.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.0.0rc0
numpy : 1.18.1
pytz : 2019.3
dateutil : 2.8.1
pip : 19.3.1
setuptools : 44.0.0
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
fastparquet : None
gcsfs : None
lxml.etree : None
matplotlib : 3.1.2
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
pytest : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
xlsxwriter : None
numba : None

Dockerfile

FROM python:3.7.6 WORKDIR /home RUN pip install pandas==1.0.0rc0 CMD ["/bin/bash"]