BUG: reduction operations failing if min_count is larger · Issue #39738 · pandas-dev/pandas (original) (raw)

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of pandas.
• (optional) I have confirmed this bug exists on the master branch of pandas.

Code Sample, a copy-pastable example

import pandas as pd df = pd.DataFrame({"x": [1, 2, 3], "y": [4, 5, 6]}) df x y 0 1 4 1 2 5 2 3 6 df.sum(min_count=10) Traceback (most recent call last): File "", line 1, in File "/nvme/0/pgali/envs/cudfdev/lib/python3.7/site-packages/pandas/core/generic.py", line 11067, in sum self, axis, skipna, level, numeric_only, min_count, **kwargs File "/nvme/0/pgali/envs/cudfdev/lib/python3.7/site-packages/pandas/core/generic.py", line 10787, in sum "sum", nanops.nansum, axis, skipna, level, numeric_only, min_count, **kwargs File "/nvme/0/pgali/envs/cudfdev/lib/python3.7/site-packages/pandas/core/generic.py", line 10774, in _min_count_stat_function min_count=min_count, File "/nvme/0/pgali/envs/cudfdev/lib/python3.7/site-packages/pandas/core/frame.py", line 8847, in _reduce res, indexer = df._mgr.reduce(blk_func, ignore_failures=ignore_failures) File "/nvme/0/pgali/envs/cudfdev/lib/python3.7/site-packages/pandas/core/internals/managers.py", line 354, in reduce nbs = blk.reduce(func, ignore_failures) File "/nvme/0/pgali/envs/cudfdev/lib/python3.7/site-packages/pandas/core/internals/blocks.py", line 400, in reduce nb = self.make_block(res_values) File "/nvme/0/pgali/envs/cudfdev/lib/python3.7/site-packages/pandas/core/internals/blocks.py", line 286, in make_block return make_block(values, placement=placement, ndim=self.ndim) File "/nvme/0/pgali/envs/cudfdev/lib/python3.7/site-packages/pandas/core/internals/blocks.py", line 2732, in make_block return klass(values, ndim=ndim, placement=placement) File "/nvme/0/pgali/envs/cudfdev/lib/python3.7/site-packages/pandas/core/internals/blocks.py", line 143, in init f"Wrong number of items passed {len(self.values)}, " ValueError: Wrong number of items passed 1, placement implies 2

This actually works in 1.1.5:

df x y 0 1 4 1 2 5 2 3 6 df.sum(min_count=10) x NaN y NaN dtype: float64

Problem description

Reduction operations like sum, prod, etc.. are failing when min_count is larger in latest pandas, whereas this used to work in 1.1.5.

Expected Output

df x y 0 1 4 1 2 5 2 3 6 df.sum(min_count=10) x NaN y NaN dtype: float64

Output of pd.show_versions()

INSTALLED VERSIONS

commit : 9d598a5
python : 3.7.9.final.0
python-bits : 64
OS : Linux
OS-release : 4.15.0-76-generic
Version : #86-Ubuntu SMP Fri Jan 17 17:24:28 UTC 2020
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.2.1
numpy : 1.19.5
pytz : 2021.1
dateutil : 2.8.1
pip : 21.0.1
setuptools : 49.6.0.post20210108
Cython : 0.29.21
pytest : 6.2.2
hypothesis : 6.1.1
sphinx : 3.4.3
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.11.3
IPython : 7.20.0
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : 0.8.5
fastparquet : None
gcsfs : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 1.0.1
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : 0.52.0