BUG: idxmin and idxmax fail for groupby of decimal columns · Issue #40685 · pandas-dev/pandas (original) (raw)

Code Sample, a copy-pastable example

Your code here

import pandas as pd from decimal import Decimal

from pandas.tests.extension.decimal import DecimalArray

df = pd.DataFrame({ 'idx': [0, 1], 'x': [Decimal('8.68'), Decimal('42.23')], 'y': [Decimal('7.11'), Decimal('79.61')], # 'x': DecimalArray([Decimal('8.68'), Decimal('42.23')]), # 'y': DecimalArray([Decimal('7.11'), Decimal('79.61')]), })

print(df.groupby("idx", sort=True).agg('idxmin'))

Empty DataFrame

Columns: []

Index: [0, 1]

Problem description

When a column contains Decimal objects, the idxmin and idxmax aggregations return empty output.

Expected Output

The expected output is

If I uncomment the commented lines in the above example (i.e. place the Decimal objects inside a DecimalArray), it works, but AFAICT that isn't really part of the intended public pandas API. Moreover, the issue is not exclusively due to singleton groups: I can also reproduce it using the following DataFrame:

df = pd.DataFrame({
    'idx': [0, 1, 0, 1],
    'x': [Decimal('8.68'), Decimal('42.23'), Decimal('8.69'), Decimal('42.24')],
    'y': [Decimal('7.11'), Decimal('79.61'), Decimal('7.12'), Decimal('79.62')],
})

I don't think this is related to #39098 because this only occurs for idxmin or idxmax, not aggregations like sum. That is the only obviously related issue I could find.

Output of pd.show_versions()

I've tested on two separate systems.

Docker Ubuntu container (running on a host Ubuntu machine)

INSTALLED VERSIONS

commit : f2c8480
python : 3.7.10.final.0
python-bits : 64
OS : Linux
OS-release : 4.15.0-76-generic
Version : #86-Ubuntu SMP Fri Jan 17 17:24:28 UTC 2020
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : None
LOCALE : en_US.UTF-8

pandas : 1.2.3
numpy : 1.20.1
pytz : 2021.1
dateutil : 2.8.1
pip : 21.0.1
setuptools : 52.0.0.post20210125
Cython : 0.29.22
pytest : 6.2.2
hypothesis : 6.3.4
sphinx : 3.5.1
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.11.3
IPython : 7.21.0
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : 0.8.7
fastparquet : None
gcsfs : None
matplotlib : 3.3.4
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 1.0.1
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : 0.53.0

conda Python on a Mac

INSTALLED VERSIONS

commit : f2c8480
python : 3.9.2.final.0
python-bits : 64
OS : Darwin
OS-release : 19.6.0
Version : Darwin Kernel Version 19.6.0: Tue Jan 12 22:13:05 PST 2021; root:xnu-6153.141.16~1/RELEASE_X86_64
machine : x86_64
processor : i386
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.2.3
numpy : 1.20.1
pytz : 2021.1
dateutil : 2.8.1
pip : 21.0.1
setuptools : 49.6.0.post20210108
Cython : 0.29.22
pytest : 6.2.2
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.11.3
IPython : 7.21.0
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : None