pandas.core.groupby.GroupBy.apply fails · Issue #20949 · pandas-dev/pandas (original) (raw)

Code Sample:

df = pd.DataFrame({'A': 'a a b'.split(), 'B': [1,2,3], 'C': [4,6, 5]}) g = df.groupby('A') g.apply(lambda x: x / x.sum())

Problem description

Applying a function to a grouped data frame fails. The code above is the example code from the official pandas documentation: https://pandas.pydata.org/pandas-docs/stable/generated/pandas.core.groupby.GroupBy.apply.html

Output to the above code:

/usr/local/lib/python2.7/dist-packages/pandas/core/computation/check.py:17: UserWarning: The installed version of numexpr 2.4.3 is not supported in pandas and will be not be used The minimum supported version is 2.4.6

ver=ver, min_ver=_MIN_NUMEXPR_VERSION), UserWarning) Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.7/dist-packages/pandas/core/groupby.py", line 805, in apply return self._python_apply_general(f) File "/usr/local/lib/python2.7/dist-packages/pandas/core/groupby.py", line 809, in _python_apply_general self.axis) File "/usr/local/lib/python2.7/dist-packages/pandas/core/groupby.py", line 1969, in apply res = f(group) File "", line 1, in File "/usr/local/lib/python2.7/dist-packages/pandas/core/ops.py", line 1262, in f return self._combine_series(other, na_op, fill_value, axis, level) File "/usr/local/lib/python2.7/dist-packages/pandas/core/frame.py", line 3944, in _combine_series try_cast=try_cast) File "/usr/local/lib/python2.7/dist-packages/pandas/core/frame.py", line 3958, in _combine_series_infer try_cast=try_cast) File "/usr/local/lib/python2.7/dist-packages/pandas/core/frame.py", line 3981, in _combine_match_columns try_cast=try_cast) File "/usr/local/lib/python2.7/dist-packages/pandas/core/internals.py", line 3435, in eval return self.apply('eval', **kwargs) File "/usr/local/lib/python2.7/dist-packages/pandas/core/internals.py", line 3329, in apply applied = getattr(b, f)(**kwargs) File "/usr/local/lib/python2.7/dist-packages/pandas/core/internals.py", line 1377, in eval result = get_result(other) File "/usr/local/lib/python2.7/dist-packages/pandas/core/internals.py", line 1346, in get_result result = func(values, other) File "/usr/local/lib/python2.7/dist-packages/pandas/core/ops.py", line 1216, in na_op yrav.fill(yrav.item()) ValueError: can only convert an array of size 1 to a Python scalar

The error can be 'fixed' by applying another command to the grouped object first:

g.sum() B C A
a 3 10 b 3 5

g.apply(lambda x: x / x.sum()) B C 0 0.333333 0.4 1 0.666667 0.6 2 1.000000 1.0

Expected Output

g.apply(lambda x: x / x.sum()) B C 0 0.333333 0.4 1 0.666667 0.6 2 1.000000 1.0

Output of pd.show_versions()

>>> pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 2.7.12.final.0
python-bits: 64
OS: Linux
OS-release: 4.4.0-122-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: en_US.utf8
LANG: en_US.UTF-8
LOCALE: None.None

pandas: 0.22.0
pytest: 2.8.7
pip: 9.0.1
setuptools: 20.7.0
Cython: 0.23.4
numpy: 1.11.0
scipy: 0.17.0
pyarrow: None
xarray: None
IPython: 5.5.0
sphinx: None
patsy: 0.4.1
dateutil: 2.4.2
pytz: 2014.10
blosc: None
bottleneck: None
tables: 3.2.2
numexpr: 2.4.3
feather: None
matplotlib: 1.5.1
openpyxl: 2.3.0
xlrd: 0.9.4
xlwt: 0.7.5
xlsxwriter: None
lxml: 3.5.0
bs4: None
html5lib: 1.0.1
sqlalchemy: 1.0.11
pymysql: 0.7.2.None
psycopg2: 2.6.1 (dt dec mx pq3 ext lo64)
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None