BUG: Pandas 1.4.x - calling groupBy(...).apply(func) on an empty dataframe invokes func · Issue #47985 · pandas-dev/pandas (original) (raw)

As of pandas 1.4.0, if you call df.groupby(['some_column']).apply(_func) on an empty dataframe, the func will be called with an empty group dataframe, causing the function to potentially error out unless it specifically checks for an empty dataframe. Obviously, there are times when you are manipulating dataframes in such a way that the end result can be empty in a non-deterministic way.

Previous versions of pandas would not call the func if the dataframe was empty.

The following code runs fine in all versions of pandas I've tried prior to 1.4.0. As of 1.4.0, it causes an IndexError('single positional indexer is out-of-bounds') due to accessing a column in an empty dataframe.

import pandas as pd

def _checker(grp): # In Pandas 1.4.0 and above, _checker will be called # even though the group dataframe was empty. This will # cause the below line to blow up sides = grp['side'].iloc[0] print(sides)

Create a single row dataframe

d = {'root_id': [], 'side': []} df = pd.DataFrame(data=d)

In all version of Pandas prior to 1.4.0, the following worked (skipping the call to _checker)

df.groupby(['root_id']).apply(_checker)