GroupBy nth with Categorical, NA Data and dropna · Issue #26454 · pandas-dev/pandas (original) (raw)
The nth method of GroupBy objects can accept a dropna keyword, but it doesn't handle Categorical data appropriately. For instance:
In [9]: cat = pd.Categorical(['a', np.nan, np.nan], categories=['a', 'b']) ...: ser = pd.Series([1, 2, 3]) ...: df = pd.DataFrame({'cat': cat, 'ser': ser})
In [10]: df.groupby('cat')['ser'].nth(0, dropna='all') ValueError: Length mismatch: Expected axis has 3 elements, new values have 2 elements
In [11]: df.groupby('cat', observed=True)['ser'].nth(0, dropna='all') ValueError: Length mismatch: Expected axis has 3 elements, new values have 1 elements
Note that you can get a nonsensical result if the length of the categories matches the length of the Categorical:
In [9]: cat = pd.Categorical(['a', np.nan, np.nan], categories=['a', 'b', 'c']) ...: ser = pd.Series([1, 2, 3]) ...: df = pd.DataFrame({'cat': cat, 'ser': ser}) In [10]: df.groupby('cat')['ser'].nth(0, dropna='all') Out[12]: cat a 1 b 2 c 3 Name: ser, dtype: int64
I'm not sure it makes sense to specify both observed and dropna anyway, so I think we should be raising if a categorical with NA values is detected within that method