BUG: Groupby rolling count slower than rolling mean & sum (v1.1.0) (original) (raw)

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of pandas.
• (optional) I have confirmed this bug exists on the master branch of pandas.

Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

Your code here

import pandas as pd

Generate sample df

df = pd.DataFrame({'column1': range(600), 'group': 5*['l'+str(i) for i in range(120)]})

sort by group for easy/efficient joining of new columns to df

df=df.sort_values('group',kind='mergesort').reset_index(drop=True)

timing of groupby rolling count, sum and mean

%timeit df['mean']=df.groupby('group').rolling(3,min_periods=1)['column1'].mean().values %timeit df['sum']=df.groupby('group').rolling(3,min_periods=1)['column1'].sum().values %timeit df['count']=df.groupby('group').rolling(3,min_periods=1)['column1'].count().values

Output

6.14 ms ± 812 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) 5.61 ms ± 179 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) 76.1 ms ± 4.78 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

Problem description

I am running a groupby rolling count, sum & mean using Pandas v1.1.0 and I notice that the rolling count is considerably slower than the rolling mean & sum. This seems counter intuitive as we can derive the count from the mean and sum and save time.

Expected Output

Expecting more efficient computation of groupby rolling count

### df Output for illustration
print(df.head(10))

   column1 group   mean     sum  count
0        0    l0    0.0     0.0    1.0
1      120    l0   60.0   120.0    2.0
2      240    l0  120.0   360.0    3.0
3      360    l0  240.0   720.0    3.0
4      480    l0  360.0  1080.0    3.0
5        1    l1    1.0     1.0    1.0
6      121    l1   61.0   122.0    2.0
7      241    l1  121.0   363.0    3.0
8      361    l1  241.0   723.0    3.0
9      481    l1  361.0  1083.0    3.0

Output of pd.show_versions()

Details

commit : d9fff27
python : 3.8.5.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.18362
machine : AMD64
processor : Intel64 Family 6 Model 142 Stepping 11, GenuineIntel
byteorder : little
LC_ALL : None
LANG : None
LOCALE : English_United States.1252

pandas : 1.1.0
numpy : 1.18.5
pytz : 2020.1
dateutil : 2.8.1
pip : 20.2.1
setuptools : 49.2.1.post20200802
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.11.2
IPython : 7.17.0
pandas_datareader: None
bs4 : 4.9.1
bottleneck : None
fsspec : 0.8.0
fastparquet : None
gcsfs : None
matplotlib : 3.3.0
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 1.0.0
pytables : None
pyxlsb : None
s3fs : None
scipy : 1.5.0
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : 0.48.0