PERF: speed up multi-key groupby by behzadnouri · Pull Request #8128 · pandas-dev/pandas (original) (raw)

I had some dependency issue, so ran the benchmarks manually;
On master:

>>> groupby_transform_multi_key1.run()
{'loops': 1, 'timing': 1880.3930282592773, 'repeat': 3, 'succeeded': True, 'units': 'ms'}
>>> groupby_transform_multi_key2.run()
{'loops': 1, 'timing': 1886.4881992340088, 'repeat': 3, 'succeeded': True, 'units': 'ms'}
>>> groupby_transform_multi_key3.run()
{'loops': 1, 'timing': 5588.175058364868, 'repeat': 3, 'succeeded': True, 'units': 'ms'}
>>> groupby_transform_multi_key4.run()
{'loops': 1, 'timing': 4688.298940658569, 'repeat': 3, 'succeeded': True, 'units': 'ms'}

on branch:

>>> groupby_transform_multi_key1.run()
{'loops': 1, 'timing': 106.5061092376709, 'repeat': 3, 'succeeded': True, 'units': 'ms'}
>>> groupby_transform_multi_key2.run()
{'loops': 10, 'timing': 73.46320152282715, 'repeat': 3, 'succeeded': True, 'units': 'ms'}
>>> groupby_transform_multi_key3.run()
{'loops': 1, 'timing': 1123.687982559204, 'repeat': 3, 'succeeded': True, 'units': 'ms'}
>>> groupby_transform_multi_key4.run()
{'loops': 1, 'timing': 200.47903060913086, 'repeat': 3, 'succeeded': True, 'units': 'ms'}

groupby_transform_multi_key[3|4] needs 6.5G memory to run on master.