Performance issues with groupby for large values of ngroups · Issue #8426 · pandas-dev/pandas (original) (raw)

Skip to content

Navigation Menu

• • GitHub Copilot Write better code with AI
  • GitHub Advanced Security Find and fix vulnerabilities
  • Actions Automate any workflow
  • Codespaces Instant dev environments
  • Issues Plan and track work
  • Code Review Manage code changes
  • Discussions Collaborate outside of code
  • Code Search Find more, search less
• Explore
  • Learning Pathways
  • Events & Webinars
  • Ebooks & Whitepapers
  • Customer Stories
  • Partners
  • Executive Insights
• • GitHub Sponsors Fund open source developers
  • The ReadME Project GitHub community articles
• • Enterprise platform AI-powered developer platform
• Pricing

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sign up

Appearance settings

Description

Per @jreback: #8410 (comment)

Groupby for large values of ngroups, 10000 in this case, is very slow.

Invoked with :
--ncalls: 3
--repeats: 3


---------------------------------------------------------
Test name                                    |    #0    |
---------------------------------------------------------

...

groupby_large_ngroups_value_counts           | 23527.0356 |
groupby_large_ngroups_nunique                | 19496.5300 |
groupby_large_ngroups_describe               | 82329.8963 |
groupby_large_ngroups_mad                    | 21441.5947 |
groupby_large_ngroups_pct_change             | 18865.5750 |

See also: https://gist.github.com/dlovell/ea3400273314e7612f6e