PERF: Indexing a multi-index is a lot slower · Issue #31648 · pandas-dev/pandas (original) (raw)

Skip to content

Navigation Menu

• • GitHub Copilot Write better code with AI
  • GitHub Advanced Security Find and fix vulnerabilities
  • Actions Automate any workflow
  • Codespaces Instant dev environments
  • Issues Plan and track work
  • Code Review Manage code changes
  • Discussions Collaborate outside of code
  • Code Search Find more, search less
• Explore
  • Learning Pathways
  • Events & Webinars
  • Ebooks & Whitepapers
  • Customer Stories
  • Partners
  • Executive Insights
• • GitHub Sponsors Fund open source developers
  • The ReadME Project GitHub community articles
• • Enterprise platform AI-powered developer platform
• Pricing

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sign up

Description

Indexing a multi-index seemingly went from O(1) to O(N):

I did a bisect, and found this was caused by the _shallow_copy here: b0f33b3#diff-4ffd1c69d47e0ac9f2de4f9e3e4a118cR643.

Code Sample

from time import perf_counter as time import pandas as pd

for N in [1000, 2000, 4000, 8000, 16000, 32000]: values = list(range(N)) df = pd.DataFrame({ 'a': values }) df['b'] = 1 df.set_index(['a', 'b'], inplace = True)

t = time()
df.loc[values]
t = time() - t
print(N, t)