PERF: Index.sort_values for already sorted index by lukemanley · Pull Request #56128 · pandas-dev/pandas (original) (raw)

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Conversation3 Commits2 Checks0 Files changed

Conversation

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters

[ Show hidden characters]({{ revealButtonHref }})

• Tests added and passed if fixing a bug or adding a new feature
• All code checks passed.
• Added type annotations to new arguments/methods/functions.
• Added an entry in the latest doc/source/whatsnew/v2.2.0.rst file if fixing a bug or adding a new feature.

Take advantage of the cached is_monotonic attributes.

import pandas as pd

N = 1_000_000

idx = pd._testing.makeStringIndex(N).sort_values()
%timeit idx.sort_values()

# 2.35 s ± 75.5 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)   <- main
# 14.6 µs ± 3.48 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)  <- PR

idx = pd.date_range("2000-01-01", freq="s", periods=N)
%timeit idx.sort_values(ascending=False)

# 90.3 ms ± 4.22 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)      <- main
# 11.6 µs ± 384 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)  <- PR

Existing ASV:

from asv_bench.benchmarks.categoricals import Indexing

b = Indexing()
b.setup()
%timeit b.time_sort_values()

# 4.85 ms ± 304 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)     <- main
# 21.4 µs ± 319 ns per loop (mean ± std. dev. of 7 runs, 10,000 loops each)  <- PR

Can you try this when this isn't cached? E.g. recreating the series for every pass

Can you try this when this isn't cached? E.g. recreating the series for every pass

without having been pre-cached:

import pandas as pd

N = 1_000_000

values = pd._testing.makeStringIndex(N).sort_values().values
%timeit pd.Index(values).sort_values()

# 2.45 s ± 48.8 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)  <- main
# 259 ms ± 9.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)   <- PR

values = pd.date_range("2000-01-01", freq="s", periods=N).values
%timeit pd.Index(values).sort_values(ascending=False)

# 91.7 ms ± 2.86 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)  <- main
# 2.89 ms ± 290 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  <- PR

Labels

Index

Related to the Index class or subclasses

Performance

Memory or execution speed performance

2 participants