PERF: avoid unnecessary reordering in MultiIndex._reorder_indexer by lukemanley · Pull Request #48384 · pandas-dev/pandas (original) (raw)

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Conversation5 Commits10 Checks0 Files changed

Conversation

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters

[ Show hidden characters]({{ revealButtonHref }})

Tests added and passed if fixing a bug or adding a new feature
All code checks passed.
Added type annotations to new arguments/methods/functions.
Added an entry in the latest doc/source/whatsnew/v1.6.0.rst file if fixing a bug or adding a new feature.

MultiIndex._reorder_indexer contains logic to determine if indexer reordering is required. This PR expands the logic a bit to identify additional cases where reordering is not necessary.

ASVs:

       before           after         ratio
     [4b645ae5]       [07679a60]
     <main>           <multiindex-need-sort>
-        5.07±1ms          726±9μs     0.14  indexing.MultiIndexing.time_loc_all_bool_indexers(True)
-        8.13±2ms         709±20μs     0.09  indexing.MultiIndexing.time_loc_all_bool_indexers(False)

lukemanley changed the title~~PREF: avoid unnecessary reordering in MultiIndex._reorder_indexer~~ PERF: avoid unnecessary reordering in MultiIndex._reorder_indexer

Sep 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @lukemanley, looks good to me.

Can you merge master into your branch? I think CI problems should be fixed now.

# check if sorting is necessary
need_sort = False
for i, k in enumerate(seq):
if com.is_null_slice(k) or com.is_bool_indexer(k) or is_scalar(k):

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have unit tests for all these new branches that short circuit?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty there are tests that hit these in one way or another. Regardless, I just added test_get_locs_reordering that explicitly tests the ordering effects of: null slices, bool indexers, scalars, and lists of length 1.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great thanks!

noatamir pushed a commit to noatamir/pandas that referenced this pull request

Nov 9, 2022

…ndas-dev#48384)

Avoid unnecessary reordering in MultiIndex._reorder_indexer
whatsnew
mypy
cleanup
add multiindex.get_locs reordering tests
fix test

Co-authored-by: Matthew Roeschke 10647082+mroeschke@users.noreply.github.com