PERF: perform a unique intersection with IntervalIndex if at least one side is unique by qwhelan · Pull Request #26711 · pandas-dev/pandas (original) (raw)

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Conversation6 Commits1 Checks0 Files changed

Conversation

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters

[ Show hidden characters]({{ revealButtonHref }})

qwhelan

Resolves #26709

       before           after         ratio
     [3ff4f38f]       [30bd5369]
     <datetime_iter~1>
-       749±100μs         365±70μs     0.49  index_object.IntervalIndexMethod.time_intersection_duplicate(1000)
-        59.6±3ms       8.04±0.2ms     0.13  index_object.IntervalIndexMethod.time_intersection_duplicate(100000)
           failed       2.50±0.01s      n/a  index_object.IntervalIndexMethod.time_intersection_duplicate(10000000)

TomAugspurger

@makbigc

@qwhelan Thanks for your quick respond.

Maybe adding check for duplicate nan in either side can resolve.
if self.isna().sum() > 1 or other.isna().sum() > 1:

@jreback

can you merge master & try @makbigc suggestion

@qwhelan

@codecov

1 similar comment

@codecov

@qwhelan

@makbigc Thanks, your suggestion works. The calls to .is_unique appear to give us that condition for free, so I've only added self.isna().sum() <= 1

@jreback Done and passing

@jreback

@qwhelan qwhelan deleted the unique_intersection branch

July 20, 2019 09:39