BUG: MultiIndex.unique losing ea dtype by phofl · Pull Request #48335 · pandas-dev/pandas (original) (raw)

the MultiIndex.unique code path is casting the values to tuples and runs through unique1d. This is really slow.

       before           after         ratio
     [e7414aa1]       [d17eb141]
     <48184^2>        <perf_mi_unique>
-         422±2ms       30.3±0.5ms     0.07  multiindex_object.Unique.time_unique_dups(('Int64', <NA>))
-        496±70ms         28.4±1ms     0.06  multiindex_object.Unique.time_unique_dups(('int64', 0))
-         811±4ms       32.9±0.2ms     0.04  multiindex_object.Unique.time_unique(('int64', 0))
-        812±20ms       32.8±0.9ms     0.04  multiindex_object.Unique.time_unique(('Int64', <NA>))