BUG: rank does not respect na_option='keep' for numpy nullable integer dtypes · Issue #56976 · pandas-dev/pandas (original) (raw)

Pandas version checks

Reproducible Example

import pandas as pd s = pd.Series([pd.NA, 2, pd.NA, 3, 3, 2, 3, 1])

s.rank() # This output is OK!

s.astype('Int64').rank() # This output fails, although it should be equal to above

Issue Description

pd.Series.rank does not keep missing values for certain dtypes, even when na_option='keep' is set (the default).
Instead they receive rank values ordered somewhere inbetween.

I experienced this behaviour in numpy-nullable integer dtypes.
It does not seem to appear for object, numpy-nullable floats or pyarrow floats/ints.

Expected Behavior

Expected output would be the one of plain s.rank()

0  NaN
1  2.5
2  NaN
3  5.0
4  5.0
5  2.5
6  5.0
7  1.0

However, running the code yields 2.5 instead of the NaN and the other numbers are shifted as well.

Installed Versions

INSTALLED VERSIONS
------------------
commit                : 7cd8ae5bdc069dcdaeb892418c932a7124a87dcf
python                : 3.11.2.final.0
python-bits           : 64
OS                    : Linux
OS-release            : 6.1.0-13-amd64
Version               : #1 SMP PREEMPT_DYNAMIC Debian 6.1.55-1 (2023-09-29)
machine               : x86_64
processor             : 
byteorder             : little
LC_ALL                : None
LANG                  : en_US.UTF-8
LOCALE                : en_US.UTF-8

pandas                : 3.0.0.dev0+119.g7cd8ae5bdc.dirty
numpy                 : 1.26.3
pytz                  : 2023.3.post1
dateutil              : 2.8.2
setuptools            : 66.1.1
pip                   : 23.0.1
Cython                : 3.0.5
pytest                : 7.4.4
hypothesis            : 6.94.0
sphinx                : 7.2.6
blosc                 : None
feather               : None
xlsxwriter            : 3.1.9
lxml.etree            : 5.1.0
html5lib              : 1.1
pymysql               : 1.4.6
psycopg2              : 2.9.9
jinja2                : 3.1.3
IPython               : 8.20.0
pandas_datareader     : None
adbc-driver-postgresql: None
adbc-driver-sqlite    : None
bs4                   : 4.12.2
bottleneck            : 1.3.7
dataframe-api-compat  : None
fastparquet           : 2023.10.1
fsspec                : 2023.12.2
gcsfs                 : 2023.12.2post1
matplotlib            : 3.8.2
numba                 : 0.58.1
numexpr               : 2.8.8
odfpy                 : None
openpyxl              : 3.1.2
pandas_gbq            : None
pyarrow               : 14.0.2
pyreadstat            : 1.2.6
python-calamine       : None
pyxlsb                : 1.0.10
s3fs                  : 2023.12.2
scipy                 : 1.11.4
sqlalchemy            : 2.0.25
tables                : 3.9.2
tabulate              : 0.9.0
xarray                : 2023.12.0
xlrd                  : 2.0.1
zstandard             : 0.22.0
tzdata                : 2023.4
qtpy                  : None
pyqt5                 : None