Factorize don't preserve na_sentinel values when sort is specified · Issue #25409 · pandas-dev/pandas (original) (raw)

Code Sample, a copy-pastable example if possible

print("\nWithout sort:") labels, uniques = pd.factorize(['b', None, 'a', 'c', 'b'], na_sentinel=100) print("Labels: {}".format(labels)) print("Uniques: {}".format(uniques))

print("\nWith sort:") labels, uniques = pd.factorize(['b', None, 'a', 'c', 'b'], na_sentinel=100, sort=True) print("Labels: {}".format(labels)) print("Uniques: {}".format(uniques))

Without sort: Labels: [ 0 100 1 2 0] Uniques: ['b', 'a', 'c']

With sort: Labels: [ 1 4556668520 0 2 1] Uniques: ['a', 'b', 'c']

Problem description

Calls to factorize with a na_sentinel value set and sort=True appears not to respect the na_sentinel values specified.

Expected Output

Without sort:
Labels: [  0 100   1   2   0]
Uniques: ['b', 'a', 'c']

With sort:
Labels: [         1 100          0          2          1]
Uniques: ['a', 'b', 'c']

Output of pd.show_versions()

[paste the output of pd.show_versions() here below this line]

INSTALLED VERSIONS

commit: None
python: 3.6.8.final.0
python-bits: 64
OS: Darwin
OS-release: 18.2.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_IE.UTF-8
LOCALE: en_IE.UTF-8

pandas: 0.24.1
pytest: None
pip: 18.1
setuptools: 40.6.3
Cython: None
numpy: 1.16.1
scipy: 1.2.0
pyarrow: None
xarray: None
IPython: 7.2.0
sphinx: None
patsy: None
dateutil: 2.8.0
pytz: 2018.9
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 3.0.2
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml.etree: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
gcsfs: None