BUG: frame.isetitem incorrectly casts Int64 columns to dtype object · Issue #49922 · pandas-dev/pandas (original) (raw)

Pandas version checks

Reproducible Example

import pandas as pd

df1 = pd.DataFrame({"foo": [0, 0, 0]}, index=[1, 2, 3]) df2 = pd.DataFrame({"foo": [1, 1, 1]}, index=[2, 3, 4])

int_cols = [0, 1] full_df1 = pd.concat([df1, df2], axis="columns") full_df2 = full_df1.copy() # to showcase different behaviour

My current code: Works, but started giving me random FutureWarning as of v1.5

(apparently happens when df1 and df2 have the same index initially, not relevant to this bug)

full_df1.iloc[:, int_cols] = full_df1.iloc[:, int_cols].astype("Int64") print(full_df1.dtypes)

foo Int64

foo Int64

dtype: object

My attempt to future-proof my code (as I quite often have columns that are non-unique):

full_df2.isetitem(int_cols, full_df2.iloc[:, int_cols].astype("Int64")) print(full_df2.dtypes)

foo object

foo object

dtype: object

Issue Description

After trying to follow the instructions in the FutureWarning referenced in the code above, I got into trouble with nullable integer columns ending up as dtype=object.

The warning in question:

FutureWarning: In a future version, df.iloc[:, i] = newvals will attempt to set the values inplace instead of always setting a new array. To retain the old behavior, use either df[df.columns[i]] = newvals or, if columns are non-unique, df.isetitem(i, newvals)

Expected Behavior

I would expect that frame.isetitem(cols, ...) would not cast Int64 columns to object, but rather work exactly similar to how frame.iloc[:, cols] = ... works.

Installed Versions

INSTALLED VERSIONS

commit : 8dab54d
python : 3.8.13.final.0
python-bits : 64
OS : Darwin
OS-release : 22.1.0
Version : Darwin Kernel Version 22.1.0: Sun Oct 9 20:15:09 PDT 2022; root:xnu-8792.41.9~2/RELEASE_ARM64_T6000
machine : arm64
processor : arm
byteorder : little
LC_ALL : None
LANG : None
LOCALE : None.UTF-8

pandas : 1.5.2
numpy : 1.23.2
pytz : 2022.2.1
dateutil : 2.8.2
setuptools : 65.4.1
pip : 22.3.1
Cython : 0.29.32
pytest : 6.2.5
hypothesis : None
sphinx : 5.1.1
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : 1.1
pymysql : None
psycopg2 : None
jinja2 : 3.1.2
IPython : 8.4.0
pandas_datareader: None
bs4 : 4.11.1
bottleneck : None
brotli : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : 3.5.3
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : None
snappy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
zstandard : None
tzdata : None