REGR: Buffer dtype mismatch, expected 'Python object' but got a string · Issue #38900 · pandas-dev/pandas (original) (raw)

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of pandas.
• (optional) I have confirmed this bug exists on the master branch of pandas.

Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

Your code here

import pandas as pd df = pd.DataFrame(['o',"o'doul's",'oad','oaf','oafish']) df = df.astype('|S') df 0 0 b'o' 1 b"o'doul's" 2 b'oad' 3 b'oaf' 4 b'oafish' import numpy as np df.replace({None:np.nan})

Problem description

This was working in 1.1.5

    df[c].replace({None: np.NaN}, inplace=True)
  File "/home/user/anaconda3/envs/uniqueness/lib/python3.8/site-packages/pandas/core/series.py", line 4479, in replace
    return super().replace(
  File "/home/user/anaconda3/envs/uniqueness/lib/python3.8/site-packages/pandas/core/generic.py", line 6840, in replace
    return self.replace(
  File "/home/user/anaconda3/envs/uniqueness/lib/python3.8/site-packages/pandas/core/series.py", line 4479, in replace
    return super().replace(
  File "/home/user/anaconda3/envs/uniqueness/lib/python3.8/site-packages/pandas/core/generic.py", line 6889, in replace
    new_data = self._mgr.replace_list(
  File "/home/user/anaconda3/envs/uniqueness/lib/python3.8/site-packages/pandas/core/internals/managers.py", line 664, in replace_list
    bm = self.apply(
  File "/home/user/anaconda3/envs/uniqueness/lib/python3.8/site-packages/pandas/core/internals/managers.py", line 427, in apply
    applied = getattr(b, f)(**kwargs)
  File "/home/user/anaconda3/envs/uniqueness/lib/python3.8/site-packages/pandas/core/internals/blocks.py", line 910, in _replace_list
    [b.convert(numeric=False, copy=True) for b in result]
  File "/home/user/anaconda3/envs/uniqueness/lib/python3.8/site-packages/pandas/core/internals/blocks.py", line 910, in <listcomp>
    [b.convert(numeric=False, copy=True) for b in result]
  File "/home/user/anaconda3/envs/uniqueness/lib/python3.8/site-packages/pandas/core/internals/blocks.py", line 2558, in convert
    values = f(None, self.values.ravel(), None)
  File "/home/user/anaconda3/envs/uniqueness/lib/python3.8/site-packages/pandas/core/internals/blocks.py", line 2542, in f
    values = soft_convert_objects(
  File "/home/user/anaconda3/envs/uniqueness/lib/python3.8/site-packages/pandas/core/dtypes/cast.py", line 1139, in soft_convert_objects
    values = lib.maybe_convert_objects(values, convert_datetime=True)
  File "pandas/_libs/lib.pyx", line 2125, in pandas._libs.lib.maybe_convert_objects
ValueError: Buffer dtype mismatch, expected 'Python object' but got a string

Expected Output

Nones are replaced with nans

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit : b5958eepython : 3.8.5.final.0 python-bits : 64 OS : Linux OS-release : 4.15.0-128-generic Version : #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 machine : x86_64 processor : x86_64 byteorder : little LC_ALL : None LANG : en_GB.UTF-8 LOCALE : en_GB.UTF-8

pandas : 1.1.5
numpy : 1.19.4
pytz : 2020.1
dateutil : 2.8.1
pip : 20.2.4
setuptools : 50.3.2
Cython : 3.0a6
pytest : 6.1.2
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : 4.5.2
html5lib : None
pymysql : None
psycopg2 : 2.8.5 (dt dec pq3 ext lo64)
jinja2 : 3.0.0a1
IPython : 7.17.0
pandas_datareader: None
bs4 : 4.9.3
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : 3.3.2
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
pyxlsb : None
s3fs : None
scipy : 1.4.1
sqlalchemy : 1.3.20
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : 0.52.0