BUG: AssertionError: Number of Block dimensions (1) must equal number of axes (2) when typing a column · Issue #35460 · pandas-dev/pandas (original) (raw)

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of pandas.
• (optional) I have confirmed this bug exists on the master branch of pandas.

Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

import pandas as pd

df = pd.DataFrame( {"e1_timestamp": [], "e2_timestamp": [] } )

The problematic code. Remove this and there are no exceptions.

for timefield in [ "e1_timestamp", "e2_timestamp", ]: df[timefield] = pd.to_datetime(df[timefield], utc=True) #

df = df.append({ "orderId": 1, "e1_timestamp": pd.to_datetime(pd.NaT, utc=True), "e2_timestamp": pd.to_datetime(pd.NaT, utc=True), },ignore_index=True,) df

Problem description

Why initializing an empty dataframe, defining its type as shown in the example and then appending a value, there is an AssertionError: Number of Block dimensions (1) must equal number of axes (2) exception.

If we remove the for loop, it does work:

df = pd.DataFrame(
            {"e1_timestamp": [],
                "e2_timestamp": []
            }
        )

df = df.append({
                    
                    "e1_timestamp": pd.to_datetime(pd.NaT, utc=True),
                    "e2_timestamp": pd.to_datetime(pd.NaT, utc=True),
                },ignore_index=True,)


df

We see that we could do df = df.astype({"e1_timestamp": 'datetime64[ns]', "e1_timestamp": 'datetime64[ns]'}) instead of that for loop, but there seems to be a bug in this new version. With pandas==1.0.5 the for loop does not raise an exception.

Expected Output

There should be no assertion error

Output of pd.show_versions()

INSTALLED VERSIONS

commit : d9fff27
python : 3.7.5.final.0
python-bits : 64
OS : Linux
OS-release : 4.9.184-linuxkit
Version : #1 SMP Tue Jul 2 22:58:16 UTC 2019
machine : x86_64
processor :
byteorder : little
LC_ALL : None
LANG : C.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.1.0
numpy : 1.16.0
pytz : 2019.1
dateutil : 2.8.1
pip : 19.3.1
setuptools : 41.4.0
Cython : None
pytest : 4.1.1
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : 0.9.3
psycopg2 : 2.8.2 (dt dec pq3 ext lo64)
jinja2 : 2.10.1
IPython : 7.11.1
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : 3.1.1
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
pyxlsb : None
s3fs : None
scipy : 1.2.0
sqlalchemy : 1.2.16
tables : None
tabulate : 0.8.3
xarray : None
xlrd : None
xlwt : None
numba : None