Inconsistent type casting between DataFrame and Series (original) (raw)

There is an inconsistency between how Series and DataFrames export types to dicts. The issue only manifests if the dataframe has multiple columns with different types. I think the issue is probably pretty far under the surface with differences between Series and DataFrame dtypes and just happened to be uncovered by export to dict.

Code Sample, a copy-pastable example if possible

import pandas as pd

orig_dict = [{0: 'a', 1: 1}]

# does not happen with
# orig_dict =[{0: 0, 1: 1}]

print 'Original dict'
for k in orig_dict[0]:
    print type(orig_dict[0][k])
print

df = pd.DataFrame(orig_dict)
dict_exported_from_dataframe = [df.loc[[x]].to_dict(orient='records')[0] for x in df.index]
dict_exported_from_series = [df.loc[x].to_dict() for x in df.index]

print 'Dict exported from DataFrame'
for k in dict_exported_from_dataframe[0]:
    print type(dict_exported_from_dataframe[0][k])
print

print 'Dict exported from Series'
for k in dict_exported_from_series[0]:
    print type(dict_exported_from_series[0][k])
print

Expected Output

Original dict
<type 'str'>
<type 'int'>

Dict exported from DataFrame
<type 'str'>
<type 'int'>

Dict exported from Series
<type 'str'>
<type 'int'>

OR

Original dict
<type 'str'>
<type 'int'>

Dict exported from DataFrame
<type 'str'>
<type 'numpy.int64'>

Dict exported from Series
<type 'str'>
<type 'numpy.int64'>

output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 2.7.10.final.0
python-bits: 64
OS: Darwin
OS-release: 14.5.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

pandas: 0.18.1
nose: None
pip: 8.1.2
setuptools: 25.2.0
Cython: None
numpy: 1.11.1
scipy: 0.13.0b1
statsmodels: None
xarray: None
IPython: 5.1.0
sphinx: None
patsy: None
dateutil: 2.5.3
pytz: 2016.6.1
blosc: None
bottleneck: None
tables: None
numexpr: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
boto: None
pandas_datareader: None