Reading a CSV with duplicated MultiRow columns · Issue #18062 · pandas-dev/pandas (original) (raw)

Problem description

I have the following .csv file:

I want to use the first two rows to create MultiIndex columns. However, the first and second columns are duplicated and pandas is handling that in a weird way.

Code Sample, a copy-pastable example if possible

print(pd.read_csv('test.csv',header=[0,1]).columns)

Index([('A', 'one'), '('A', 'one').1', ('B', 'two')], dtype='object')

This output is a total mess, the first column and last are tuples while the second column is a string. Also, this is not a MultiIndex, it's an Index. Raising an error or having the option to delete the duplicate columns would be a better solution.

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.5.2.final.0
python-bits: 64
OS: Linux
OS-release: 4.4.0-97-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_GB.UTF-8
LOCALE: en_GB.UTF-8

pandas: 0.21.0
pytest: 2.9.2
pip: 9.0.1
setuptools: 27.2.0
Cython: 0.24.1
numpy: 1.13.3
scipy: 0.18.1
pyarrow: None
xarray: None
IPython: 5.1.0
sphinx: 1.4.6
patsy: 0.4.1
dateutil: 2.6.1
pytz: 2017.3
blosc: None
bottleneck: 1.1.0
tables: 3.2.3.1
numexpr: 2.6.1
feather: None
matplotlib: 1.5.3
openpyxl: 2.3.2
xlrd: 1.0.0
xlwt: 1.1.2
xlsxwriter: 0.9.3
lxml: 3.6.4
bs4: 4.5.1
html5lib: None
sqlalchemy: 1.0.13
pymysql: None
psycopg2: None
jinja2: 2.8
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None