BUG: Reading a subset of columns with read_fwf fails if names are given when widths argument is used · Issue #46580 · pandas-dev/pandas (original) (raw)

Pandas version checks

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of pandas.
• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

from io import StringIO import pandas as pd

test_data = StringIO("""0 1 n -0.4100.1 4.6 0 2 p 0.2 90.1 4.7 0 3 n -0.3140.4 14.8""")

widths = (3, 5, 1, 5, 5, 8) usecols = (0, 1, 3, 5) col_names = ('c0', 'c1', 'c3', 'c5') df = pd.read_fwf(test_data, header=None, usecols=usecols, widths=widths, index_col=False, names=col_names) print(df)

Issue Description

If the widths argument is passed to read_fwf instead of colspecs when a subset of the columns is requested (and they are named using the names arugment) a ValueError: Length of colspecs must match length of names is raised.

This didn't happen in 1.3 and seems to have been introduced by #42920 – that one fixed an issue of len(colspecs) not matching len(names) but when widths is used instead, colspecs is built from all the column widths (including the ones not needed) and leads to a mismatch in lengths.

A work-around is to not set the column names in the read_fwf call but to do it afterwards with df.columns = ('c0', 'c1', 'c3', 'c5').

Expected Behavior

It should work as before: the above code should return the DataFrame

   c0  c1   c3    c5
0   0   1 -0.4   4.6
1   0   2  0.2   4.7
2   0   3 -0.3  14.8

Installed Versions

INSTALLED VERSIONS

commit : 06d2301
python : 3.9.1.final.0
python-bits : 64
OS : Darwin
OS-release : 21.2.0
Version : Darwin Kernel Version 21.2.0: Sun Nov 28 20:28:54 PST 2021; root:xnu-8019.61.5~1/RELEASE_X86_64
machine : x86_64
processor : i386
byteorder : little
LC_ALL : None
LANG : en_GB.UTF-8
LOCALE : en_GB.UTF-8

pandas : 1.4.1
numpy : 1.22.2
pytz : 2022.1
dateutil : 2.8.2
pip : 22.0.3
setuptools : 52.0.0
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 3.0.3
IPython : 8.0.1
pandas_datareader: None
bs4 : None
bottleneck : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : 3.5.1
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : 1.8.0
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
zstandard : None