BUG: pandas.read_fwf doesn't correctly works with skiprows as list and nrows · Issue #44021 · pandas-dev/pandas (original) (raw)

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of pandas.
• I have confirmed this bug exists on the master branch of pandas.

Reproducible Example

import io import pandas

fwf_data = ( "a b c d\n" "id8141 360.242940 149.910199 11950.7\n" "id1594 444.953632 166.985655 11788.4\n" "id1849 364.136849 183.628767 11806.2\n" "id1230 413.836124 184.375703 11916.8\n" "id1948 502.953953 173.237159 12468.3\n" "id8141 360.242940 149.910199 11950.7\n" "id1594 444.953632 166.985655 11788.4\n" "id1849 364.136849 183.628767 11806.2\n" )

df1 = pandas.read_fwf(filepath_or_buffer=io.StringIO(fwf_data), skiprows=[2, 5]) print(df1) '''Output a b c d 0 id8141 360.242940 149.910199 11950.7 1 id1849 364.136849 183.628767 11806.2 2 id1230 413.836124 184.375703 11916.8 3 id8141 360.242940 149.910199 11950.7 4 id1594 444.953632 166.985655 11788.4 5 id1849 364.136849 183.628767 11806.2 '''

df2 = pandas.read_fwf(filepath_or_buffer=io.StringIO(fwf_data), skiprows=[2, 5], nrows=7) print(df2) '''Output a b c d 0 id8141 360.242940 149.910199 11950.7 1 id1849 364.136849 183.628767 11806.2 2 id1230 413.836124 184.375703 11916.8 3 id1948 502.953953 173.237159 12468.3 4 id8141 360.242940 149.910199 11950.7 5 id1594 444.953632 166.985655 11788.4 6 id1849 364.136849 183.628767 11806.2 '''

Issue Description

For df2, not all specified lines are skipped.

Expected Behavior

df2 = pandas.read_fwf(filepath_or_buffer=io.StringIO(fwf_data), skiprows=[2, 5], nrows=7) print(df2) '''Expected output a b c d 0 id8141 360.242940 149.910199 11950.7 1 id1849 364.136849 183.628767 11806.2 2 id1230 413.836124 184.375703 11916.8 3 id8141 360.242940 149.910199 11950.7 4 id1594 444.953632 166.985655 11788.4 '''

Installed Versions

INSTALLED VERSIONS

commit : 73c6825
python : 3.8.10.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.19042
machine : AMD64
processor : Intel64 Family 6 Model 142 Stepping 12, GenuineIntel
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : English_United States.1252

pandas : 1.3.3
numpy : 1.21.1
pytz : 2020.1
dateutil : 2.8.2
pip : 21.2.2
setuptools : 57.4.0
Cython : None
pytest : 6.2.4
hypothesis : None
sphinx : 4.1.2
blosc : None
feather : 0.4.1
xlsxwriter : None
lxml.etree : 4.6.3
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 3.0.1
IPython : 7.27.0
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : 2021.07.0
fastparquet : None
gcsfs : None
matplotlib : 3.2.2
numexpr : 2.7.3
odfpy : None
openpyxl : 3.0.7
pandas_gbq : 0.15.0
pyarrow : 3.0.0
pyxlsb : None
s3fs : 0.4.2
scipy : 1.7.0
sqlalchemy : 1.4.22
tables : 3.6.1
tabulate : None
xarray : 0.19.0
xlrd : 2.0.1
xlwt : None
numba : None