skipfooter doesn't really "skip" in read_csv · Issue #13879 · pandas-dev/pandas (original) (raw)

On master:

from pandas import read_csv from pandas.compat import StringIO data = 'a,b,c\ncat,foo,bar\ndog,foo,"baz' # Note the stray quotation mark read_csv(StringIO(data), engine='python', skipfooter=1) ... _csv.Error: unexpected end of data

If we were truly "skipping" the last row, no error should have been raised. However, this occurs because the data is all parsed in memory first with Python's csv library.

Whether this is intended behaviour or not has implications for the C engine in terms of implementing analogous skipfooter behaviour. Or perhaps it has something to do with the fact that error_bad_lines and error_warn_lines parameters not with the Python engine?

xref #5232