BUG: Python parser breaks with quotes and multi-char sep · Issue #13374 · pandas-dev/pandas (original) (raw)

On master (b722222):

data = 'a,,b\n1,,a\n2,,"2,,b"' read_csv(StringIO(data), sep=',,', engine='python') ... ValueError: Expected 2 fields in line 3, saw 3

I expect this command to work, but because no parsing is done on quoted fields as can be seen here, an extra field is produced, breaking the parser. Note that this does not affect the C parser because multi-char delimiters are not supported. Similar to what we saw in #10911 and #12775, but unless we want to write the tokenizer.c code in Python, a similar fix does not seem trivial.