[Python-Dev] Re: [Csv] csv module and universal newlines (original) (raw)

Skip Montanaro skip at pobox.com
Thu Jan 13 03:36:54 CET 2005 

>> The idea of the check is to enforce binary mode on those objects that
>> support a mode if the desired line terminator doesn't match the
>> platform's line terminator.

Andrew> Where that falls down, I think, is where you want to read an
Andrew> alien file - in fact, under unix, most of the CSV files I read
Andrew> use \r\n for end-of-line.

Well, you can either require 'b' in that situation or "know" that 'b' isn't needed on Unix systems.

Andrew> Also, I *really* don't like the idea of looking for a mode
Andrew> attribute on the supplied iterator - it feels like a layering
Andrew> violation. We've advertised the fact that it's an iterator, so
Andrew> we shouldn't be using anything but the iterator protocol.

The fundamental problem is that the iterator protocol on files is designed for use only with text mode (or universal newline mode, but that's just as much of a problem in this context). I think you either have to abandon the iterator protocol or peek under the iterator's covers to make sure it reads and writes in binary mode. Right now, people on windows create writers like this

writer = csv.writer(open("somefile", "w"))

and are confused when their csv files contain blank lines. I think the reader and writer objects have to at least emit a warning when they discover a source or destination that violates the requirements.

Skip

More information about the Python-Dev mailing list