[Python-Dev] Re: [Csv] csv module and universal newlines (original) (raw)

Skip Montanaro skip at pobox.com
Wed Jan 12 02:59:22 CET 2005 

Andrew> The csv parser consumes lines from an iterator, but it also has
Andrew> it's own idea of end-of-line conventions, which are currently
Andrew> only used by the writer, not the reader, which is a source of
Andrew> much confusion. The writer, by default, also attempts to emit a
Andrew> \r\n sequence, which results in more confusion unless the file
Andrew> is opened in binary mode.

Andrew> I'm looking for suggestions for how we can mitigate these
Andrew> problems (without breaking things for existing users).

You can argue that reading csv data from/writing csv data to a file on Windows if the file isn't opened in binary mode is an error. Perhaps we should enforce that in situations where it matters. Would this be a start?

terminators = {"darwin": "\r",
               "win32": "\r\n"}

if (dialect.lineterminator != terminators.get(sys.platform, "\n") and
   "b" not in getattr(f, "mode", "b")):
   raise IOError, ("%s not opened in binary mode" %
                   getattr(f, "name", "???"))

The elements of the postulated terminators dictionary may already exist somewhere within the sys or os modules (if not, perhaps they should be added). The idea of the check is to enforce binary mode on those objects that support a mode if the desired line terminator doesn't match the platform's line terminator.

Skip

More information about the Python-Dev mailing list