[Python-Dev] when is binary mode required for pickle? (original) (raw)

Guido van Rossum guido@python.org
Wed, 12 Feb 2003 10:35:14 -0500

Unused as I am to programming on Windows, it's not clear to me under what circumstances I might need to open a file in binary mode for Pickle's consumption. The reason I ask is that while working on the csv PEP and module, we had some input complaining about our requirement that CSV files be opened with the 'b' flag (where it matters). Andrew mentioned that Pickle requires files be opened in binary mode. However, the documentation doesn't explicitly state this. Must, for example, a file be opened with the 'b' flag if an ASCII pickle is to be written? The doc for the Unpickler class says the file-like object must support both read(n) and readline() methods. The requirement for readline() suggests the file be opened in text mode.

The situation for pickling is complex. Pickling protocol 0 allows opening the file in text mode, as long as this is done both for reading and for writing. Pickling protocol 1 (AKA known as "binary mode") and protocol 2 (new in Python 2.3) require opening the file in binary mode (but nobody ever checks).

There's no requirement that files are opened in text mode to use readline() -- but when a file was written in text mode and then readline() is used in binary mode, all your line endings will look like "\r\n".

(Maybe this is just a nit with the Pickle docs and I don't know it?)

Who knows.

--Guido van Rossum (home page: http://www.python.org/~guido/)