[Python-Dev] new features for 2.3? (original) (raw)

Guido van Rossum guido@python.org
Wed, 08 Jan 2003 21:06:54 -0500

> We can do that in Python 2.3. Because this is backwards incompatible, > I propose that you have to request this protocol explicitly. I > propose to "upgrade' the binary flag to a general "protocol version" > flag, with values: > > 0 - original protocol > 1 - binary protocol > 2 - new protocol

If you're going to do this, why not go all the way, and encode the python version in the protocol? The value to use could be encoded as constants in pickle.py (and cPickle), with a CURRENTPROTOCOL or some such to make things easy. It might also be an idea to allow passing sys.versioninfo. The thing I'm trying to avoid is a proliferation of magic numbers when pickling ... what happens when we get another backwards-incompatible feature ... do we bump the number up to 3 and call it 'newer protocol'? I think using pickle constants (be they version numbers or otherwise) is the way to go here.

No way. If any kind of version number is encoded in a pickle, it should be the pickle protocol version. Unlike marshalled data, pickles are guraranteed future-proof, and also backwards compatible, unless the pickled data itself depends on Python features (for example, an instance of a list subclass pickled in Python 2.2 can't be unpickled in older Python versions). Each subsequent pickle protocol is a superset of the previous protocol.

There are use cases that have many pickles containing very small amounts of data, where adding a version header to each pickle would amount to a serious increase of data size. Maybe we can afford adding one byte to indicate the new version number (so that unpicklers that don't speak the new protocol will die cleanly), but I think that's about it.

--Guido van Rossum (home page: http://www.python.org/~guido/)