[Python-Dev] Pickle alternative in stdlib (Was: On breaking modules into packages) (original) (raw)

anatoly techtonik techtonik at gmail.com
Thu Nov 4 17:15:57 CET 2010

On Thu, Nov 4, 2010 at 3:38 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

On Thu, Nov 4, 2010 at 4:28 PM, anatoly techtonik <techtonik at gmail.com> wrote:

On Wed, Nov 3, 2010 at 9:08 PM, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:

This is the strongest reason why I recommend to everyone I know that they not use pickle for storage they'd like to keep working after upgrades [not just of stdlib, but other 3rd party software or their own software]. :) +1. Twisted actually tried to preserve pickle compatibility in the bad old days, but it was impossible.  Pickles should never really be saved to disk unless they contain nothing but lists, ints, strings, and dicts. But what is alternative in stdlib? Don't you think that Python doesn't provide any? Python 3.2a3+ (py3k:85817, Oct 24 2010, 19:25:28) [GCC 4.4.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. import json dir(json) ['JSONDecoder', 'JSONEncoder', 'all', 'author', 'builtins', 'cached', 'doc', 'file', 'name', 'package', 'path', 'version', 'defaultdecoder', 'defaultencoder', 'decoder', 'dump', 'dumps', 'encoder', 'load', 'loads', 'scanner'] pickle gets overspecific in many ways, and hence (despite our best efforts, and those of third parties) may break when changing Python versions. Serialising to something more language natural (be it JSON, YAML, XML or one of the multitude of other state encoding formats out there) is far more likely to be future proof. As a tool for communicating between different instances of the same version of Python though, pickle is fine.

pickle is insecure, marshal too. What about JSON? IIUC you need a definition of a class to be able to unserialize it in all cases. I wonder how is this definition validated, i.e. what to watch for when modifying classes that can be serialized.

anatoly t.

More information about the Python-Dev mailing list