[Python-Dev] Pickling problems are hard to debug (original) (raw)

Gary Poster gary at modernsongs.com
Sun Mar 26 04:07:20 CEST 2006

On Mar 25, 2006, at 8:13 PM, Greg Ewing wrote:

There seems to be a need for better diagnostics when pickle encounters something that can't be pickled.

Recently when attempting to pickle a rather large and complicated data structure, I got the following incomprehensible message: cPickle.PicklingError: args[0] from newobj args has the wrong class Trying again with protocol 1 instead of 2, I get TypeError: can't pickle function objects which I'm guessing is because somewhere I've tried to pickle a nested function or a bound method. But it still doesn't give me any idea which function I tried to pickle or where abouts it turns up in the data structure. Anyone have any ideas how the situation could be improved? At the very least, it could include some info about the type and identity of the offending object.

You are asking for ideas on how to change the pickle story to help.
However, just reading your issue, I thought I might have done a
debugging hack like this, at least for the protocol 1 traceback.

We'll assume that the error is more mysterious than what I've
manufactured here.

import cPickle cPickle.dumps({'foo': lambda: 42}) Traceback (most recent call last): File "", line 1, in ? File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ python2.4/copy_reg.py", line 69, in _reduce_ex raise TypeError, "can't pickle %s objects" % base.name TypeError: can't pickle function objects import copy_reg def debug(obj): ... import pdb; pdb.set_trace() ... import types copy_reg.pickle(types.FunctionType, debug) cPickle.dumps({'foo': lambda: 42}) --Return-- (2)debug()->None (Pdb) p obj <function at 0x63230> (Pdb) p obj.module 'main'

I also might have used pickle, rather than cPickle, to try and see
what happened, if that ended up being necessary.

I don't use protocol 2 much: that error message in particular looked
a bit difficult, and my hack might not be any help there. I agree
that it would be nice to have a better message there, in particular.

back to lurking...

Gary

More information about the Python-Dev mailing list