(original) (raw)

Hi there.

I was made aware of this oddity here:

import cPickle

�

reffed = "xKITTENSx"\[1:-1\]

print repr(cPickle.dumps(reffed))

 

print repr(cPickle.dumps("xKITTENSx"\[1:-1\]))

 

These strings are different, presumably because of the (ob\_refcnt \== 1) optimization used during object pickling.

This might come as a suprise to many, and is potentially dangerous if pickling is being used as precursor to some hashing function.
For example, we use code that caches function calls, using something akin to:

myhash = hash(cPickle.dumps(arguments))

try:

� cached\_args, cached\_value = cache\[myhash\]

� if cached\_args == arguments: return cached\_value

except KeyError:

� value = Function(\*args)

� cache\[myhash\] = artuments, value

� return value

 

The non-uniqueness of the pickle string will cause unnecessary cache misses in this code.� Pickling is useful as a precusor because it allows for more varied object types than hash() alone would.

 

I just wanted to point this out.� We‘ll attempt some local workarounds here, but it should otherwise be simple to modify pickling to optionally turn off this optimization and always generate the same output irrespective of the internal reference counts of the objects.

 

Cheers,

 

Kristj�n