[Python-Dev] RE: [Zope-CVS] CVS: Products/Ape/lib/apelib/zodb3 - serializers.py:1.3storage.py:1.8 (original) (raw)
Tim Peters tim at zope.com
Tue Sep 16 21:34:50 EDT 2003
- Previous message: [Python-Dev] Re: [Python-checkins]python/dist/src/Objects unicodeobject.c, 2.197, 2.198
- Next message: [Python-Dev] RE: [Zope-CVS] CVS: Products/Ape/lib/apelib/zodb3 - serializers.py:1.3storage.py:1.8
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
[Shane Hathaway]
Modified Files: serializers.py storage.py Log Message: Fixed Python 2.3 compatibility.
- If you append to a list while it is being pickled, Python 2.3 will pick up the new items. Previous versions of Python did not.
How do you grow a list while it's being pickled? Must be another thread doing that, and if so that's mighty dubious. If you're pickling to a genuine file object, then cPickle releases the GIL around its fwrite() calls, so that's one way. But in the context of the patch:
p.persistent_id = lambda ob: None # Stop recording references p.dump(unmanaged) s = outfile.getvalue() event.addUnmanagedPersistentObjects(unmanaged)
the use of .getvalue() suggests the pickle target isn't a real file. Then cPickle never releases the GIL (cStringIO doesn't either), and no other thread should dare muck with a Python object (like the Python list getting pickled) without holding the GIL.
However this ends up happening, if you instead shrink a list while it's being pickled, Pythons earlier than 2.3 can end up trying to pickle random recycled memory. Their cPickles captured the list length once at the start of pickling the list, and that coding optimization is appropriate only if the GIL is never released. 2.3 uses the iteration protocol instead, so "sees" dynamic changes in the list size.
Copying Python-Dev because this may reveal a new mountain of ways to crash the interpreter: mutate objects in devious ways while they're getting cPickled.
- Previous message: [Python-Dev] Re: [Python-checkins]python/dist/src/Objects unicodeobject.c, 2.197, 2.198
- Next message: [Python-Dev] RE: [Zope-CVS] CVS: Products/Ape/lib/apelib/zodb3 - serializers.py:1.3storage.py:1.8
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]