[Python-Dev] Extendability of C vs Python pickle (original) (raw)
Antoine Pitrou solipsis at pitrou.net
Mon Jul 10 05:13:13 EDT 2017
- Previous message (by thread): [Python-Dev] Pure pickle bechmark.
- Next message (by thread): [Python-Dev] Improve test coverage for standard library
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
On Mon, 10 Jul 2017 02:35:37 +0200 Victor Stinner <victor.stinner at gmail.com> wrote:
I already proposed to remove this benchmark: https://mail.python.org/pipermail/speed/2017-April/000554.html but Antoine Pitrou mentionned that the cloudpickle project uses it. Maybe we should try to understand what's wrong with pickle (C module) for cloudpickle?
That's a good question, Victor. cloudpickle uses three hooks inside pickle.py's Pickler:
- the "dispatch" dictionary
- overriding the "save_global" method to support saving more objects (such as closures, etc.)
- overriding the "save_reduce" method; this one doesn't seem really necessary, perhaps some leftover from previous attempts
_pickle.c's Pickler does seem to allow a custom "dispatch_table", but it doesn't allow overriding "save_global".
Of course, if _pickle.c were improved to allow such extensions, it would suddenly allow cloudpickle to be much more performant!
Regards
Antoine.
- Previous message (by thread): [Python-Dev] Pure pickle bechmark.
- Next message (by thread): [Python-Dev] Improve test coverage for standard library
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]