Issue 1338: pickling bytes? - Python tracker (original) (raw)

Issue1338

Created on 2007-10-26 19:06 by gvanrossum, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (6)
msg56809 - (view)	Author: Guido van Rossum (gvanrossum) *	Date: 2007-10-26 19:06
Alexandre Vassalotti suggested the following: A simple way to add specific pickling support for bytes/buffer objects would be to define two new constants: BYTES = b'\x8c' # push a bytes object BUFFER = b'\x8d' # push a buffer object And add the following pickling and unpickling procedures: def save_bytes(self, obj, pack=struct.pack): n = len(obj) self.write(BYTES + pack("<i", n) + obj) def save_buffer(self, obj, pack=struct.pack): n = len(obj) self.write(BUFFER + pack("<i", n) + obj) def load_bytes(self): len = mloads(b'i' + self.read(4)) self.append(self.read(len)) def load_buffer(self): len = mloads(b'i' + self.read(4)) self.append(buffer(self.read(len))) The only problem with this approach is that bytes object bigger than 4GB cannot be pickled. Currently, this applies to all string-like objects, so I don't think this restriction will cause any trouble. Also, it would be a good idea to bump the protocol version to 3 to ensure that older Python versions don't try to load pickle streams created with these new constants. By the way, would it be a good idea to add specific pickling support for sets (and frozensets)?
msg56812 - (view)	Author: Georg Brandl (georg.brandl) *	Date: 2007-10-26 19:28
Having explicit support for sets at least would be consistent with all other types for which there is a display form. (Or else, {1,2,3} could be a different thing after unpickling if the set builtin is overwritten.)
msg58056 - (view)	Author: Alexandre Vassalotti (alexandre.vassalotti) *	Date: 2007-12-01 17:15
Please assign this to me. I am planning to fix this, along a few other bugs, with my new revision of the pickle protocol.
msg58059 - (view)	Author: Georg Brandl (georg.brandl) *	Date: 2007-12-01 17:56
I added the Developer role to your roundup account, so you can now assign bugs to yourself. :)
msg58061 - (view)	Author: Alexandre Vassalotti (alexandre.vassalotti) *	Date: 2007-12-01 17:59
Thank you, Georg!
msg65862 - (view)	Author: Alexandre Vassalotti (alexandre.vassalotti) *	Date: 2008-04-27 01:21
Guido fixed this issue in r61467. Closing.