[Python-Dev] Accepting PEP 3154 for 3.4? (original) (raw)

Guido van Rossum guido at python.org
Tue Nov 19 20:26:44 CET 2013

Well, both fixed 8-byte framing and variable-size framing it introduces a new way of representing numbers in the stream, which means that everyone parsing and generating pickles must be able to support both styles. (But fixed is easier since the XXX8 opcodes use the same format.)

I'm thinking of how you correctly read a pickle from a non-buffering pipe with the minimum number of read() calls without ever reading beyond the end of a valid pickle. (That's a requirement, right?)

If you know it's protocol 4: with fixed framing: read 10 bytes, that's the magic word plus first frame; then you can start buffering with variable framing: read 3 bytes, then depending on the 3rd byte read some more to find the frame size; then you can start buffering with mandatory frame opcode: pretty much the same with optional frame opcode: pretty much the same (the 3rd byte must be a valid opcode, even if it isn't a frame opcode)

if you don't know the protocol number: read the first byte, then read the second byte (or not if it's not explicitly versioned), then you know the protocol and can do the rest as above

On Tue, Nov 19, 2013 at 11:11 AM, Antoine Pitrou <solipsis at pitrou.net>wrote:

On Tue, 19 Nov 2013 11:05:45 -0800 Guido van Rossum <guido at python.org> wrote:

> So using an opcode for framing is out? (Sorry, I've lost track of the > back-and-forth.) It doesn't seem to bring anything, and it makes the overhead worse for tiny pickles (since it will be two bytes at least, instead of one byte with the current variable length encoding proposal). If overhead doesn't matter, I'm fine with keeping a simple 8-bytes frame size :-) Regards Antoine.

-- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20131119/fe342929/attachment.html>

More information about the Python-Dev mailing list