[Python-Dev] Accepting PEP 3154 for 3.4? (original) (raw)

Antoine Pitrou solipsis at pitrou.net
Wed Nov 20 01:10:50 CET 2013

On Wed, 20 Nov 2013 00:56:13 +0100 "Martin v. Löwis" <martin at v.loewis.de> wrote:

AFAICT, the real driving force is the desire to not read-ahead more than the pickle is long. This is what complicates the code. The easiest (and most space-efficient) solution to that problem would be to prefix the entire pickle with a data size field (possibly in a variable-length representation), i.e. to make a single frame.

Pickling then becomes very problematic: you have to keep the entire pickle in memory until the end, when you finally can write the size at the beginning of the pickle.

If that was done, I would guess that Tim's concerns about brittleness would go away (as you couldn't have a length field in the middle of data). IMO, the PEP has nearly the same flaw as the HTTP chunked transfer, which also puts length fields in the middle of the payload (except that HTTP makes it worse by making them optional).

Tim's concern is easily addressed with a FRAME opcode, without changing the overall scheme (as he lately proposed).

Regards

Antoine.

More information about the Python-Dev mailing list