[Python-Dev] Accepting PEP 3154 for 3.4? (original) (raw)
Antoine Pitrou solipsis at pitrou.net
Tue Nov 19 00:57:08 CET 2013
- Previous message: [Python-Dev] Accepting PEP 3154 for 3.4?
- Next message: [Python-Dev] Accepting PEP 3154 for 3.4?
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
On Mon, 18 Nov 2013 16:44:59 -0600 Tim Peters <tim.peters at gmail.com> wrote:
[Tim] >> But it has a different kind of advantage: PREFETCH was optional. As >> Guido said, it's annoying to bloat the size of small pickles (which >> may, although individually small, occur in great numbers) by 8 bytes >> each. There's really no point to framing small chunks of data, right?
[Antoine] > You can't know how much space the pickle will take until the pickling > ends, though, which makes it difficult to decide whether you want to > emit a PREFETCH opcode or not. Ah, of course. Presumably the outgoing pickle stream is first stored in some memory buffer, right? If pickling completes before the buffer is first flushed, then you know exactly how large the entire pickle is. If "it's small" (say, < 100 bytes), don't write out the PREFETCH part. Else do.
Yet another possibility: keep framing but use a variable-length encoding for the frame size:
- first byte: bits 7-5: N (= frame size bytes length - 1)
- first byte: bits 4-0: first 5 bits of frame size
- remaning N bytes: remaining bits of frame size
With this scheme, very small pickles have a one byte overhead; small ones a two byte overhead; and the max frame size is 261 rather than 264, which should still be sufficient.
And the frame size is read using either one or two read() calls, which is efficient.
Regards
Antoine.
- Previous message: [Python-Dev] Accepting PEP 3154 for 3.4?
- Next message: [Python-Dev] Accepting PEP 3154 for 3.4?
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]