Issue 34204: Bump the default pickle protocol in shelve (original) (raw)

I wrote a short script to see the impact of file size depending on the protocol:

import shelve import os.path

print("== Short value ==") for proto in (0, 1, 2, 3, 4, 5): filename = 'shelve-picklev%s' % proto with shelve.open(filename, protocol=proto) as db: assert db._protocol == proto for x in range(1000): db[str(x)] = str(x) print(f'Protocol {proto}: {os.path.getsize(filename)} bytes') os.unlink(filename) print() print("== Large value ==") large_value = [str(x) for x in range(1000)] for proto in (0, 1, 2, 3, 4, 5): filename = 'shelve-picklev%s' % proto with shelve.open(filename, protocol=proto) as db: assert db._protocol == proto for x in range(10): db[str(x)] = large_value print(f'Protocol {proto}: {os.path.getsize(filename)} bytes') os.unlink(filename)

Output with Python 3.9.0b1 (on Fedora 32):

== Short value == Protocol 0: 90112 bytes Protocol 1: 94208 bytes Protocol 2: 94208 bytes Protocol 3: 94208 bytes Protocol 4: 94208 bytes Protocol 5: 94208 bytes

== Large value == Protocol 0: 139264 bytes Protocol 1: 139264 bytes Protocol 2: 139264 bytes Protocol 3: 139264 bytes Protocol 4: 98304 bytes Protocol 5: 98304 bytes

For short string values, protocol 0 produces smaller files than protocol 1 and higher.

For large value, protocol 4 and higher produce smaller files than protocol 3 and lower.