Issue 23344: Faster marshalling - Python tracker (original) (raw)

Currently writing marshalled data to buffer is not very efficient. Data is written byte by byte with testing conditions p->fp != NULL and p->ptr != p->end for every byte. Proposed patch makes writing to buffer faster.

Benchmark results:

$ ./python -m timeit -s "import marshal; d = compile(open('Lib/_pydecimal.py').read(), '_pydecimal.py', 'exec')" -- "marshal.dumps(d)" Unpatched: 100 loops, best of 3: 4.64 msec per loop Patched: 100 loops, best of 3: 3.39 msec per loop

$ ./python -m timeit -s "import marshal; a = ['%010x' % i for i in range(10**4)]" -- "marshal.dumps(a)" Unpatched: 1000 loops, best of 3: 1.96 msec per loop Patched: 1000 loops, best of 3: 1.32 msec per loop

$ ./python -m timeit -s "import marshal; a = ['%0100x' % i for i in range(10**4)]" -- "marshal.dumps(a)" Unpatched: 100 loops, best of 3: 10.3 msec per loop Patched: 100 loops, best of 3: 3.39 msec per loop