| msg188317 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) *  |
Date: 2013-05-03 18:16 |
| When C implementation of the unpickle reads a data, it also prefetches some data using peek() (if available) and then concatenates read and peeked chunks in the one input buffer. This causes an additional copying when a large data is read. The proposed patch gets rid of concatenating by moving a peeking before reading. |
|
|
| msg188335 - (view) |
Author: Alexandre Vassalotti (alexandre.vassalotti) * |
Date: 2013-05-03 23:50 |
| Do you have benchmark results to show the code with the patch is faster? |
|
|
| msg188343 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2013-05-04 08:08 |
| No, I have no any interesting results. ;) If an effect exists, it should reveal itself only in rare cases and be very small. This patch may be considered rather as a tiny refactoring (it decreases a number of code lines). |
|
|
| msg193212 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2013-07-17 08:48 |
| See for test case. |
|
|
| msg193214 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2013-07-17 08:52 |
| I'd prefer to see efforts go towards finishing the PEP 3154 implementation... |
|
|
| msg193215 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2013-07-17 09:04 |
| This patch is orthogonal to the PEP 3154 framing. The affecting of is only an unexpected side effect. |
|
|
| msg204832 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2013-11-30 18:09 |
| Patch is synchronized with tip (it was desynchronized since 23459df0753e). |
|
|
| msg204833 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2013-11-30 18:32 |
| Microbenchmark: $ ./python -c "import pickle; d = b'x' * 10**6; f = open('test.pickle3', 'wb'); pickle.dump(d, f, 3); f.close()" $ ./python -m timeit -s "from pickle import load" "with open('test.pickle3', 'rb') as f: load(f)" Unpatched: 100 loops, best of 3: 7.27 msec per loop Patched: 100 loops, best of 3: 4.87 msec per loop |
|
|
| msg204843 - (view) |
Author: Alexandre Vassalotti (alexandre.vassalotti) * |
Date: 2013-11-30 20:53 |
| Looks good to me! Feel free to commit. |
|
|
| msg204846 - (view) |
Author: Roundup Robot (python-dev)  |
Date: 2013-11-30 21:16 |
| New changeset d565310e7ae3 by Serhiy Storchaka in branch 'default': Issue #17897: Optimized unpickle prefetching. http://hg.python.org/cpython/rev/d565310e7ae3 |
|
|
| msg204847 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) * |
Date: 2013-11-30 21:17 |
| Thank you Alexandre. |
|
|