(original) (raw)

...

\=======

Mutability
\----------

PEP 3118 buffers \[#pep-3118\]\_ can be readonly or writable. Some objects,
such as Numpy arrays, need to be backed by a mutable buffer for full
operation. Pickle consumers that use the \`\`buffer\_callback\`\` and \`\`buffers\`\`
arguments will have to be careful to recreate mutable buffers. When doing
I/O, this implies using buffer-passing API variants such as \`\`readinto\`\`
(which are also often preferrable for performance).

Data sharing
\------------

If you pickle and then unpickle an object in the same process, passing
out-of-band buffer views, then the unpickled object may be backed by the
same buffer as the original pickled object.

For example, it might be reasonable to implement reduction of a Numpy array
as follows (crucial metadata such as shapes is omitted for simplicity)::

class ndarray:

def \_\_reduce\_ex\_\_(self, protocol):
if protocol == 5:
return numpy.frombuffer, (PickleBuffer(self), self.dtype)
\# Legacy code for earlier protocols omitted

Then simply passing the PickleBuffer around from \`\`dumps\`\` to \`\`loads\`\`
will produce a new Numpy array sharing the same underlying memory as the
original Numpy object (and, incidentally, keeping it alive)::