bpo-35900: Add a state_setter arg to save_reduce by pierreglaser · Pull Request #12588 · python/cpython (original) (raw)
Second PR of bpo-35900.
What
The aim of this PR is to allow users to specify custom state setting methods for types they have no control on. This is done via adding a state_setter keyword argument in Pickler.save_reduce, expected to be a callable. This callable will be called at unpickling time with the following signature state_setter(obj, state, slotstate).
Why
A practical use case is the one of cloudpickle that provides extended serialization procedures, especially for functions and classes. In cloudpickle, functions and classes are reconstructed from scratch at unpickling time. As for other built-in types reconstructed from scratch (list, dicts...), circular-references are handled by initalizing a skeleton-object, and then updating it in a state-setting phase. This state-setting phase currently only relies on potential __getstate__ methods, or fallbacks to a default procedure, and this is simply not enough to properly reconstruct functions and classes, hence this proposal.
See cloudpipe/cloudpickle#253 for more details.
- implementaion
- unit-tests
Notes
dependencies
This PR comes alongside #12499 . Without it, leveraging #12499 changes won't work for functions and classes.
cross-protocol/version usage
An interrogation that remains is whether using this new API with a protocol < 5 should raise an error at pickling time. If not, unpickling will not succeed if the pickle string is loaded using a python < 3.8. This may be surprising for users as one can assume that the point of using low protocols is to ensure retro-compatibility with older python versions. On the other side, this API will probably be of interested for advanced user only, that would understand why using a new API with an old protocol. As of today, this is still an open question.