Towards a solution for rvalue holder arguments in wrapped functions by rhaschke · Pull Request #2046 · pybind/pybind11 (original) (raw)

In #2040 I was complaining about a fundamental issue in pybind11, namely that rvalue function arguments (of custom types and particularly of std::unique_ptr) are not supported, i.e. the following function cannot be wrapped:

void consume(std::unique_ptr&& holder) { CustomType sink(std::move(*holder.release()); }

As this is a major blocker for my project to transition from boost::python to pybind11, I tried to investigate the issue in more detail.
It looks like, that a major design decision was to explicitly forbid transferring ownership from python to C++. This allows maintaining a raw value pointer (in addition to the holder) for fast and direct access to the raw pointer. (I noticed that the holder's get() is only rarely called.)

If you are not willing to support this important use case, you should explicitly state this fact very prominently on some main documentation page(s). However, I hope that you will add this feature in the future as pybind11 is a great replacement for boost::python, providing many benefits and exhibiting much better code readability.

The above-mentioned design decision becomes noticeable - amongst other things - by the fact that move_only_holder_casters do not support loading of arguments and thus conversion from python to C++ is not possible for move-only holder types.
The commits of this PR replace the existing implementation of move_only_holder_caster with that of copyable_holder_caster (with some minor tweaks), which essentially enables the requested feature.
However, a major caveat remains: The holder pointer remains accessible after moving, while it should become None and thus raise an exception if accessed again. I tried to accomplish this here, but it didn't have the desired effect - I guess because pybind11 mainly accesses object instances by their raw value pointers instead of going through the holder.

Another major caveat comes from the overload-resolution process: During this process, several function overloads are tried to match in sequence, each one requiring to load the python arguments into the type_caster. However, this loading process already moves the holder into the caster and thus is only possible once.