Address #1138 when mixing holder_type's upon casting an object. by EricCousineau-TRI · Pull Request #1139 · pybind/pybind11 (original) (raw)

The core issue in #1138 is that we don't check the holder type at all when initializing an instance: we just assume that a function returning a holder is returning the right kind of holder. I've just submitted #1161 to detect that and fail if it happens when attempting to register the type and/or function.

There are some other cases this PR doesn't catch that the other one does: for example returning a std::shared_ptr<A> that needs a std::unique_ptr<A>, or one that returns a boost::shared_ptr<A> to a type needing an std::shared_ptr<A>, or various other combinations.

More problematic, the change here would also break backwards-compatibility (and indeed breaks the current test suite) by requiring that any move-only holder adds a release() method.

Conversion between holder types--e.g. being able to convert an rvalue unique_ptr into a shared_ptr seems feasible, but I think it's going to need a different path than stealing the pointer and stashing it in the new type. std::shared_ptr has a std::unique_ptr && constructor for doing exactly this promotion, but we'd need a nice way to detect that case, and ideally, detect it at compile time rather than run-time.