| msg14422 - (view) |
Author: Ralf W. Grosse-Kunstleve (rwgk) |
Date: 2003-02-04 19:55 |
| We are having problems using Boost.Python with Python 2.3a1 because under some circumstances the result of __module__ is different compared to earlier Python versions: Python 2.2.1 (#2, Jun 17 2002, 12:06:51) [GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import simple >>> simple.empty.__module__ 'simple' >>> Python 2.3a1 (#1, Jan 6 2003, 14:17:56) [GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import simple >>> simple.empty.__module__ '__main__' >>> Because of this we can no longer pickle our extension classes. For your reference the code for the simple module is attached. This is using Boost release 1.29.0 (www.boost.org). We have done some debugging. Boost.Python's internal idea of the module associated with an extension class is still correct even when using Python 2.3a1. David Abrahams (main Boost.Python author) is telling me that he "changed Boost.Python to work the way Guido suggested before 2.2.2." Therefore we suspect that the __module__ problem is due to a change/bug in Python 2.3a1. Ralf #include <boost/python/module.hpp> #include <boost/python/class.hpp> namespace sandbx { namespace { struct empty {}; void init_module() { using namespace boost::python; class_("empty"); } }} // namespace sandbx:: BOOST_PYTHON_MODULE(simple) { sandbx::init_module(); } |
|
|
| msg14423 - (view) |
Author: Michael Hudson (mwh)  |
Date: 2003-02-05 14:10 |
| Logged In: YES user_id=6656 There has certainly been a change here. Look at typeobject.c:type_module() (about 100 lines in). If you can tell me what you want that function to do, I can have a go at it. Is simple.empty a HEAPTYPE? |
|
|
| msg14424 - (view) |
Author: Ralf W. Grosse-Kunstleve (rwgk) |
Date: 2003-02-05 18:25 |
| Logged In: YES user_id=71407 > Look at typeobject.c:type_module() (about 100 lines in). > > Is simple.empty a HEAPTYPE? Yes, it is a HEAPTYPE. I've established this by adding print statements in typeobject.c:type_module(): static PyObject * type_module(PyTypeObject *type, void *context) { PyObject *mod; char *s; if (type->tp_flags & Py_TPFLAGS_HEAPTYPE) { printf("IS HEAPTYPE\n"); mod = PyDict_GetItemString(type- >tp_dict, "__module__"); Py_XINCREF(mod); s = PyString_AsString(mod); printf("type->tp_dict, __module__ = %s\n", s); return mod; } Result: Python 2.3a1 (#2, Feb 5 2003, 09:39:30) [GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import simple >>> simple.empty.__module__ IS HEAPTYPE type->tp_dict, __module__ = __main__ '__main__' >>> > If you can tell me what you want that function to do, I can > have a go at it. We expect "simple" as the result of simple.empty.__module__. Current result if simple.so is moved to a package: Python 2.3a1 (#2, Feb 5 2003, 09:39:30) [GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from sandbx import simple >>> simple.empty.__module__ IS HEAPTYPE type->tp_dict, __module__ = __main__ '__main__' >>> Here we expect "sandbx.simple" . Python 2.2.x produces the expected results using the exact same Boost.Python source code. David Abrahams provides this additional information: Remember that simple.empty is created by calling the metatype, so it's created on the heap, but it's Python's internal type creation mechanisms which do it -- we're not setting the flags manually. Thanks! Ralf |
|
|
| msg14425 - (view) |
Author: Michael Hudson (mwh) |
Date: 2003-02-06 11:22 |
| Logged In: YES user_id=6656 OK, thanks for the info. More questions, I'm afraid: In 2.2.x, how are you telling Python that __module__ should be "simple"? By setting tp_name to "simple.empty"? What's simple.__name__? I think I know how __module__ is getting there, it's the chunk of code currently at typeobject.c:1750. Can you point me to the bit of the boost source that creates the typeobject? 'cept sf's just fallen off the net. |
|
|
| msg14426 - (view) |
Author: Ralf W. Grosse-Kunstleve (rwgk) |
Date: 2003-02-06 18:36 |
| Logged In: YES user_id=71407 > In 2.2.x, how are you telling Python that __module__ should > be "simple"? By setting tp_name to "simple.empty"? What's > simple.__name__? Python 2.3a1 (#2, Feb 5 2003, 09:39:30) [GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import sandbx.simple >>> sandbx.simple.__name__ 'sandbx.simple' >>> sandbx.simple.empty.__name__ 'sandbx.simple.empty' >>> sandbx.simple.empty.__module__ IS HEAPTYPE type->tp_dict, __module__ = __main__ '__main__' >>> > I think I know how __module__ is getting there, it's the > chunk of code currently at typeobject.c:1750. Yes, that's right. I've verified this by adding more print statements. > Can you point me to the bit of the boost source that creates > the typeobject? Here is where it happens (file boost/libs/python/src/object/class.cpp): object result = object(class_metatype())(module_prefix() + name, bases, d); With print statements I've verified that module_prefix() + name is correctly passed in as "sandbx.simple.empty" . I am attaching the file class.cpp. Could it be that we have to update our class_metatype_object? Thanks! Ralf |
|
|
| msg14427 - (view) |
Author: Michael Hudson (mwh) |
Date: 2003-02-07 11:29 |
| Logged In: YES user_id=6656 Ah, I start to remember the details now. In 2.2, if tp_name contained a period, everything to the left of the period was __module__, that to the right __name__. However, at some point (maybe even by 2.3) we want class X: class Y: pass print X.Y.__name__ to print 'X.Y' (it prints 'Y' today). Also we made __name__ assignable for 2.3 and this behaviour caused confusion. So we made things so that the '__module__' entry in the type's dict always wins. In summary, if you poke a '__module__' key into the boost::python::dict object you pass to the call of the metatype and change the first argument to just 'name', I think you'll be set. I pretty sure this will also work with 2.2.x, but if I were you I'd check. HTH! |
|
|
| msg14428 - (view) |
Author: Ralf W. Grosse-Kunstleve (rwgk) |
Date: 2003-02-07 17:44 |
| Logged In: YES user_id=71407 > In summary, if you poke a '__module__' key into the > boost::python::dict object you pass to the call of the > metatype and change the first argument to just 'name', I > think you'll be set. Yes, that works! > I pretty sure this will also work with 2.2.x, but if I were > you I'd check. I've tested with Python 2.2, 2.2.1, 2.2.2 and 2.3a1. The same code works perfectly with any of these releases. Thank you very much for your help! Ralf |
|
|
| msg14429 - (view) |
Author: Michael Hudson (mwh) |
Date: 2003-02-07 17:48 |
| Logged In: YES user_id=6656 No problem. Glad it got sorted. I'll leave this open, but mark it as a doc bug -- there should probably be something in whatsnew about this, and probably somewhere in the reference docs, but I've no idea where. |
|
|
| msg14430 - (view) |
Author: Michael Hudson (mwh) |
Date: 2003-02-11 14:27 |
| Logged In: YES user_id=6656 OK, I hope this is adequately documented in revision 1.21 of Doc/api/newtypes.tex revision 1.118 of Doc/whatsnew/whatsnew23.tex Closing. |
|
|