| msg10274 - (view) |
Author: James Henstridge (jamesh) |
Date: 2002-04-12 13:18 |
| In bug #504343, I proposed a patch that made the __doc__ attribute of new style types to things other than strings. That bug was about unicode doc strings, but my main aim was to use a descriptor for __doc__ (the original bug I submitted was #507394, which got marked as a dup). The patch I proposed in that bug turned out to cause some problems for pydoc in another bit of code, as documented in bug #530070 (specifically, getting the help for __builtin__.property). The patch on that bug broke things again, so that my classes (which were non heap type classes defined in extension modules) started returning None when trying to get the documentation, because their tp_doc slot was NULL (they had a __doc__ in the type dict though). Additionally, for heap type classes, it appears that it won't call the tp_descr_get slot of the __doc__ object found in the module dictionary, so you end up getting the descriptor itself :( The sample program attached to bug #507394 (my first one on this topic) demonstrates part of the problem. With the current 2.2.1 candidate, for NewClass.__doc__, it returns the descriptor object itself. A translation of the type and descriptor to a C extension would demonstrate the problem in the non heap type case (I haven't written this test case yet). The development releases of pygtk can act as a test case though (running "import gtk; print gtk.Widget.__doc__"). Maybe the best fix would be to get pydoc to handle non string __doc__ by running str() on it, and ignoring the docstring if an exception occurs during the conversion. This would allow display of some unicode docstrings provided they could be represented in the system codeset (if you are using a utf8 system encoding, all unicode strings should be representable). I would like to see this working okay for python 2.2.1, but if the change is too big, maybe it could be moved out to python 2.3. I would like to see this issue considered. |
|
|
| msg10275 - (view) |
Author: Michael Hudson (mwh)  |
Date: 2002-04-12 13:35 |
| Logged In: YES user_id=6656 James, Python 2.2.1 was released two days ago. Nothing can change that now... |
|
|
| msg10276 - (view) |
Author: James Henstridge (jamesh) |
Date: 2002-04-12 13:37 |
| Logged In: YES user_id=146903 Here is a patch for inspect.py that fixes the bug reported in #530070 without breaking the use of descriptors for __doc__. I think this patch would be a good candidate for 2.2.1 (along with backing out the patch on #530070). I am sorry that I didn't bring this up earlier -- I hadn't realised there was any other patches to the __doc__ handling in the 2.2 maintenance branch. |
|
|
| msg10277 - (view) |
Author: James Henstridge (jamesh) |
Date: 2002-04-12 13:39 |
| Logged In: YES user_id=146903 damn. I was on the plane back from GUADEC then and am still catching up with things :( Sorry about that. |
|
|
| msg10278 - (view) |
Author: Martin v. Löwis (loewis) * |
Date: 2002-04-12 14:23 |
| Logged In: YES user_id=21627 Invoking str() on all doc strings has been considered and rejected; it is desirable that Unicode doc strings come out as-is. I'm not sure what kind of patch you are suggesting; if I merely revert typeobject.c 2.130 (i.e. removing the type.__doc__ getter), then property.__doc__ will not be the doc string of property, but the __doc__ member. This is undesirable. |
|
|
| msg10279 - (view) |
Author: Martin v. Löwis (loewis) * |
Date: 2002-04-12 14:33 |
| Logged In: YES user_id=21627 Also, if you want to have types with computed doc strings, couldn't you use a different metatype? |
|
|
| msg10280 - (view) |
Author: James Henstridge (jamesh) |
Date: 2002-04-12 15:06 |
| Logged In: YES user_id=146903 Maybe getting pydoc to ignore any __doc__ attributes (that is, what gets returned by getattr) that weren't strings or unicode would work. As for using a different metatype, a reason for getting this into the the standard metatype is that it is also an inconsistency between how new and old style classes act, and is fairly easy to reconcile (granted descriptors didn't exist in python < 2.2, but everytime new style classes act a bit differently, people seem to start submitting pygtk bug reports). |
|
|
| msg10281 - (view) |
Author: Martin v. Löwis (loewis) * |
Date: 2002-04-12 15:29 |
| Logged In: YES user_id=21627 This is not (primarily) a pydoc issue: with your change, property.__doc__ gets changed. pydoc could ignore the object returned, but then property would have no documentation at all - even though there is a tp_doc in it. So I don't see any easy solution. If you do, please attach a patch for the current CVS (we can then decide whether to backport it to 2.2.2). |
|
|
| msg10282 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2002-04-15 00:31 |
| Logged In: YES user_id=6380 James, can you please explain the problem again, but without referring to other existing patches? With all the references to pydoc, inspect, properties, descriptors, other patches and SF bug reports I have completely lost the trail, and I don't think it's productive for me to try and gather all the separate bits together. |
|
|
| msg10283 - (view) |
Author: James Henstridge (jamesh) |
Date: 2002-04-15 04:12 |
| Logged In: YES user_id=146903 Okay. Here is a summary of the story so far: I am working on the bindings for the GTK 2.0 gui toolkit. Gtk widgets can have properties (basically a combination of the name, type, some documentation and getters/setters; similar to python), and signals (part of gtk's notification framework). I wanted to provide documentation on the signals and properties of each widget, and putting them in the __doc__ attribute seemed natural. As information on the properties and signals can be read via gtk's introspection APIs, it seemed natural to want to generate this documentation at runtime. Now gtk tries to delay initialisation as much as possible (classes are only initialised on first use). So generating documentation for every class on module initialisation would mean initialising every class in gtk, which is a fairly big speed hit I don't want to incur -- especially since online documentation is mainly used during development. This is why I wanted to use a descriptor -- so that the signal and property documentation only gets generated when the user asks for it. Unfortunately this didn't work in python 2.2, as the type.__doc__ member descriptor (which returns tp_doc) would take precedence over the __doc__ attribute in the class itself. To me, this seemed like a needless difference between new and old style classes. My initial patch made __doc__ act more like the old style classes by getting rid of the type.__doc__ descriptor and getting PyType_Ready to set __doc__ appropriately from tp_doc, if it hadn't already been set. This allowed me to use descriptors to generate signal/property documentation, and also allowed other non string __doc__ attributes (such as unicode strings). This fix was applied to head and the 2.2 branch but unfortunately, it also caused pydoc to raise an exception when displaying the documentation for __builtin__.property, as its __doc__ attribute was now being returned when you asked for property.__doc__ (rather than the type.__doc__ descriptor taking precedence and returning tp_doc). property.__doc__ is a descriptor which returns a string for property instances, but returns itself in the class context (ie. getting property.__doc__). As this is not a string or unicode string, pydoc gets an error. The fix (again, checked into both head and 2.2 branches) that was checked in added a __doc__ descriptor again. This descriptor returns tp_doc for non heap type classes, and looks up __doc__ in the class dictionary for heap type classes. This causes problems for both heap and non heap classes: for non heap classes, __doc__ in the class dict is never accessed. For heap classes, the tp_descr_get method of __doc__ is never called. Here is my prefered way to fix the problem (which you may disagree with): 1. remove the type.__doc__ descriptor again 2. make the property.__doc__ descriptor return something meaningful in the class context 3. make pydoc handle non string-like __doc__ attributes better (either ignoring them or converting them to strings). Currently I can get help() to error out with a simple class like: class foo: __doc__ = 42 (I am not saying that people should use integers as __doc__; just that pydoc should handle things like this more gracefully). |
|
|
| msg10284 - (view) |
Author: Martin v. Löwis (loewis) * |
Date: 2002-04-15 11:09 |
| Logged In: YES user_id=21627 I would summarize this differently: For the attached descr.py, Python currently prints in the third line NewClass.__doc__ = <__main__.DocDescr object at 0x1094d0> James requests that it prints NewClass.__doc__ = 'object=None type=NewClass' instead. |
|
|
| msg10285 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2002-04-15 13:38 |
| Logged In: YES user_id=6380 I like Martin's summary better. :-) Some remarks: - property.__doc__ works fine, it's property.__dict__['__doc__'] that behaves strange (but nobody should care) - I can't get help() to break by setting __doc__ = 42 on either an old or a new class - I think I'd rather not lose the __doc__ descriptor in type, but I can make it look for __dict__['__doc__'] first, *and* call that if it appears a descriptor. I'll post a patch here once I've had breakfast. |
|
|
| msg10286 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2002-04-18 00:19 |
| Logged In: YES user_id=6380 After some email exchanged with James, I've decided to check something in that does what he wants, makes Martin's example work, and doesn't break property.__doc__, without having to change the property implementation. Coming soon to a CVS HEAD near you. I'll check this into the 2.2 maint branch too. |
|
|