[Python-Dev] Need advice, maybe support (original) (raw)

Guido van Rossum guido@python.org
Mon, 19 May 2003 16:47:39 -0400

Guido van Rossum wrote:

[me, about how to add nr cfunction versions in a compatible way] > I don't think we can just add an extra field to PyMethodDef, because > it would break binary incompatibility. Currently, in most cases, a > 3r party extension module compiled for an earlier Python version can > still be used with a later version. Because PyMethodDef is used as an > array, adding a field to it would break this. Bad news. I hoped you would break binary compatibility between major versions (like from 2.2 to 2.3), but well, now I also understand why there are so many flags in typeobjects :-) > I have less of a problem with extending PyTypeObject, it grows all the > time and the tpflags bits tell you how large the one you've got is. > (I still have some problems with this, because things that are of no > use to the regular Python core developers tend to either confuse them, > or be broken on a regular basis.) For the typeobjects, I'm simply asking for reservation of a bit number. What I used is #ifdef STACKLESS #define PyTPFLAGSHAVECALLNR (1L<<15)_ _#else_ _#define PyTPFLAGSHAVECALLNR 0_ _#endif_ _but I think nobody needs to know about this, and maybe_ _it is better (requiring no change of Python) if I used_ _a bit from the higer end (31) or such?_ _> Maybe you could get away with defining an alternative structure for > PyMethodDef and having a flag in tpflags say which it is; there are > plenty of unused bits and I don't mind reserving one for you. Then > you'd have to change all the code that uses tpmethods, but there > isn't much of that; in fact, the only place I see is in typeobject.c. The problem is that I need to give extra semantics to existing objects, which are PyCFunction objects. I think putting an extra bit into the type object doesn't help, unless I use a new type. But then I don't need the flag. An old extension module which is loaded into my Python will always use my PyCFunction, since this is always borrowed. > If this doesn't work for you, maybe you could somehow fold the two > implementation functions into one, and put something special in the > argument list to signal that the non-recursive version is wanted? > (Thinking aloud here -- I don't know exactly what the usage pattern of > the nr versions will be.) This is hard to do. I'm adding nr versions to existing functions, and I don't want to break their parameter lists.

Ok, what I did is rather efficient, quite a bit ugly of course, but binary compatible as much as possible. It required to steal some bits of mlflags as a small integer, which are interpreted as "distance to my sibling". I'm extending the MethodDef arrays in a special way by just adding some extra records without name fields at the end of the array, which hold the nr pointers. An initialization functions initializes the small integer in mlflags with the distance to this "sibling", and the nice thing about this is that it will never fail if not initialized: A distance of zero gives just the same record. So what I'm asking for in this case is a small number of bits of the mlflags word which will not be used, otherwise. Do you think the number of bits in mlflags might ever grow beyond 16, or should I just assume that I can safely abuse them? thanks a lot -- chris

It's better to reserve bits explicitly. Can you submit a patch to SF that makes reservations of the bits you need? All they need is a definition of a symbol and a comment explaining what it is for; "reserved for Stackless" is fine.

--Guido van Rossum (home page: http://www.python.org/~guido/)