[Python-Dev] Use C extensions compiled in release mode on a Python compiled in debug mode (original) (raw)

Victor Stinner vstinner at redhat.com
Mon Apr 29 20:22:59 EDT 2019

You have my support is you work on removing static types :-)

Here are my notes on the current C APIs to define a type: https://pythoncapi.readthedocs.io/type_object.html

IMHO static types should go away in the long term. They are causing too many practical issues.

Victor

Le mar. 30 avr. 2019 à 02:01, Neil Schemenauer <nas-python at arctrix.com> a écrit :

On 2019-04-27, Nathaniel Smith wrote: > For PyTRACEREFS specifically, IIUC the only goal is to be able to produce > a list of all live objects on demand. If that's the goal, then static type > objects aren't a huge deal. You can't add extra data into the type objects > themselves, but since there's a fixed set of them and they're immortal, you > can just build a static list of all of them in PyTypeReady. As far as I understand, we have a similar problem already for gc.getobjects() because those static type objects don't have a PyGCHead. My 2-cent proposal for fixing things in the long term would be to introduce a function like PyTypeReady that returns a pointer to the new type. The argument to it would be what is the current static type structure. The function would copy things from the static type structure into a newly allocated type structure. We have a kind of solution already with PyTypeFromSpec, etc. However, I think it is harder to convert existing extension module source code to use that API. We want to make it very easy for people to fix source code. If we can remove static types, that would allow us to kill off PyTYPE(o)->tpisgc(o). I understand why that exists but I think it is quite an ugly detail of the current GC implementation. I wonder about the performance impact of it given current memory latencies. When we do a full GC run, we call PyObjectISGC() on many objects. I fear having to lookup and call tpisgc could be quite expensive. I've been playing with the idea of using memory bitmaps rather then the PyGCHead. That idea seems to depend on removing static type objects. Initially I was thinking of it as reducing the memory overhead for GC types. Now I think the memory overhead doesn't matter too much but perhaps the bitmaps would be much faster due to memory latency. There is an interesting Youtube video that compares vector traversals vs linked list traversals in C++. Linked lists on modern machines are really terrible. Regards, Neil

-- Night gathers, and now my watch begins. It shall not end until my death.

More information about the Python-Dev mailing list