[Python-Dev] C-level duck typing (original) (raw)

Dag Sverre Seljebotn d.s.seljebotn at astro.uio.no
Wed May 16 12:48:19 CEST 2012

On 05/16/2012 11:50 AM, "Martin v. Löwis" wrote:

Agreed in general, but in this case, it's really not that easy. A C function call involves a certain overhead all by itself, so calling into the C-API multiple times may be substantially more costly than, say, calling through a function pointer once and then running over a returned C array comparing numbers. And definitely way more costly than running over an array that the type struct points to directly. We are not talking about hundreds of entries here, just a few. A linear scan in 64 bit steps over something like a hundred bytes in the L1 cache should hardly be measurable. I give up, then. I fail to understand the problem. Apparently, you want to do something with the value you get from this lookup operation, but that something won't involve function calls (or else the function call overhead for the lookup wouldn't be relevant).

In our specific case the value would be an offset added to the PyObject*, and there we would find a pointer to a C function (together with a 64-bit signature), and calling that C function (after checking the 64 bit signature) is our final objective.

I still think this is out of scope for python-dev. If this is something you want to be able to do for Python 2.4 as well, then you don't need any change to Python - you can do whatever you come up with for all Python versions, no need to (or point in) changing Python 3.4 (say).

We can go ahead and hijack tp_flags bit 22 to make things work in existing versions. But what if Python 3.8 then starts using that bit for something else?

As this is apparently only relevant to speed fanatics, too, I suggest that you check how fast PyPI works for you. Supposedly, they have very efficient lookup procedures, supported by the JIT. If this doesn't work for some reason, I suggest that you'll have to trade speed for convenience: a compile-time fixed layout will beat any dynamic lookup any time. Just define a common base class, and have all interesting types inherit from it.

Did you mean PyPy? Me and Stefan are Cython developers, so that's kind of our angle... And I'm a Cython developer because it solves a practical need (in my case in scientific computation), not because I think it's that beautiful. PyPy won't work for me (let's not go down that road now...)

Defining a common base class is what NumPy already does, and Cython would be forced to without this proposal. We just think it has significant disadvantages and were looking for something else.

Dag

More information about the Python-Dev mailing list