[Python-Dev] Let's change to C API! (original) (raw)

Antoine Pitrou solipsis at pitrou.net
Tue Jul 31 02:58:16 EDT 2018

On Tue, 31 Jul 2018 02:29:42 +0200 Victor Stinner <vstinner at redhat.com> wrote:

>> Last year, I gave a talk at the Language Summit (during Pycon) to >> explain that CPython should become 2x faster to remain competitive. >> IMHO all attempts to optimize Python (CPython forks) have failed >> because they have been blocked by the C API which implies strict >> constraints. > > Well, that's your opinion, but did you prove it?

I discussed with many Python developers who agree with me that the C API blocked CPython forks. For example, I heard that Pyston was very fast and very promising before starting to support the C API.

What exactly in the C API made it slow or non-promising?

The C API requires that your implementations make almost all the same design choices that CPython made 25 years ago (C structures, memory allocators, reference couting, specific GC implementation, GIL, etc.).

Yes, but those choices are not necessarily bad.

Multiple PyPy developers told me that cpyext remains a blocker issue to use PyPy.

Probably, but we're talking about speeding up CPython here, right?

If we're talking about making more C extensions PyPy-compatible, that's a different discussion, and one where I think Stefan is right that we should push people towards Cython and alternatives, rather than direct use of the C API (which people often fail to use correctly, in my experience).

But the C API is still useful for specialized uses, including for development tools such as Cython.

Oh, by the way, I will not promise anything about any potential performance gain. When I write "2x faster", I mean that our current approach for optimization failed to make Python 2x faster over the last 10 years. Python 3 is more or less as fast, or a little bit faster, than Python 2. But Python 2 must not be used as an example of performance. People hesitate between Go, Javascript and Python. And Python is not the winner in term of performance.

I agree about the overall diagnosis. I just disagree that changing the C API will open up easy optimization opportunities.

Actually I'd like to see a list of optimizations that you think are held up by the C API.

I have to confess that helping Larry is part of my overall plan.

Which is why I'd like to see Larry chime in here.

Regards

Antoine.

More information about the Python-Dev mailing list