[Python-Dev] A new webpage promoting Compiler technology for CPython (original) (raw)

Stefan Behnel stefan_ml at behnel.de
Fri Feb 15 23:36:43 CET 2013

Paul Boddie, 15.02.2013 22:44:

Travis Oliphant wrote:

This page is specifically for Compiler projects that either integrate with or work directly with the CPython run-time which is why PyPy is not presently listed. The PyPy project is a great project but we just felt that we wanted to explicitly create a collection of links to compilation projects that are accessible from CPython which are likely less well known. I think that given the scope of the projects listed, it shouldn't preclude PyPy from being listed, really. After all, interoperability with CPython extensions is something of a focus area for PyPy: http://pypy.org/compat.html I don't have an agenda here - I don't use PyPy actively, my only involvement with Shedskin (which is referenced and which can produce CPython extension modules) is in packaging it for Debian, and although I do have a static analysis library I see no pressing need to promote it extensively - but I feel that when it comes to educational resources people should be a bit careful about drawing boundaries that exclude things that would benefit people substantially if they only knew about it. My reason for speaking up about this is that I've had to tell a room full of people who were told to use Cython, NumPy and even plain C to make their Python programs faster that PyPy existed. (Of course, one can justify ignoring the elephant in the room by claiming things like scientific users rely on native libraries or CPython extensions - since "science" is a very broad term, this obviously isn't universally true - but I think people should be entitled to make their own minds up, and I was not completely certain that everyone involved in the case in question was oblivious to PyPy's existence or status.)

This is off-topic for this list, but the main problem with PyPy is that you'll quickly hit a lot of walls when you try to use it for anything serious in the area. It's true that there is a certain level of interoperability with CPython extensions, but calling it a "focus area" makes it sound bigger than it actually is in my ears. Even trying to get bugs fixed to at least make things work at all often means running against walls on their side. I can tell, trying to port Cython mostly consisted of bugging PyPy developers to fix stuff, which took anything from days to still-not-done, each time. And, by design, PyPy makes it very hard and time consuming to debug it and to try to fix bugs in their code base.

So, while I agree that PyPy is worth a try in certain application areas, and can be helpful for some special needs, also in the field of scientific computing, it's lightyears away from a production-ready state in that area. It just doesn't integrate with the huge bulk of software that people use in their daily work. And once you rely on that software, which is hand tuned, well integrated and real-world proven in so many ways, over the time span of way more than a decade, the most visible advantage of PyPy to make Python code run faster becomes almost pointless. In that light, telling people to try PyPy and to drop (most of) their current ecosystem for it doesn't really sound helpful and clearly appears outside of the focus of the web site in question.

Stefan

More information about the Python-Dev mailing list