Python Compilers Workshop (July 11-12, 2016) (original) (raw)

We’ll start at 9:30 am (Austin time, 14:30 UTC) on July 11in room 104 of theAT&T Center at UT Austin.

Basic facts

What: A workshop to bring together folks working on different approaches to high-performance compilation for Python, to share experience, discuss common interests, and start sketching out a roadmap for how the broader Python ecosystem can adapt to take maximal advantage of these new implementations.

When and where: Starting at 9:30 am, July 11-12, 2016 inAustin, Texas, co-located with theSciPy 2016 conference. (This is just before the main conference, and overlaps with the SciPy tutorial days.)

Venue: AT&T Center, room 104

Who: Open to the public; no registration fee.

Organized by: Nathaniel Smith(njs@pobox.com)

Motivation

There’s an intense and growing interest in techniques for compiling Python or near-Python languages to native code; a partial list includes PyPy, Numba,Pyston,Cython, Jython,Nuitka,Pythran,Pyjion,Numexpr,HOPE,GT-Py…

Now seems like a good time to compare notes! Plus there are lots of questions that seem like they could benefit from some cross-project collaboration. Our exact agenda will depend on participants’ interests, but here are some examples to give the flavor:

total = 0.0  
for num in range(10000):  
    total += num

and perform inlining, unboxing, etc. to compile it down to a tight native loop that far out-performs CPython. This is possible because they have intimate knowledge of built-in Python constructs likefloat, int, and range. And a number of them have good enough CPython C API compatibility that they can – or will soon be able to – execute the same code, but using numpy: 

import numpy as np  
total = 0.0  
for num in np.arange(10000):  
    total += num

But if you’re using the CPython C API to call numpy, then you can’t do unboxing/inlining/loop fusion/etc., because numpy objects are a total black box to the compiler, and so this numpyified version of the loop will run at about the same speed in a state-of-the-art JIT as it would in CPython.
The traditional way to solve this – as seen in e.g. Numba or Numexpr – is to give the compiler built-in knowledge of numpytypes and operations, essentially reimplementing numpy inside each compiler. But this strategy has a number of obvious downsides in terms of duplicated effort, subtle incompatibilities, increased testing load for downstream projects, reduced ability to evolve and improve numpy’s semantics, and the potential need to then repeat the whole exercise for other projects likedynd (a numpy competitor) orpandas. And this question is becoming particularly urgent, since not only has Numba already started down this road, but Pyston and PyPy are both working on passing the numpy test suite right now and are likely to soon move from worrying about correctness to worrying about speed.
Can we do better? Could there be some way to write a library likenumpy so that a single codebase could simultaneously target CPython and the newer compilers, while achieving competitive speed in all cases? If so, what would it take to make that happen? If not, then what’s the next-best alternative?

Schedule

TBD – probably we’ll start with some talks from different projects to outline their approaches and name some problems they’re worrying about, and then switch to unconference mode.

A partial list of confirmed attendees

We’re grateful to our sponsors for helping make this event possible:

Intel

We’re also very grateful tothe SciPy organizers(especially Jill Cowan) for providing space and other assistance.

Thanks!

Travel information

See the SciPy 2016 web site for suggestions onlodging,transportation, and other travel details.

Code of conduct

Workshop attendees are expected to adhere to theSciPy code of conduct. Please report any violations or concerns toNathaniel Smith or a member of SciPy staff.