(original) (raw)

FWIW, we have a similar tool to 'pyzaa' at Google, although a lot older (it has a pure-python implementation of zipimport, because it had to work with Python 2.2 back in the day) that \*does\* support extension modules and other shared libraries (by extracting them to disk on program startup.) We're also working to replace that by loading the extension modules directly from the archive (using dlopen\_with\_offset as described by ppluzhnikov in https://sourceware.org/bugzilla/show\_bug.cgi?id=11767, although it should also be possible using dlopen\_ehdr/dlopen\_phdr), which requires that extension modules are stored uncompressed (simple) and page-aligned (harder, as the zipfile.ZipFile class doesn't directly support page-aligning anything, but it turns out it's easy to hack around by overriding \_writecheck :P) And yes, we really are in a position to modify glibc to make our life distributing Python applications easier, internally; the old code involves shell and Python bootstrap code that tries very hard to avoid race conditions and security problems, which is a pain to maintain.

It might be nice to consider those use-cases in pyzapp as well, especially once the glibc feature is released. It requires some fairly big changes to zipimport (I ended up rewriting the whole thing) but we can easily opensource the code, if there's any interest at all. I'd be happy to discuss this in more detail at PyCon, at which time we should be deploying code internally using all of this.

--