[Python-Dev] PEP 441 - Improving Python ZIP Application Support (original) (raw)

Thomas Wouters thomas at python.org
Sun Feb 15 17:21:08 CET 2015

FWIW, we have a similar tool to 'pyzaa' at Google, although a lot older (it has a pure-python implementation of zipimport, because it had to work with Python 2.2 back in the day) that does support extension modules and other shared libraries (by extracting them to disk on program startup.) We're also working to replace that by loading the extension modules directly from the archive (using dlopen_with_offset as described by ppluzhnikov in https://sourceware.org/bugzilla/show_bug.cgi?id=11767, although it should also be possible using dlopen_ehdr/dlopen_phdr), which requires that extension modules are stored uncompressed (simple) and page-aligned (harder, as the zipfile.ZipFile class doesn't directly support page-aligning anything, but it turns out it's easy to hack around by overriding _writecheck :P) And yes, we really are in a position to modify glibc to make our life distributing Python applications easier, internally; the old code involves shell and Python bootstrap code that tries very hard to avoid race conditions and security problems, which is a pain to maintain.

It might be nice to consider those use-cases in pyzapp as well, especially once the glibc feature is released. It requires some fairly big changes to zipimport (I ended up rewriting the whole thing) but we can easily opensource the code, if there's any interest at all. I'd be happy to discuss this in more detail at PyCon, at which time we should be deploying code internally using all of this.

-- Thomas Wouters <thomas at python.org>

Hi! I'm an email virus! Think twice before sending your email to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20150215/f8a5d2b7/attachment.html>

More information about the Python-Dev mailing list