Issue 25710: zipimport is not PEP 3147 or PEP 488 compliant (original) (raw)

Issue25710

Created on 2015-11-23 15:44 by byrnes, last changed 2022-04-11 14:58 by admin.

Messages (4)
msg255178 - (view)	Author: Robert Byrnes (byrnes)	Date: 2015-11-23 15:44
zipimport is not PEP 3147 compliant: i.e., it looks for foo.pyc (in the same directory as foo.py) instead of __pycache__/foo.cpython-35.pyc. This is counterintuitive, and unfortunate because it means that installation directories (that obey PEP 3147 conventions) can't be zip archived from filesystems and then used by zipimport. When support for PEP 488 was added to Python 3.5, zipimport was modified to just delete references to .pyo files, but it needs to know about the new .opt-[12] filename components (as well as the PEP 3147 version tags). As far as I can tell, the current version of zipimport can't import optimized bytecode files, using either the old or new filename conventions. Finally, none of this behavior is documented. The zipimport docs still mention .pyo files (which were eliminated by PEP 488), and say nothing about the filename conventions expected within zip archives.
msg255184 - (view)	Author: Brett Cannon (brett.cannon) *	Date: 2015-11-23 16:29
Not finding bytecode files is not that big of a deal. While it's a very minor performance loss, it isn't critical to how Python works (and since it looks for bytecode-only files that use-case isn't broken either). But the real problem is that zipimport is such a nasty chunk of code that no one wants to fix it. Yet more evidence we just need to rewrite the whole module. I created issue #25711 to track a rewrite.
msg259446 - (view)	Author: Łukasz Langa (lukasz.langa) *	Date: 2016-02-02 23:22
If you're affected, there's a workaround. Use the following script to rewrite paths in your .zip file to be zipimport-compatible: https://gist.github.com/ambv/909d38bdac4f3e719b7c It preserves the preamble and all file metadata. A little context why this is useful: for archives that are effectively "static" environments with entire dependency trees in them, this bug causes both a big slowdown in startup time (counted in seconds), as well as tens of megabytes of .pyc created on the fly. Examples include zipapp-style .pyz, Twitter's .pex or Facebook's .par. So while we're waiting for a new zipimport implementation, the workaround listed above should unblock people uzing this form of Python package distribution.
msg325732 - (view)	Author: Serhiy Storchaka (serhiy.storchaka) *	Date: 2018-09-19 07:57
zipimport has been rewritten in pure Python ().