[Python-Dev] Does Zip Importer have to be Special? (original) (raw)

Brett Cannon brett at python.org
Thu Jul 24 20:26:21 CEST 2014

On Thu Jul 24 2014 at 2:12:20 PM, Phil Thompson <phil at riverbankcomputing.com> wrote:

On 24/07/2014 6:48 pm, Brett Cannon wrote: > On Thu Jul 24 2014 at 1:07:12 PM, Phil Thompson > <phil at riverbankcomputing.com> > wrote: > >> I have an importer for use in applications that embed an interpreter >> that does a similar job to the Zip importer (except that the storage >> is >> a C data structure rather than a .zip file). Just like the Zip >> importer >> I need to import my importer and add it to sys.pathhooks. However the >> earliest opportunity I have to do this is after the PyInitialize() >> call >> returns - but this is too late because some parts of the standard >> library have already needed to be imported. >> >> My current workaround is to include a modified version of >> bootstrap.py >> as a frozen module that has the necessary steps added to the end of >> its >> install() function. >> >> The Zip importer doesn't have this problem because it gets special >> treatment - the call to its equivalent code is hard-coded and happens >> exactly when needed. >> >> What would help is a table of functions that were called where >> PyImportZipInit() is currently called. By default the only entry in >> the table would be PyImportZipInit. There would be a way of >> modifying >> the table, either like how PyImportFrozenModules is handled or how >> Inittab is handled. >> >> ...or if there is a better solution that I have missed that doesn't >> require a modified bootstrap.py. >> > > Basically you want a way to specify arguments into > importlib.bootstrap.install() so that sys.pathhooks and > sys.metapath > were configurable instead of hard-coded (it could also be done just > past > importlib being installed, but that's a minor detail). Either way there > is > technically no reason not to allow for it, just lack of motivation > since > this would only come up for people who embed the interpreter AND have a > custom importer which affects loading the stdlib as well (any reason > you > can't freeze the stdblib as a solution?).

Not really. I'd lose the compression my importer implements. (Are there any problems with freezing packages rather than simple modules?)

Nope, modules and packages are both supported.

> We could go the route of some static array that people could modify. > Another option would be to allow for the specification of a single > function > which is called just prior to importing the rest of the stdlib, > > The problem with all of this is you are essentially asking for a hook > to > let you have code have access to the interpreter state before it is > fully > initialized. Zipimport and the various bits of code that get loaded > during > startup are special since they are coded to avoid touching anything > that > isn't ready to be used. So if we expose something that allows access > prior > to full initialization it would have to be documented as having no > guarantees of interpreter state, etc. so we are not held to some API > that > makes future improvements difficult. > > IOW allowing for easy patching of Python is probably the best option I > can > think of. Would tweaking importlib.bootstrap.install() to accept > specified values for sys.metapath and sys.pathhooks be enough so that > you > can change the call site for those functions? My importer runs under PathFinder so it needs sys.path as well (and doesn't need sys.metapath).

sys.path can be set via PYTHONPATH, etc. so that shouldn't be as much of an issue. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20140724/567ed9ea/attachment.html>

More information about the Python-Dev mailing list