[Python-Dev] PEP 451 update (original) (raw)

Brett Cannon brett at python.org
Fri Oct 25 21:15:55 CEST 2013

On Fri, Oct 25, 2013 at 2:10 PM, PJ Eby <pje at telecommunity.com> wrote:

On Fri, Oct 25, 2013 at 1:15 PM, Brett Cannon <brett at python.org> wrote: > > On Fri, Oct 25, 2013 at 12:24 PM, PJ Eby <pje at telecommunity.com> wrote: >> At least through all 2.x, reload() just uses module.name to >> restart the module find-and-load process, and does not assume that >> loader is valid in advance. > > > That doesn't make much sense in a post-importlib world where import makes > sure that loader is set (which it has since Python 3.3). Otherwise you > are asking for not just a reload but a re-find as well.

That's a feature, not a bug. A reload() after changing sys.path should take into account the change, not to mention any changes to metapath, path hooks, etc. (And it's how reload() worked before importlib.)

Fair enough, but in my mind that obviously doesn't really click for what I view as a reload in an importlib world where secret import code no longer exists. When I think re-load I think "load again", not "find the module again and execute a load with a possibly new loader".

It's not worth changing in Python 3.3, but if you want to propose to redefine importlib.reload() to use importlib.find_spec() to reset the spec on a module then I'm not going to argue against it (but I'm not going to fight for it either).

And in a PEP 451 world it should be dead-simple to make this work the way you want in your own code even if this doesn't go the way you want::

spec = importlib.find_spec(name) module.spec = spec importlib.reload(module) # Which in itself is essentially init_module_attrs(spec, module); spec.loader.exec_module(module)

Heck, you can do this in Python 3.3 right now::

loader = importlib.find_loader(name) module = sys.modules[name] module.loader = loader importlib.reload(module)

I suppose it's not really documented all that well,

Nope =)

but way way back in the 2.3 timeframe I asked for a tweak to PEP 302 to make sure that reload() (in the re-find sense) would work properly with PEP 302 loaders like zipimport -- some of the language still in the PEP is there specifically to support this use case. (Specifically, the bit that says loaders must use the existing module object in sys.modules if there's one there already, so that reload() will work. It was actually in part to ensure that reload() would work in the case of a re-find.)

Ah, okay. That is not explicit in the PEP beyond coming off a total nuisance in order to support reloading by the loader, not an explicit finder + loader use-case.

It appears that since then, the PEP has been changed in a way that invalidates part of the purpose of the prior change; I guess I missed the discussion of that change last year. :-( ISTM there should've been such a discussion, since IIRC importlib wasn't supposed to change any Python semantics, and this is a non-trivial change to the semantics of reload() even in cases that aren't doing lazy imports or other such munging.

Well, not change them to the best of its abilities. I'm sure there are edge cases that I couldn't match properly since I could only go by the test suite and bug reports filed since Python 3.1.

reload() used to take sys.* and path changes into account, and IMO should continue to do so.

Unfortunately this was never explicitly documented (at least that I noticed) nor was there a test in the stdlib to enforce compliance with it, else this never would have happened.

If this is an intentional change in reload() semantics, other Python implementations need to know about this too!

Not really. As of Python 3.3 the semantic shift included re-implementing the function in pure Python so they pick up the change for free; remember the function is not a builtin anymore to somewhat de-emphasize its use since it's kind of a hack and really only used typically at the interpreter prompt as an easy part of a load-edit-reload dev cycle, not some fundamental operation.

(That being said, I'm not saying I shouldn't or couldn't have tested this in 3.3 and found out about it that way. And the existence of issue18698 suggests that nobody's relying yet on even the fundamental semantics of PEP 302 reload() working properly in 3.3, since that was an even bigger change that nobody spotted till a couple of months ago.)

Yep, never got a bug report on this (although there have been reports of typos in the docs as well as not returning what's in sys.modules so it is being used by people). -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20131025/5f93172e/attachment.html>

More information about the Python-Dev mailing list