[Python-Dev] PEP 451 update (original) (raw)

Eric Snow ericsnowcurrently at gmail.com
Fri Nov 1 18:52:47 CET 2013

On Fri, Nov 1, 2013 at 7:35 AM, Brett Cannon <brett at python.org> wrote:

On Thu, Oct 31, 2013 at 6:10 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

At the time when findspec runs, sys.modules hasn't been touched yet, so the old module state is still available when reloading. Passing the spec in lets the loader decide whether or not it can actually load that module given the information about the previous load operation. Do you mean loader or finder? You say "findspec" which is for finders, but then you say "loader".

However, perhaps it makes more sense to pass the entire existing module, rather than just the spec? I'd like to minimise the need for new-style loader authors to retrieve things directly from the sys module, and you're right that "can I reload into this particular module?" is a slightly different question from "can I reload a module previously loaded using this particular spec?". A spec-based API could still be used to answer the first question by looking up sys.modules, but so could the name based API. Passing in the module reference, on the other hand, should let loaders answer both questions without needing to touch the sys module. I also now think this pass-the-module approach when finding the spec approach could also clean up some of the messiness that is main initialisation, where we repeatedly load things into the same module object. See you mention "finding" but before now everything has been "loader". Let's say we be completely explicit and call the new parameter something like "loadtarget". If we do that then I would make the same change to runpy.runpath and runpy.runmodule: let you pass in an existing module object under that name to say "hey, run in this namespace, rather than a fresh one". (Those APIs currently only support pre-populating a fresh namespace, rather than allowing you to directly specify an existing one) Most loaders won't care, but those that do will have all the info they need to throw an exception saying "I could provide a spec for this, but it's not compatible with that load target". Exactly what APIs -- new or changed -- are you proposing? Method signatures only please to avoid ambiguity. =)

I'm about to update the PEP but I'll clarify here. The signature of finder.find_spec() will grow an extra "existing" parameter (default to None) that is an existing module. During reload the module from sys.modules will be passed in. Having this be part of find_spec() is consistent with the job of the finder to identify a loader that should be used to load the module (reload or not).

As to the inconsistency in what Nick said, it still fits in with the way I see it. I'm sure he'll correct me if he had something else in mind. :-) finder.find_spec() will be making a decision on what loader (if any) to return. As part of that, when it finds a loader it can further decide on whether or not the loader supports loading into an existing module (if one was provided). As part of that it may or may not consult with the loader. Either way it stands as proxy for the loader in the latter decision.

In lieu of an explicit loader.supports_reload() method, a loader will rely on its associated finder to be a proxy for reload-related decisions, likely communicated via spec.loader_state. Perhaps Nick meant something else, but my understanding is that he was referring to this

-eric

More information about the Python-Dev mailing list