[Python-Dev] regrtest.py mystery (original) (raw)

Tim Peters tim.one@home.com
Mon, 17 Dec 2001 21:41:23 -0500

[Ka-Ping Yee]

Hi, Ping! If we don't hear from you again, Merry Christmas too .

I remember suggesting exactly that some time ago (i think it was motivated at the time by the extreme pain that broken modules were causing for webserver-mode pydoc in its attempt to update loaded modules if the source files had changed on disk).

Guido rejected it because you can't guarantee that the refcount on M is 1 at the point where you attempt to 'del sys.modules[M]' above.

If somebody wants to hold on to a reference to a damaged module object, that's OK by me. That's an unusual case, though, so shouldn't wag the dog.

(For example, some other module imported by M could have imported M again, and so hold a reference to it while M is running its startup commands. This is why the entry is added to sys.modules before the body of M starts to run.)

Understood, but it's rare.

He deemed the situation where M is loaded-but-missing-from-sys.modules to be even worse than for M to be left loaded-but-broken therein.

Why? The module is broken no mater what. I'm much keener on seeing that the next attempt to import the module doesn't falsely appear to succeed.

If you allow M to stay in sys.modules, then you can at least maintain the guarantee that there is a one-to-one association between loaded module names and loaded module objects. If you remove M from sys.modules but it lingers in memory, referenced elsewhere, you lose even that -- there can be many modules loaded all with the same name and it's a nightmare.

Whether there's one copy or a million, it's broken. Given that, I don't see value in maintaining the one_name<->one_module invariant: of what practical use is it if the module is an insane state regardless? If there is some value to it I'm missing, it would be easy to give it a unique random (re)name .

The argument is compelling and i am forced to agree, but i still think that we should look for a better solution. I'll ponder this and post any ideas i come up with.

Greg's "broken module" flag would do it for me too, if an attempt to import a known-to-be-broken module raised ImportError (in addition to Greg's suggestion that attribute references complain). That scheme has holes too, but it gets increasingly harder to fall into them. The current scheme has so many pits it's proved impossible not to fall into them when using regrtest -r, just doing utterly vanilla non-recursive imports.

compelling-my-foot-ly y'rs - tim