[Python-Dev] test___all__ polluting sys.modules? (original) (raw)

Gregory P. Smith greg at krypto.org
Tue Jan 1 04:26:54 CET 2013

On Sun, Dec 30, 2012 at 5:33 AM, Eli Bendersky <eliben at gmail.com> wrote:

On Sat, Dec 29, 2012 at 5:34 PM, Nick Coghlan <ncoghlan at gmail.com> wrote: On Sun, Dec 30, 2012 at 10:46 AM, Benjamin Peterson <benjamin at python.org> wrote: > 2012/12/29 Eli Bendersky <eliben at gmail.com>: >> Hi, >> >> This came up while investigating some test-order-dependency failures in >> issue 16076. >> >> test_all goes over modules that have _all_ in them and does_ 'from >> import *' on them. This leaves a lot of modules in sys.modules, >> which may interfere with some tests that do fancy things with sys,modules. >> In particular, the ElementTree tests have trouble with it because they >> carefully set up the imports to get the C or the Python version of etree >> (see issues 15083 and 15075). >> >> Would it make sense to save the sys.modules state and restore it in >> test_all so that sys.modules isn't affected by this test?_ > > Sounds reasonable to me.

I've tried this as an inherent property of regrtest before (to resolve some problems with testpydoc), and it didn't work - we have too many modules with non-local side effects (IIRC, mostly related to the copy and pickle registries). Given that it checks the whole standard library, test_all is_ likely to run into the same problem.

Yes, I'm running into all kinds of weird problems when saving/restoring sys.modules around test_all. This is not the first time I get to fight_ this test-run-dependency problem and it's very frustrating. This may be a naive question, but why don't we run the tests in separate interpreters? For example with -j we do (which creates all kinds of strange intermittent problems depending on which tests got bundled into the same process). Is this a matter of performance? Because that would help get rid of these dependencies between tests, which would probably save core devs some work and headache.

For test___all__ having it run itself within a subprocess to guarantee it is in its own interpreter is wise.

Doing that for all tests by default though is a good way to hide problems with particular tests and underhandedly encourage more bad behavior from tests messing up the global environment as there are no consequences. We should aim to keep our tests as clean as possible. That doesn't mean that they are (we've got a lot of old messy code) but it is a good goal.

-gps

After all, since a lot of the interpreter state is global (for example sys.modules), does it not make sense to run each test in a clean environment? Many tests do fancy things with the global environment which makes them difficult to keep clean and separate.

Hence test.support.importfreshmodule - it can ensure you get the module you want, regardless of the preexisting contents of sys.modules. ( http://docs.python.org/dev/library/test#test.support.importfreshmodule) Yes, this is the solution currently used in testxmletree. However, once pickling tests are added things stop working. Pickle uses import to import the module a class belongs to, bypassing all such trickery. So if test_all got elementtree into sys.modules, pickle's import finds_ it even if all the tests in testxmletree manage to ignore it for the Python version because they use importfreshmodule. Eli

Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/greg%40krypto.org -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20121231/6a7477d2/attachment.html>

More information about the Python-Dev mailing list