(original) (raw)

On Sat, Dec 29, 2012 at 5:34 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:

On Sun, Dec 30, 2012 at 10:46 AM, Benjamin Peterson <benjamin@python.org> wrote:

> 2012/12/29 Eli Bendersky <eliben@gmail.com>:

>> Hi,

>>

>> This came up while investigating some test-order-dependency failures in

>> issue 16076.

>>

>> test___all__ goes over modules that have `__all__` in them and does 'from

>> import *' on them. This leaves a lot of modules in sys.modules,

>> which may interfere with some tests that do fancy things with sys,modules.

>> In particular, the ElementTree tests have trouble with it because they

>> carefully set up the imports to get the C or the Python version of etree

>> (see issues 15083 and 15075).

>>

>> Would it make sense to save the sys.modules state and restore it in

>> test___all__ so that sys.modules isn't affected by this test?

>

> Sounds reasonable to me.

I've tried this as an inherent property of regrtest before (to resolve
some problems with test\_pydoc), and it didn't work - we have too many
modules with non-local side effects (IIRC, mostly related to the copy
and pickle registries).

Given that it checks the whole standard library, test\_\_\_all\_\_ is
likely to run into the same problem.

Yes, I'm running into all kinds of weird problems when saving/restoring sys.modules around test\_\_\_all\_\_. This is not the first time I get to fight this test-run-dependency problem and it's very frustrating.

This may be a naive question, but why don't we run the tests in separate interpreters? For example with -j we do (which creates all kinds of strange intermittent problems depending on which tests got bundled into the same process). Is this a matter of performance? Because that would help get rid of these dependencies between tests, which would probably save core devs some work and headache.

After all, since a lot of the interpreter state is global (for example sys.modules), does it not make sense to run each test in a clean environment? Many tests do fancy things with the global environment which makes them difficult to keep clean and separate.

�

Hence test.support.import\_fresh\_module - it can ensure you get the
module you want, regardless of the preexisting contents of
sys.modules. (http://docs.python.org/dev/library/test#test.support.import\_fresh\_module)

Yes, this is the solution currently used in test\_xml\_etree. However, once pickling tests are added things stop working. Pickle uses \_\_import\_\_ to import the module a class belongs to, bypassing all such trickery. So if test\_\_\_all\_\_ got \_elementtree into sys.modules, pickle's \_\_import\_\_ finds it even if all the tests in test\_xml\_etree manage to ignore it for the Python version because they use import\_fresh\_module.

Eli