bpo-40055: distutils tests now disable docutils import by vstinner · Pull Request #19139 · python/cpython (original) (raw)

I would also prefer to either clear the warnings filters or clear the warnings filters and eject pkg_resources from sys.modules, but I see all of these solutions as somewhat brittle hacks around a somewhat sloppy definition of "no side-effects".

I don't see the change to the warnings filters as terribly different from modifying sys.modules, since this is evidently part of the package initialization of pkg_resources. Entering pkg_resources into sys.modules in one test will prevent the package from being executed again if you run import pkg_resources, which means that later tests using pkg_resources will suffer from the side-effect of the cache having been populated and the filters having been cleared.

My guess is that the principle behind tests not having side effects is that we want tests or test modules to be independent of one another. If some random test is manipulating filters that other tests may use, that would be a violation of that principle, but this is just pkg_resources settings its own defaults for the warnings filters related to its own warnings - if anything, we should be making sure that once pkg_resources is in the sys.modules cache, anything that changes those warnings filters doesn't restore them to the pre-import stage.

I feel like the right long-term solution would be to move to a testing scenario where every test module is run as separate processes with their own namespaces rather than trying to make sure everything plays nicely with process-wide globals, but that feels like a much bigger undertaking than is warranted for this minor issue, so I say let's go with something where we clear the warnings filters and maybe eject pkg_resources from sys.modules (which seems like the right thing to do, but also I don't think these warnings filters matter at all for our purposes so 🤷‍♂️).