(original) (raw)

On Aug 19, 2016 6:46 PM, Chris Angelico wrote:
\>
\> On Sat, Aug 20, 2016 at 2:25 AM, Steve Dower wrote:
\> > On 19Aug2016 0910, Chris Angelico wrote:
\> >>
\> >> On Sat, Aug 20, 2016 at 1:26 AM, Steve Dower
\> >> wrote:
\> >>>
\> >>> Check any .pth files you can find. I suspect mpl\_toolkits has some magic
\> >>> in
\> >>> it to make the namespace package work on 2.7.
\> >>
\> >>
\> >> $ cat
\> >> /usr/local/lib/python3.6/site-packages/matplotlib-1.5.1-py3.6-nspkg.pth
\> >> import sys, types, os;p =
\> >> os.path.join(sys.\_getframe(1).f\_locals\['sitedir'\],
\> >> \*('mpl\_toolkits',));ie =
\> >> os.path.exists(os.path.join(p,'\_\_init\_\_.py'));m = not ie and
\> >> sys.modules.setdefault('mpl\_toolkits',
\> >> types.ModuleType('mpl\_toolkits'));mp = (m or \[\]) and
\> >> m.\_\_dict\_\_.setdefault('\_\_path\_\_',\[\]);(p not in mp) and mp.append(p)
\> >>
\> >> Is it possible that that's the cause? And if so, should "python -I"
\> >> remove /usr/local/lib/python3.6/site-packages from sys.path?
\> >
\> >
\> > That explains the extra modules - types.py imports collections and
\> > collections.abc, which would seem to account for the rest.
\> >
\> > I'm not sure whether that's causing the problem or not, but temporarily
\> > renaming that pth file so it isn't picked up may help you track down the
\> > test issue.
\>
\> Sure enough, that did fix the problem - that is to say, renaming it
\> did allow test\_site to pass. Thanks!
\>
\> > "python -S" is the only way to avoid .pth files, but that's a pretty heavy
\> > hammer.
\>
\> Hmmmmmmm. So the question is, what is this test testing? For
\> reference, here's the code:
\>
\> class StartupImportTests(unittest.TestCase):
\>
\> def test\_startup\_imports(self):
\> # This tests checks which modules are loaded by Python when it
\> # initially starts upon startup.
\> popen = subprocess.Popen(\[sys.executable, '-I', '-v', '-c',
\> 'import sys; print(set(sys.modules))'\],
\> stdout=subprocess.PIPE,
\> stderr=subprocess.PIPE)
\> stdout, stderr = popen.communicate()
\> stdout = stdout.decode('utf-8')
\> stderr = stderr.decode('utf-8')
\> modules = eval(stdout)
\>
\> self.assertIn('site', modules)
\>
\> # http://bugs.python.org/issue19205
\> re\_mods = {'re', '\_sre', 'sre\_compile', 'sre\_constants', 'sre\_parse'}
\> # \_osx\_support uses the re module in many placs
\> if sys.platform != 'darwin':
\> self.assertFalse(modules.intersection(re\_mods), stderr)
\> # http://bugs.python.org/issue9548
\> self.assertNotIn('locale', modules, stderr)
\> if sys.platform != 'darwin':
\> # http://bugs.python.org/issue19209
\> self.assertNotIn('copyreg', modules, stderr)
\> # http://bugs.python.org/issue19218>
\> collection\_mods = {'\_collections', 'collections', 'functools',
\> 'heapq', 'itertools', 'keyword', 'operator',
\> 'reprlib', 'types', 'weakref'
\> }.difference(sys.builtin\_module\_names)
\> self.assertFalse(modules.intersection(collection\_mods), stderr)
\>
\>
\> It asserts that 'site' is in the list of loaded modules, so simply
\> adding '-S' to the subpython parameters doesn't work. But this test,
\> as written, depends on the behaviour of arbitrary third-party modules
\> installed with pip, which seems wrong for a standard library test.
\> AIUI this is trying to catch changes that accidentally cause large
\> modules to be loaded on startup, correct? Which is all well and good,
\> but Python shouldn't have test failures due to matplotlib being
\> installed.
\>
\> I see two (or three) ways to deal with this.
\>
\> 1) Declare it a bug in matplotlib, and demand that .pth files not
\> import anything that wouldn't otherwise be imported.
\>
\> 2) Change the test to add '-S' and remove the assertion about 'site'
\> being present; this means that site.py itself is no longer covered by
\> the test.
\>
\> 3) Accept the status quo: this test can fail if .pth files are present.
\>

4) split the test in two: one running with -S to check no collections, (and for \*not\* site.py). The other without -S, and check for site.py.