[Python-Dev] SVN rev. 85392 broke module handling in py3k (original) (raw)

Benjamin Peterson benjamin at python.org
Fri Oct 22 16:17:44 CEST 2010

2010/10/22 Stefan Behnel <stefan_ml at behnel.de>:

Benjamin Peterson, 22.10.2010 16:03:

2010/10/22 Stefan Behnel:

since SVN rev. 85392, Cython's installation fails on the py3k branch with a weird globals error. I think it is related to some sys.modules magic that we do in order to support running Cython in Python 3 using lib2to3. Basically, what we do is, we import some parts of Cython at the beginning that are Py3 clean, specifically some distutils buildext replacement for building Cython modules. Then we start up distutils, which first runs lib2to3 on Cython's sources to convert them into Py3 code. When it then gets to building the binary modules, we remove all Cython modules and packages from sys.modules and reimport their 2to3-ed sources so that we can run the complete compiler during the installation (to bootstrap parts of Cython into binary modules). Since the above revision, this process bails out with an error when accessing "os.path" because "os" is None. The "os" module is imported globally in our early-imported buildext module, more or less like this:  import os  from distutils.command import buildext as buildext  class buildext(buildext.buildext):  def buildextensions(self):  print(os) # prints None! I suspect that the fact that we remove the modules from sys.modules somehow triggers the cleanup of these modules while there are still objects from these modules alive that refer to their globals. So, what I think is happening is that the module cleanup sets the module's globals to None before the objects from that module that refer to these globals have actually gone out of scope. Could someone (benjamin?) please look into this? Is this broken before 2.7, ie 2.6 and 2.6? I can't tell. Py2 doesn't need 2to3, so we don't unload the modules there.

What about 3.1.0 then?

-- Regards, Benjamin

More information about the Python-Dev mailing list