[Python-Dev] Import lock knowledge required! (original) (raw)

M.-A. Lemburg mal@lemburg.com
Sun, 16 Feb 2003 10:49:30 +0100

Jack Jansen wrote:

On zaterdag, feb 15, 2003, at 16:49 Europe/Amsterdam, M.-A. Lemburg wrote:

I think we're running into a similar bootstrap problem as we did when we had exceptions coded in Python. The solution back then was to recode it in C. Perhaps the warnings module should get the same treatment ?! There are other modules with the same potential problem. For instance, the code that prints MacOS error messages imports a Python module to get at the texts of the messages. I haven't had any reports of deadlocks, but I haven't looked for them either, so it may be possible this code gets called with the import lock held. And at first glance code using cobject's PyCobjectimport() may be at risk too (only at first glance: I didn't check to see whether such code is ever executed with with import lock held).

I wonder whether a general lock such as the one used in import is such a good idea. Perhaps it should only lock the importing of a specific module, keeping the locks in a dictionary indexed by module name instead of a static C variable ?!

Then again it's hard to know the real name of the module being searched before finding it...

I see a more general problem here: the lock prevent starting up threaded applications which use client-server logic between the threads. If the application's main thread starts a client thread as a result of an import which then tries to import other Python modules, you have a deadlock. (At least that's how I understand the current implementation.)

Don't know about others, but I frequently use the idiom of placing the server's main code in a separate module and then have small startup script importing this module. This kind of setup is also advertised for CGI programs, so it may not be uncommon out there.

-- Marc-Andre Lemburg eGenix.com

Professional Python Software directly from the Source (#1, Feb 16 2003)

Python/Zope Products & Consulting ... http://www.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

Python UK 2003, Oxford: 44 days left EuroPython 2003, Charleroi, Belgium: 128 days left