[Python-Dev] Import lock knowledge required! (original) (raw)

eric jones eric@enthought.com
Mon, 17 Feb 2003 12:29:31 -0600

[MAL] > I wonder whether a general lock such as the one used in import > is such a good idea. Perhaps it should only lock the importing > of a specific module, keeping the locks in a dictionary indexed by > module name instead of a static C variable ?!

I am interested in this also.

[Guido] I've been thinking along the same lines. We could make the import locking much finer-grained, and limit the blocking only to threads that are importing a module that is in the middle of being loaded by some other thread. But this is hard work, and I suggest that we put this off until Python 2.4 so we can do it right.

I ran into this exact thing when trying to get wxPython windows (data plots) to co-exist peacefully on the screen in parallel with an interactive shell. The module is called gui_thread, and we use it in SciPy.

http://www.scipy.org/site_content/tutorials/gui_thread

The command line took the main thread, and a wxPython app was started in the background thread. The first import of wxPython had to occur in the background thread for wxPython's to be happy. Initially, I put a lock in the import of gui_thread which waited for the wxPython import to complete before allowing the gui_thread import to complete. If 'import gui_thread' is the first statement executed, this guarantees that wxPython is always imported first in the background thread. Unfortunately, it also causes deadlock because of the import lock.

A discussion that occurred about this problem on the thread-sig with is summarized here:

http://www.scipy.org/site_content/tutorials/import_thread_lock_discussio n

It includes an alternative version of several import.c routines (compared to 1.5.2 I believe) that solved my problem as proposed by MAL above and passed all the regression tests at the time. I just plugged the code into the 2.3CVS, and it mainly works. But, there are some import related errors in the regression tests for:

test_threaded_import test_loggingllbacks

I could spend some time on these if there is interest in getting this in the current release. Also, this code would need review by someone that is an expert on the subtleties of the import code to make sure it is sound. Let me know, and I'll submit a patch with of the current code.

eric