[Python-Dev] multiple interpreters and extension modules (original) (raw)

Bob Ippolito bob at redivi.com
Sat Dec 23 03:54:39 CET 2006

On 12/23/06, Jeremy Kloth <jeremy.kloth at 4suite.org> wrote:

On Friday 22 December 2006 5:02 pm, Josiah Carlson wrote: > Jeremy Kloth <jeremy.kloth at 4suite.org> wrote: > > [[ This may be somewhat c.l.p.-ish but I feel that this crossed into > > CPython development enough to merit posting here ]] > > > > I have received a bug report for 4Suite that involves a > > PyObjectIsInstance check failing for what appears to be the correct > > class, that is, the class names match. With some investigating, I have > > found that the true problem is with multiple interpreters. The reason > > for this is that each sub-interpreter has a "new" copy of any pure Python > > module. The following issues are also true for modules that have been > > reloaded, but I think that is common knowledge. I mention it only for > > completeness. > > If I remember correctly, Python allows you to use multiple interpreters > in the same process, but it makes no guarantees as to their correctness > when running. > > See this post for further discussion on the issue: > http://mail.python.org/pipermail/python-list/2004-January/244343.html > > You can also search for 'multiple python interpreters single process' in > google without quotes to hear people lament over the (generally broken) > multiple Python interpreter support.

The problem here is that it is modpython using the multiple interpreters. We have no control over that. What I'm proposing is fixing the extension module support for multiple interpreters with the bonus of adding extension module finalization which I've seen brought up here before. Fixing this does require support by the extension module author, but if that author doesn't feel the need to work in modpython (if, of course, they load module level constants), that is their loss. Is 4Suite that different in its use of hybrid Python and C extensions? There is lots of back and forth between the two layers and performance is critical. I really don't feel like recoding thousands of lines of Python code into C just to get 4Suite to work in modpython without error.

It's a whole lot more practical to just stop using mod_python and go for one of the other ways of exposing Python code to the internet. I bet you can get the same or better performance out of another solution anyway, and you'd save deployment headaches.

-bob

More information about the Python-Dev mailing list