| msg12739 - (view) |
Author: Jamin W. Collins (jamincollins) |
Date: 2002-10-10 20:25 |
| I originally noticed this problem with Debian's OfflineIMAP package and filed a report there: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=162369 The short of it is that when running offlineimap for an extended period of time it will eventually leave a large number of defunct threads behind that eventually use up all available system processes. Killing the original offlineimap thread clears the problem for a while. The Debian maintainer of OfflineIMAP referred this over to the python maintainer, who in turn has asked (as you can see from the link above) that I file a report here. If there is any more information I can provide (beyond that in the Debian case already) please let me know. |
|
|
| msg12740 - (view) |
Author: Martin v. Löwis (loewis) *  |
Date: 2002-10-13 11:05 |
| Logged In: YES user_id=21627 The analysis that this is a Python bug is most likely wrong. You write > a number of defunct/zombie processes are spawned. then John Goerzen writes > It is likely, in any case, that this is a bug in Python itself. The > OfflineIMAP program does not fork However, John later points out that there are many threads used in this package. Please understand that threads show up as processes in Linux ps(1). IOW, what you are seeing are the many threads, not any additional processes. It appears that the system is waiting for somebody to call pthread_join. This should not be necessary, since Python invokes pthread_detach for all of its threads. So if the system does not reclaim terminated threads, it seems there is a bug in your C library. |
|
|
| msg12741 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2002-10-15 00:35 |
| Logged In: YES user_id=6380 I agree with Martin von Loewis - this does not appear to be a Python bug. When a Python thread exits, it does not show up in the ps listing as ; that to me suggests that there *is* some forking going on, perhaps under the guise of system() or popen(); or perhaps there's a bug in the C library's thread support. Lowering the priority as a result. I'll try to followup in the Debian thread as well, if there's a web interface. |
|
|
| msg12742 - (view) |
Author: John Goerzen (jgoerzen) |
Date: 2003-04-17 19:27 |
| Logged In: YES user_id=491567 OfflineIMAP has only two cases where one of these things might happen: 1) in the Curses interface module, where it does a fork at the very beginning to check to see if Curses will work (just a quick fork/exit/cleanup) 2) with the preauth code, where it does a popen2 to a local imap daemon. Few people use preauth, and the Curses problem can easily be dismissed by switching to another ui (TTY.TTYUI perhaps?) I have observed that even the TTY.TTYUI leaves processes hanging around in Python 2.2 but not in Python 2.3. I suspect the original submitter is running the Tk interface; my own experimentation has shown it leaves lots of processes hanging around. However -- I have never seen zombie processes hanging around. The processes I see have blocked themselves with rt_sigsuspend. |
|
|
| msg12743 - (view) |
Author: Jamin W. Collins (jamincollins) |
Date: 2003-04-17 22:41 |
| Logged In: YES user_id=88136 I used both the TK and the TTY interfaces. In both cases (using both Python 2.2 and 2.3) zombie threads would accumlate over time when it was left running in daemon mode. |
|
|
| msg12744 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2003-04-18 00:27 |
| Logged In: YES user_id=6380 I don't understand aything that is being said in the last two comments. If anything, the bug appears to be in the application (with which I am not familiar). I'm going to close soon this unless someone has a bug they can demonstrate with a small sample program that does not involve OfflineIMAP. |
|
|
| msg12745 - (view) |
Author: Jamin W. Collins (jamincollins) |
Date: 2003-04-18 00:55 |
| Logged In: YES user_id=88136 Go ahead and close it. Problem still exists. I don't know enough about either to say which is at fault. Nor do I know enough about python to attempt to replicate it. I only know the the OfflineIMAP folks say it's not them and it would seem now it's not Python either. *shrug* I tried. |
|
|
| msg12746 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2003-06-16 14:48 |
| Logged In: YES user_id=6380 In an email conversation with OfflineIMAP author John Goerzen, we've decided that this must be a bug in the platform pthread library, not in OfflineIMAP nor in Python. |
|
|