| msg32092 - (view) |
Author: Yang Zhang (yangzhang) |
Date: 2007-05-20 22:24 |
| Hi, I sometimes see the following exceptions when shutting down my app (using Python 2.5.1): Unhandled exception in thread started by Error in sys.excepthook: Original exception was: Exception in thread Thread-3 (most likely raised during interpreter shutdown): Traceback (most recent call last): File "/usr/local/lib/python2.5/threading.py", line 460, in __bootstrap File "/usr/local/lib/python2.5/threading.py", line 440, in run File "/home/yang/local/armed/lib/python2.5/site-packages/afx/threads.py", line 71, in worker File "/usr/local/lib/python2.5/Queue.py", line 176, in get File "/usr/local/lib/python2.5/threading.py", line 248, in notify <type 'exceptions.TypeError'>: exceptions must be classes, instances, or strings (deprecated), not NoneType Unhandled exception in thread started by Error in sys.excepthook: Original exception was: Exception in thread Thread-6 (most likely raised during interpreter shutdown): Traceback (most recent call last): File "/usr/local/lib/python2.5/threading.py", line 460, in __bootstrap File "/usr/local/lib/python2.5/threading.py", line 440, in run File "/home/yang/local/armed/lib/python2.5/site-packages/afx/threads.py", line 71, in worker File "/usr/local/lib/python2.5/Queue.py", line 176, in get File "/usr/local/lib/python2.5/threading.py", line 248, in notify <type 'exceptions.TypeError'>: exceptions must be classes, instances, or strings (deprecated), not NoneType Unhandled exception in thread started by Error in sys.excepthook: Original exception was: Here is the code from my application: def worker(): debug( 'starting worker' ) while True: msg = i.get() # <-- THIS IS LINE 71 if msg is stop_msg: break resultbuf, func, args, kwargs = msg result, exc = None, None try: result = func( *args, **kwargs ) except: t, v, tb = exc_info() exc = t, v, tb.tb_next o.put( ( resultbuf, result, exc ) ) s.send( 'x' ) # assuming socket.send is thread-safe debug( 'stopping worker' ) Here is the origin of the exception (in threading.py): def notify(self, n=1): assert self._is_owned(), "notify() of un-acquire()d lock" # <-- THIS IS LINE 248 __waiters = self.__waiters waiters = __waiters[:n] if not waiters: if __debug__: self._note("%s.notify(): no waiters", self) return self._note("%s.notify(): notifying %d waiter%s", self, n, n!=1 and "s" or "") for waiter in waiters: waiter.release() try: __waiters.remove(waiter) except ValueError: pass I'm not sure why this is happening. The threads are not daemon threads; I terminate them cleanly. When I get a SIGINT (I usu. shut down my app with ctrl-C), I enqueue n stop_msg's to the 'i' Queue so that the n workers can all exit. Note I usually launch 5 workers, so I'm not consistently getting an exception per worker. Also, I've been unable to reproduce this at will. |
|
|
| msg32093 - (view) |
Author: Gabriel Genellina (ggenellina) |
Date: 2007-05-21 12:01 |
| Do you join() the worker threads, waiting until they finish, before exiting the main thread? |
|
|
| msg32094 - (view) |
Author: Yang Zhang (yangzhang) |
Date: 2007-05-21 14:47 |
| No, as they are not daemon threads. |
|
|
| msg32095 - (view) |
Author: Thomas Dybdahl Ahle (thomasda) |
Date: 2007-06-06 13:32 |
| I'm getting the same kind of errors. I'm using a lot of threads, and one of them throws this thread nearly everytime I close my program (by gtk.main_quit) It seems that python sets every variable to None, and then wakeup sleeping threads, which then crash, as they try to work with the None variables Exception in thread Thread-3 (most likely raised during interpreter shutdown): Traceback (most recent call last): File "/usr/lib/python2.4/threading.py", line 442, in __bootstrap File "/home/thomas/Programmering/python/skak/0.7/lib/pychess/System/ThreadPool.py", line 49, in run File "/usr/lib/python2.4/Queue.py", line 89, in put File "/usr/lib/python2.4/threading.py", line 237, in notify exceptions.TypeError: exceptions must be classes, instances, or strings (deprecated), not NoneType Unhandled exception in thread started by Error in sys.excepthook |
|
|
| msg32096 - (view) |
Author: Greg Kochanski (gpk) |
Date: 2007-07-30 22:13 |
| I see the same problem. I'm not sure my code is clean, but I'll be darned if I can find the problem. I've traced my code with print statements, and I see some threads reach their return statement, but the process hangs when you try to join() that thread. It seems that there is something in the clean-up code in threading.py (presumably) that can hang under some obscure circumstances. As a result, these threads don't terminate on time, and various exceptions happen as python is being dismantled and all the variables de-allocated. |
|
|
| msg62073 - (view) |
Author: James Cooper (jamescooper) |
Date: 2008-02-05 17:57 |
| Though these exceptions while shutting down are mostly harmless, they are very noisy and must be squelched in a production application. Here is the patch which we at Solido Design (www.solidodesign.com) are using to hide the exceptions. Note that this doesn't fix the problem, but it does hide the exception for the purposes of releasing an application to the public. |
|
|
| msg62078 - (view) |
Author: Brett Cannon (brett.cannon) *  |
Date: 2008-02-05 20:01 |
| I think the general idea of the problem has been stated, but I figured I would state the official issue. When Python begins to shutdown it takes each module and sets each variable in the global namespace to None. If a thread has not terminated before the interpreter terminates then the thread tries to use a global variable which has been set to None. This is not about to change since this occurs because of coding "errors". You must make sure that either your thread is as safe as a __del__ method (which means no global namespace access) or you can't let the app exit until you are positive all of your threads have terminated, not just asked them to shutdown since this is all asynchronous. |
|
|
| msg62079 - (view) |
Author: Adam Olsen (Rhamphoryncus) |
Date: 2008-02-05 21:24 |
| Py_Main calls WaitForThreadShutdown before calling Py_Finalize, which should wait for all these threads to finish shutting down before it starts wiping their globals. However, if SystemExit is raised (such as via sys.exit()), Py_Exit is called, and it directly calls Py_Finalize, bypassing the WaitForThreadShutdown. Can someone who's experienced this bug check if they're using SystemExit/sys.exit? |
|
|
| msg62080 - (view) |
Author: Adam Olsen (Rhamphoryncus) |
Date: 2008-02-05 21:32 |
| To put it another way: SystemExit turns non-daemon threads into daemon threads. This is clearly wrong. Brent, could you reopen the bug? |
|
|
| msg62081 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2008-02-05 22:57 |
| Hold on, why is that wrong? What if the threads block forever, preventing shutdown? sys.exit() is not exactly some namby-pamby function but a forced shutdown of the interpreter that should guarantee that the interpreter quits. Changing its semantics now would take that away. |
|
|
| msg62083 - (view) |
Author: Adam Olsen (Rhamphoryncus) |
Date: 2008-02-05 23:20 |
| I disagree. sys.exit() attempts to gracefully shutdown the interpreter, invoking try/finally blocks and the like. If you want to truly force shutdown you should use os.abort() or os._exit(). Note that, as python doesn't call a main function, you have to use sys.exit() to have an exit status. |
|
|
| msg62084 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2008-02-05 23:22 |
| OK, I will re-open to see if some other core developer wants to take this on, but personally I am passing. |
|
|
| msg62091 - (view) |
Author: Thomas Dybdahl Ahle (lobais) |
Date: 2008-02-06 11:36 |
| > which means no global namespace access Does that mean that you cannot use len and range in a Thread? |
|
|
| msg62108 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2008-02-06 17:55 |
| > > which means no global namespace access > Does that mean that you cannot use len and range in a Thread? No, it means you have to be careful if you do. Shutting down properly will take care of things. Otherwise you need to save a reference locally (either on an object or as a local variable) and use that reference instead of relying on the one defined in the global namespace. |
|
|
| msg66054 - (view) |
Author: Adam Olsen (Rhamphoryncus) |
Date: 2008-05-01 23:56 |
| This bug was introduced by r53249, which was fixing bug #1566280. Fixed by moving the WaitForThreadShutdown call into Py_Finalize, so all shutdown paths use it. I also tweaked the name to follow local helper function conventions. Martin, since you did the previous fix, can you review this one? |
|
|
| msg66055 - (view) |
Author: Adam Olsen (Rhamphoryncus) |
Date: 2008-05-02 00:04 |
| Oh, and the patch includes a testcase. The current test_threading.py doesn't work with older versions, but a freestanding version of this testcase passes in 2.1 to 2.4, fails in 2.5 and trunk, and passes with the patch. |
|
|
| msg94283 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2009-10-20 15:08 |
| The patch looks good to me. And since Py_Finalize() claims to "destroy all sub-interpreters" and "free all memory allocated by the Python interpreter", I guess your approach makes sense. Can you commit? |
|
|
| msg94290 - (view) |
Author: Adam Olsen (Rhamphoryncus) |
Date: 2009-10-20 16:11 |
| Nope, no access. |
|
|
| msg94291 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2009-10-20 16:48 |
| Ok, I'll do it then! |
|
|
| msg94303 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2009-10-20 22:09 |
| Patch was committed in trunk, py3k and 3.1. Waiting for 2.6 to be unfrozen before I commit it there too. |
|
|
| msg94558 - (view) |
Author: Antoine Pitrou (pitrou) * |
Date: 2009-10-27 13:10 |
| Backported to 2.6 in r75749. |
|
|
| msg99671 - (view) |
Author: Sorin Sbarnea (ssbarnea) * |
Date: 2010-02-21 17:11 |
| Any idea if there is a nightly build for Python 2.6? The latest release was 2.6.4 and was 2 days before submitting the patch. Or the only alternative is to build it myself? Any ideas on when we could see 2.6.5? - I tried to look for a release timeline but I wasn't able to locate one. |
|
|
| msg99686 - (view) |
Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * |
Date: 2010-02-21 20:50 |
| I have seen somewhere (ask google), that python 2.6.5 would be released mid-march. But except for a few platforms, python.org does not provide compiled binaries. |
|
|
| msg99694 - (view) |
Author: R. David Murray (r.david.murray) * |
Date: 2010-02-21 22:36 |
| According to Barry's latest email on the subject, the dates are: 2009-03-01 Python 2.6.5 rc 1 2009-03-15 Python 2.6.5 final And no, there are no nightly builds, you have to build it yourself, I'm afraid. |
|
|
| msg103025 - (view) |
Author: Thijs Triemstra (thijs) |
Date: 2010-04-13 09:18 |
| Looks like this influenced mod_wsgi as well: http://groups.google.com/group/modwsgi/browse_thread/thread/ba82b2643564d2dd |
|
|