Issue 6634: [doc] sys.exit() called from threads other than the main one: undocumented behaviour (original) (raw)
Created on 2009-08-03 19:21 by jgehrcke, last changed 2022-04-11 14:56 by admin. This issue is now closed.
Messages (27)
Author: Dr. Jan-Philip Gehrcke (jgehrcke) *
Date: 2009-08-03 19:21
Hey there,
hopefully I fill out this form in an adequate way!
I ran into some problems while using sys.exit('msg') together with threads, which could have been avoided with slightly more information in the docs here: http://docs.python.org/library/sys.html#sys.exit
Maybe the following two statements should not stay as they are:
(1) "Exit from Python."
This is not true when called from a thread other than the main one. We could add a hint, saying that sys.exit() then actually behaves like thread.exit(), which causes only the calling thread to exit, but not the main program.
- "[...] and any other object is printed to sys.stderr"
This is also not true when called from a thread other than the main one. Calling sys.exit('msg') then doesn't print anything to stderr. That was annoying in my case and required debugging a bug that would have discovered itself via stderr, if the message would have been printed.. :-) After some research, I think this behaviour is described in the documentation for thread.exit(): "[...] this will cause the thread to exit silently."
Okay, now that I am aware of this behaviour, I won't run into these problems again. But the next one?
I think (1) is clearly a documentation thing. Regarding (2): first of all, the documentation should say that the message is suppressed in special cases (child threads). But: what argues against printing to stderr here? I don't get the point and only see a lost feature, affording a quick way to kill a thread while dropping an error message. Was this kicked out intentionally? Maybe someone could help me with a good argument here :-)
Thank you for your work,
Jan-Philip Gehrcke
Author: Gabriel Genellina (ggenellina)
Date: 2009-08-05 00:14
I agree with you; the docs should be improved, and I see no reason for sys.exit("msg") NOT to write to stderr inside a child thread.
Author: Terry J. Reedy (terry.reedy) * 
Date: 2010-08-03 21:35
Please suggest a specific doc change at a specific location. IE, how should they read, not just what is wrong.
Author: Dr. Jan-Philip Gehrcke (jgehrcke) *
Date: 2010-09-25 13:23
Sorry for the delay.
Before suggesting a doc change to correct/complete the description of the current situation, we actually should consider changing this situation. I think this is reasonable and I feel encouraged by Gabriel Genellina:
I see no reason for sys.exit("msg") NOT to write to stderr inside a child thread.
This patch enables printing to stderr from child threads and clones the behavior of sys.exit(arg) called from the main thread:
PATCH BEGIN
--- C:/Python27/Lib/threading.py Sat Apr 10 18:55:48 2010 +++ C:/python_sys_exit_issue/threading.py Sat Sep 25 14:50:24 2010 @@ -531,6 +531,15 @@ except SystemExit: if debug: self._note("%s.__bootstrap(): raised SystemExit", self)
Now get and handle the "exit code", given by the user via
the second expression after
raiseor via the argument ofsys.exit().
- code = self.__exc_info()[1].code
Ignore None and integer exit code. Print any other object
to stderr as it is the behavior of sys.exit(arg) called
from the main thread.
- if code is not None and not isinstance(code, int):
_sys.stderr.write("%s\n" % code)
except: if debug: self._note("%s.__bootstrap(): unhandled exception", self)
PATCH END
A script with different testcases including output is attached.
What do you think?
All the best,
Jan-Philip Gehrcke
Author: Terry J. Reedy (terry.reedy) *
Date: 2010-09-25 16:20
I do not use threads so I cannot comment on the technical issue.
Since the current behavior is not clearly a bug, I do not think a change would or know that it should be applied to 2.7/3.1. So I suggest that you both
Suggest a doc patch on this issue. That should not controversial and might be quickly applied.
Submit a separate feature request issue to change behavior in 3.2 or beyond. You could request that the change be backported to 2.7 and try to make a case for doing so.
Unless you can quickly write a tested patch, such a change in unlikely to make it into 3.2 anyway. And even then, no guarantee.
Author: David Manowitz (David.Manowitz)
Date: 2012-03-23 19:16
I don't see why this should be considered acceptable behavior. Why don't threads have their own ThreadExit exception, rather than overloading the use, and therefore, the meaning, of the SystemExit exception? As indicated by their names, sys.exit and the SystemExit exception should only be used to exit the entire system, not just a thread!
Author: Antoine Pitrou (pitrou) *
Date: 2012-03-24 10:47
I don't see why this should be considered acceptable behavior. Why don't threads have their own ThreadExit exception, rather than overloading the use, and therefore, the meaning, of the SystemExit exception? As indicated by their names, sys.exit and the SystemExit exception should only be used to exit the entire system, not just a thread!
I agree the situation isn't optimal. However, fixing this would also break compatibility with any application that uses sys.exit() in a thread and expects it to exit the thread, not the whole process. So we're kind of stuck with it.
Author: David Manowitz (David.Manowitz)
Date: 2012-03-26 17:40
I have a couple of issues with that argument:
1.) Until fairly recently, the fact that sys.exit() when called from a non-primary thread only causes the thread to die, was not clearly documented (and still isn't in the python2.6 docs). Admittedly, thread.exit() does say that it raises the SystemExit exception, but as most people are encouraged to use the threading module, rather than the thread module directly, this is still fairly obscure.
2.) A ThreadExit exception could be derived from the SystemExit exception, so existing code that works by catching a SystemExit exception would still work.
--David
On Sat, Mar 24, 2012 at 6:47 AM, Antoine Pitrou <report@bugs.python.org>wrote:
Antoine Pitrou <pitrou@free.fr> added the comment:
I don't see why this should be considered acceptable behavior. Why don't threads have their own ThreadExit exception, rather than overloading the use, and therefore, the meaning, of the SystemExit exception? As indicated by their names, sys.exit and the SystemExit exception should only be used to exit the entire system, not just a thread!
I agree the situation isn't optimal. However, fixing this would also break compatibility with any application that uses sys.exit() in a thread and expects it to exit the thread, not the whole process. So we're kind of stuck with it.
nosy: +pitrou
Python tracker <report@bugs.python.org> <http://bugs.python.org/issue6634>
Author: Dr. Jan-Philip Gehrcke (jgehrcke) *
Date: 2015-02-02 18:38
For Python 2.7, we will not change behavior, even if unexpected. Instead, the sys.exit-docs should be adjusted and
- warn about the fact that nothing is written to stderr
if sys.exit(msg) gets called from a non-primary thread, and
- note that SystemExit raised in a non-primary thread lets the
thread exit silently.I have attached a corresponding patch, please review my wording!
Author: Dr. Jan-Philip Gehrcke (jgehrcke) *
Date: 2015-02-02 21:18
For Python 3.5, I have attached a patch that
- adds relevant test cases to test_threading.py which probe
the interpreter's stderr output for compliance with what
the docs state.
- makes sys.exit(msg) write msg to stderr, even if called
from a non-primary thread, so that the tests succeed.If we take this path, the documentation for 3.5 does not need to be adjusted.
The discussion in this thread diversified itself a bit:
Why don't threads have their own ThreadExit exception, rather than overloading the use, and therefore, the meaning, of the SystemExit exception? sys.exit and the SystemExit exception should only be used to exit the entire system, not just a thread!
While I absolutely agree that this would be conceptually cleaner, implementing this would be a larger refactoring task. Deciding whether this should be done or not slows this issue down, and I think this discussion should probably be taken elsewhere.
Author: Martin Panter (martin.panter) *
Date: 2015-02-02 21:29
Regarding the documentation patch: I like to start sentences with a capital letter. Perhaps change it to start “Calling :func:exit only terminates . . .”.
With the code change patch, it might be neater to use the SystemExit.code attribute rather than e.args[0].
Author: Dr. Jan-Philip Gehrcke (jgehrcke) *
Date: 2015-02-02 22:33
Regarding the documentation patch: I like to start sentences with a capital letter. Perhaps change it to start “Calling :func:
exitonly terminates . . .”.
Thanks for feedback. Have now used "Invocation of ...." to not repeat "call*" in the sentence, because I left the "when called from the main thread" part unchanged. Okay?
With the code change patch, it might be neater to use the SystemExit.code attribute rather than e.args[0].
Oh, thanks. Was not aware of the existence of the code attribute. If anyone else was wondering: existence and behavior are defined in Objects/exceptions.c via static PyMemberDef SystemExit_members[] and via static int SystemExit_init().
It is populated upon construction of a SystemExit instance:
Py_CLEAR(self->code); if (size == 1) self->code = PyTuple_GET_ITEM(args, 0); else /* size > 1 */ self->code = args;
Hence, the translation from arguments to exit code considers all arguments. I adjusted the patch to use the code attribute.
Author: Martin Panter (martin.panter) *
Date: 2015-02-03 00:49
New patches look fine. BTW SystemExit.code is also documented at <https://docs.python.org/dev/library/exceptions.html#SystemExit>.
Author: Antoine Pitrou (pitrou) *
Date: 2015-02-03 09:15
I'm not sure what the doc patch achieves. It only states more verbosely what is already said in the current version. Also, warnings are really for important issues (such as security issues); using them too liberally is a disservice to the reader.
Author: Dr. Jan-Philip Gehrcke (jgehrcke) *
Date: 2015-02-03 14:00
Thanks for your feedback Antoine.
I'm not sure what the doc patch achieves.
Let me try to bring things in order. It should achieve two things:
Properly describe the stderr-writing behavior of sys.exit().
Current 2.7 docs:
"..., and any other object is printed to stderr".
This is wrong in its generality and requires clarification. Agreed? How would you improve the docs in this regard? This is the reasoning I had in mind:
We could change the original sentence, but IMO it then contains too many pieces of important information and becomes difficult to digest, e.g.: "If another type of object is passed, None is equivalent to passing zero, and any other object is printed to stderr (only when called in the main thread) and results in an exit code of 1.". Difficult, right?
I thought about removing the stderr part from this sentence and discussing this topic separately, in simpler sentences. But: I think the stderr part needs to stay in the original sentence, because it clarifies how different argument types are dealt with.
Then I took another point of view: we have acknowledged that the behavior is problematic, we just cannot change it anymore for 2.7. The direct consequence from this point of view is to warn about current behavior. 6 years ago, I took the docs literally and relied on getting proper error messages printed to stderr. And I didn't get those. I would have appreciated a warning, I guess.
With this being said, I'd love to look at an alternative proposal of how we could change the docs in this regard.
Clarify that when called from a thread, the thread exits silently
Current 2.7 docs:
"Since exit ultimately "only" raises an exception, it will only exit the process when called from the main thread."
This sentence is problematic. It relates a cause to an effect, and this relation is wrong or at least incomplete. The only valuable information left in the sentence is the effect, without providing an explanation. I would rather want to take the inverse approach, and explain the cause as correct as possible. In other words: the above sentence misses to explain what actually happens when sys.exit() is called in a non-primary thread.
The actual behavior and cause for mentioned effect is, as far as I am aware:
If raised in a non-primary thread, SystemExit is caught automatically and causes the calling thread to exit silently.
This is what I added as a note.
And I think that we agree that this cause may have many effects, whereas only one of them is that sys.exit() called from a non-primary thread can not terminate the process.
It only states more verbosely what is already said in the current version.
I hope to have convinced you that this is not true.
Also, warnings are really for important issues (such as security issues); using them too liberally is a disservice to the reader.
I agree. I really think that both points need to be addressed in the docs, and I have now clarified my reasoning, but am very open to further suggestions. And I really do not want to make a strong point about whether things should end up in red or blue boxes :-).
Author: Dr. Jan-Philip Gehrcke (jgehrcke) *
Date: 2015-02-09 21:14
I'd love to find an agreement here. I think we are quite close to getting this closed, so further input is very welcome.
Author: Martin Panter (martin.panter) *
Date: 2015-02-10 01:20
If it were me, I’d drop the Notes and Warnings (never been a fan), and put Thing #2 first, and then Thing #1. Maybe something like
''' sys.exit([arg])
Exit from Python. . . . and it is possible to intercept the exit attempt at an outer level. When called from a thread other than the main thread, this causes the thread to exit silently instead, and is equivalent to calling :func:thread.exit.
The optional argument arg can be an integer giving the exit status (defaulting to zero). Passing None is equivalent to passing zero. Any other object is printed to stderr and results in an exit status of 1. In
particular, sys.exit("some error message") is a quick way to exit a
program when an error occurs. When called from a thread other than the main thread, nothing is printed and the argument is ignored.
An exit status of zero is considered “successful termination” and any nonzero status is considered “abnormal termination” . . . Unix programs generally use 2 for command line syntax errors and 1 for all other kinds of errors. '''
Author: Dr. Jan-Philip Gehrcke (jgehrcke) *
Date: 2015-02-11 12:38
Martin, I very much like the order you suggested, thanks. I did not feel confident enough for re-structuring the entire entry. So, can we agree on using that for Python 2.7?
Is there a consensus regarding the approach to take for Python 3.5? Except from Martin there was no feedback on the patch.
Author: Irit Katriel (iritkatriel) *
Date: 2021-12-12 11:13
Since 3.8 we have threading.excepthook() so the situation is different now:
https://github.com/python/cpython/pull/13515
I think it still makes sense to change the wording of the doc for sys.exit() a bit, and add a reference to the relevant section of the threading doc.
Author: Vidhya (vidhya) *
Date: 2022-03-01 01:17
[Entry level contributor seeking guidance] If this is still open, I can work on this.
I plan to add the following in sys.exit() and add a reference to thread.exit() at https://docs.python.org/3/library/sys.html#sys.exit:
When called from a thread other than the main thread, this causes the thread to exit silently, and is equivalent to calling :func:thread.exit.
Please correct if anything wrong.
Author: Irit Katriel (iritkatriel) *
Date: 2022-03-01 17:33
Vidhya, I think the sentence you are suggesting to add would overlap with one which is already there ("Since exit() ultimately “only” raises an exception, it will only exit the process when called from the main thread, and the exception is not intercepted.")
It seems to me that what could be improved is the first paragraph, which starts with: "Exit from Python.". Maybe instead it could say something like "Raise a SystemExit exception, which has the effect of ..." and then say what it is (i.e., that it exits python if you are in the main thread, unless the exception is caught... ).
Author: Vidhya (vidhya) *
Date: 2022-03-01 21:59
Thanks for your comments :). The PR for the same is: https://github.com/python/cpython/pull/31639
Author: Irit Katriel (iritkatriel) *
Date: 2022-03-03 14:23
New changeset 10117f1d8cb49ce95493555c06050faf636ccee7 by vidhya in branch 'main': bpo-6634: [doc] clarify that sys.exit() does not always exit the interpreter (GH-31639) https://github.com/python/cpython/commit/10117f1d8cb49ce95493555c06050faf636ccee7
Author: miss-islington (miss-islington)
Date: 2022-03-03 14:49
New changeset 9d9dc59d07d51d73e5af7dd506d0da63aa336995 by Miss Islington (bot) in branch '3.10': bpo-6634: [doc] clarify that sys.exit() does not always exit the interpreter (GH-31639) https://github.com/python/cpython/commit/9d9dc59d07d51d73e5af7dd506d0da63aa336995
Author: Irit Katriel (iritkatriel) *
Date: 2022-03-03 15:16
New changeset 09819863a3fb7092ca5cbdfcb722882ebbac806b by Miss Islington (bot) in branch '3.9': bpo-6634: [doc] clarify that sys.exit() does not always exit the interpreter (GH-31639) (GH-31661) https://github.com/python/cpython/commit/09819863a3fb7092ca5cbdfcb722882ebbac806b
Author: Irit Katriel (iritkatriel) *
Date: 2022-03-03 15:17
Thank you @vidhya.
Author: Vidhya (vidhya) *
Date: 2022-03-03 15:18
Thanks Irit for your help.
On Thu., Mar. 3, 2022, 10:17 a.m. Irit Katriel, <report@bugs.python.org> wrote:
Irit Katriel <iritkatriel@gmail.com> added the comment:
Thank you @vidhya.
resolution: -> fixed stage: patch review -> resolved status: open -> closed
Python tracker <report@bugs.python.org> <https://bugs.python.org/issue6634>
History
Date
User
Action
Args
2022-04-11 14:56:51
admin
set
github: 50883
2022-03-03 15🔞42
vidhya
set
messages: +
2022-03-03 15:17:31
iritkatriel
set
status: open -> closed
resolution: fixed
messages: +
stage: patch review -> resolved
2022-03-03 15:16:34
iritkatriel
set
messages: +
2022-03-03 14:49:44
miss-islington
set
messages: +
2022-03-03 14:24:05
miss-islington
set
pull_requests: + <pull%5Frequest29779>
2022-03-03 14:23:59
miss-islington
set
nosy: + miss-islington
pull_requests: + <pull%5Frequest29778>
2022-03-03 14:23:55
iritkatriel
set
messages: +
2022-03-01 21:59:30
vidhya
set
stage: needs patch -> patch review
messages: +
pull_requests: + <pull%5Frequest29760>
2022-03-01 17:33:49
iritkatriel
set
messages: +
2022-03-01 01:17:14
vidhya
set
nosy: + vidhya
messages: +
2021-12-12 11:13:31
iritkatriel
set
type: behavior
title: sys.exit() called from threads other than the main one: undocumented behaviour -> [doc] sys.exit() called from threads other than the main one: undocumented behaviour
keywords: + easy
nosy: + iritkatriel
versions: + Python 3.9, Python 3.10, Python 3.11, - Python 2.7, Python 3.5
messages: +
2015-02-11 13:09:40
jgehrcke
set
files: + issue6634_py27.patch
2015-02-11 12:38:07
jgehrcke
set
messages: +
2015-02-10 01:20:33
martin.panter
set
messages: +
2015-02-09 21:14:53
jgehrcke
set
messages: +
2015-02-03 14:00:58
jgehrcke
set
messages: +
2015-02-03 09:15:16
pitrou
set
messages: +
2015-02-03 00:49:49
martin.panter
set
messages: +
2015-02-02 22:34:12
jgehrcke
set
files: + issue6634_py27.patch
2015-02-02 22:33:59
jgehrcke
set
files: - issue6634_py27.patch
2015-02-02 22:33:42
jgehrcke
set
files: + issue6634_py35.patch
messages: +
2015-02-02 22:20:27
jgehrcke
set
files: - issue6634_py35.patch
2015-02-02 22:19:35
jgehrcke
set
files: + issue6634_py35.patch
2015-02-02 22:19:23
jgehrcke
set
files: - issue6634_py35.patch
2015-02-02 21:29:55
martin.panter
set
nosy: + martin.panter
messages: +
2015-02-02 21🔞50
jgehrcke
set
files: + issue6634_py35.patch
messages: +
versions: + Python 3.5, - Python 3.1, Python 3.2
2015-02-02 19:56:36
jgehrcke
set
files: - thread_sys_exit_test.py
2015-02-02 18:38:57
jgehrcke
set
files: + issue6634_py27.patch
keywords: + patch
messages: +
2012-03-27 19:49:51
Arfrever
set
nosy: + Arfrever
2012-03-26 17:40:32
David.Manowitz
set
messages: +
2012-03-24 10:47:51
pitrou
set
nosy: + pitrou
messages: +
2012-03-23 19:16:33
David.Manowitz
set
nosy: + David.Manowitz
messages: +
components: + Library (Lib)
2011-04-14 17:49:35
stutzbach
set
nosy: + stutzbach
2010-09-25 16:20:42
terry.reedy
set
messages: +
2010-09-25 13:23:06
jgehrcke
set
files: + thread_sys_exit_test.py
messages: +
2010-08-03 21:35:46
terry.reedy
set
versions: + Python 3.2, - Python 2.6, Python 2.5, Python 2.4, Python 3.0
nosy: + docs@python, terry.reedy, - georg.brandl
messages: +
assignee: georg.brandl -> docs@python
stage: needs patch
2009-08-05 00:14:48
ggenellina
set
nosy: + ggenellina
messages: +
2009-08-03 19:21:49
jgehrcke
create