Issue 5179: subprocess leaves open fds on construction error (original) (raw)

Created on 2009-02-07 20:04 by georg.brandl, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
sp-patch.py georg.brandl,2009-02-07 20:17
fix_subprocess_leak_on_failure_on_windows.patch ocean-city,2009-03-03 21:04
Messages (12)
msg81347 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2009-02-07 20:04
The test case below will (on Linux here) eventually quit with "OSError: [Errno 24] Too many open files". I assume that some additional cleaning up is in order. ------------------------------------------------------------------- from subprocess import Popen, PIPE while 1: try: Popen(['nonexisting'], stdout=PIPE, stderr=PIPE) except OSError, err: if err.errno != 2: # ignore "no such file" raise
msg81348 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2009-02-07 20:17
Proposed patch attached.
msg81387 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-02-08 16:06
Perhaps you could add a test?
msg82001 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2009-02-14 08:49
I would, but how this fails is likely to be highly platform-specific. Can you try it on Windows and tell me what the resulting exception is?
msg82028 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-02-14 12:18
Well, I'm not under Windows. I'll try to launch a VM if nobody beats me to it...
msg82082 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2009-02-14 16:00
On second thought, if the patch is applied, there *shouldn't* be an exception. And simply running the loop for a fixed number of repetitions isn't a good test either, since different platforms may have a different maximum for open file descriptors. Hmm, is there a way to get the current number of open file descriptors?
msg82083 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-02-14 16:12
> Hmm, is there a way to get the current number of open file descriptors? Under Unix, you can use resource.getrlimit(): (1024, 1024) But 1024 is a very common value, so you could simply loop 1024 times if it's not too slow. Besides, you definitely don't want to loop 2**31 times if 2**31 happens to be the current limit.
msg82084 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-02-14 16:13
Wow, the Roundup email gateway borks code snippets: >>> import resource >>> resource.getrlimit(resource.RLIMIT_NOFILE) (1024, 1024)
msg82092 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2009-02-14 17:01
OK, Windows has 2048. Added test and fixed in r69620.
msg83097 - (view) Author: Hirokazu Yamamoto (ocean-city) * (Python committer) Date: 2009-03-03 21:04
This issue is not fixed on windows yet. (test_leaking_fds_on_error fails) I think attached patch will fix this. ====================================================================== ERROR: test_writes_before_communicate (__main__.ProcessTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_subprocess.py", line 398, in test_writes_before_communicate File "e:\python-dev\py3k\lib\subprocess.py", line 632, in __init__ File "e:\python-dev\py3k\lib\subprocess.py", line 746, in _get_handles IOError: [Errno 24] Too many open files
msg83099 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2009-03-03 21:05
Since I can't test on Windows, I'll leave that in your hands :)
msg83108 - (view) Author: Hirokazu Yamamoto (ocean-city) * (Python committer) Date: 2009-03-03 22:56
Thanks, fixed in r70137(trunk), r70142(py3k), r70146(release30-maint), r70147(release26-maint)
History
Date User Action Args
2022-04-11 14:56:45 admin set github: 49429
2009-03-03 22:56:40 ocean-city set status: open -> closedresolution: fixedmessages: + priority: release blocker -> high
2009-03-03 21:05:40 georg.brandl set messages: +
2009-03-03 21:04:27 ocean-city set status: closed -> openfiles: + fix_subprocess_leak_on_failure_on_windows.patchversions: + Python 3.0, Python 3.1keywords: + patchnosy: + ocean-citymessages: + resolution: fixed -> (no value)priority: high -> release blocker
2009-02-14 17:01:53 georg.brandl set status: open -> closedresolution: fixedmessages: +
2009-02-14 16:13:59 pitrou set messages: +
2009-02-14 16:12:58 pitrou set messages: +
2009-02-14 16:00:45 georg.brandl set messages: +
2009-02-14 12🔞40 pitrou set messages: +
2009-02-14 08:49:30 georg.brandl set messages: +
2009-02-08 16:06:50 pitrou set nosy: + pitroumessages: +
2009-02-07 20:17:55 georg.brandl set files: + sp-patch.pymessages: +
2009-02-07 20:04:37 georg.brandl create