Issue 9895: Speed up test_subprocess (original) (raw)

Created on 2010-09-18 18:24 by pitrou, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
noleaking.patch pitrou,2010-09-18 18:24
Messages (6)
msg116821 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-09-18 18:24
test_subprocess is currently very slow and the main culprit is test_no_leaking (here, it takes 80s out of the 100s needed for the whole run). This tests spawns as many subprocesses as would be needed to reach the max file descriptor limit. Spawning a subprocess is a heavy operation and consequently the test is very long. This patch adopts another approach: it opens many file descriptors (using os.open()) until the max fd limit is reached, closes some of them (10 is enough) and then launches a bunch of subprocesses in a loop to check that they succeed. Consequently, the test is very fast (even on a Windows VM in debug mode).
msg116826 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2010-09-18 21:12
Overall I like the approach. A few questions / clarifications: is errno.EMFILE portable? will that errno test work on both posix and windows? should the is_resource_enabled('subprocess') stuff be preserved (why was it there to begin with? because the old test took so long?)
msg116827 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-09-18 21:27
> A few questions / clarifications: > > is errno.EMFILE portable? will that errno test work on both posix and > windows? It does (tested under Linux and a Win64 VM). > should the is_resource_enabled('subprocess') stuff be preserved (why was it > there to begin with? because the old test took so long?) I guess the latter: the test was so long, especially on slow machines, that it was only enabled selectively.
msg116828 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2010-09-18 21:29
looks good to me. commit it. :)
msg116829 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2010-09-18 21:37
+1 fo faster regrtest
msg116832 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-09-18 22:44
Done, thank you. (r84883, r84884, r84885)
History
Date User Action Args
2022-04-11 14:57:06 admin set github: 54104
2010-09-18 22:44:57 pitrou set status: open -> closedresolution: fixedmessages: + stage: patch review -> resolved
2010-09-18 21:37:05 flox set nosy: + floxmessages: +
2010-09-18 21:35:21 flox set files: - unnamed
2010-09-18 21:35:11 flox set files: - unnamed
2010-09-18 21:29:57 gregory.p.smith set files: + unnamedmessages: +
2010-09-18 21:27:36 pitrou set messages: +
2010-09-18 21:12:43 gregory.p.smith set files: + unnamedmessages: +
2010-09-18 18:24:54 pitrou create