bpo-36670: regrtest WindowsLoadTracker computes LOAD_FACTOR_1 by vstinner · Pull Request #16556 · python/cpython (original) (raw)

I modified the code to initialize the load to the first processor_queue_length. This is not perfect, but it helps to get an idea of the real load average in the first 60 seconds. After one minute, the value using the old and the new code should converge anyway.

I made more tests and using a single value is really too raw. I modified my PR to use the arithmetic mean on the first 5 values instead. It helps to converge faster to the "real" load average.

Example of debug traces:

The load average is initialized to 4.4 and it moves to 5.36 after 1 minute: 4.4 is a better estimation than 0.0 for the first value ;-)

exp: 4.4 -- mean: 4.40 -- last 10 values: 10, 9, 3, 0, 0
0:00:06 load avg: 4.40 running: test__osx_support (3.1 sec), test_heapq (3.1 sec)
exp: 4.3 -- mean: 3.67 -- last 10 values: 10, 9, 3, 0, 0, 0
0:00:07 load avg: 4.33 [  5/419] test__osx_support passed -- running: test_heapq (3.6 sec)
0:00:07 load avg: 4.33 [  6/419] test_heapq passed (1.6 sec)
0:00:07 load avg: 4.33 [  7/419] test_positional_only_arg passed
0:00:07 load avg: 4.33 [  8/419] test_crashers passed
exp: 4.4 -- mean: 4.00 -- last 10 values: 10, 9, 3, 0, 0, 0, 6
0:00:08 load avg: 4.35 running: test_grammar (1.0 sec), test_pathlib (1.0 sec)
exp: 4.3 -- mean: 3.62 -- last 10 values: 10, 9, 3, 0, 0, 0, 6, 1
0:00:09 load avg: 4.30 running: test_grammar (2.0 sec), test_pathlib (2.0 sec)
exp: 4.2 -- mean: 3.22 -- last 10 values: 10, 9, 3, 0, 0, 0, 6, 1, 0
0:00:10 load avg: 4.23 running: test_grammar (3.0 sec), test_pathlib (3.0 sec)
exp: 4.2 -- mean: 2.90 -- last 10 values: 10, 9, 3, 0, 0, 0, 6, 1, 0, 0
0:00:12 load avg: 4.16 running: test_grammar (4.0 sec), test_pathlib (4.0 sec)
exp: 4.1 -- mean: 2.64 -- last 10 values: 9, 3, 0, 0, 0, 6, 1, 0, 0, 0
0:00:12 load avg: 4.09 [  9/419] test_grammar passed -- running: test_pathlib (5.0 sec)
0:00:12 load avg: 4.09 [ 10/419] test_pathlib passed (4.5 sec)
exp: 4.2 -- mean: 3.33 -- last 10 values: 3, 0, 0, 0, 6, 1, 0, 0, 0, 11
0:00:13 load avg: 4.20 [ 11/419] test_dictcomps passed
0:00:13 load avg: 4.20 [ 12/419] test_filecmp passed
exp: 4.3 -- mean: 3.85 -- last 10 values: 0, 0, 0, 6, 1, 0, 0, 0, 11, 10
0:00:14 load avg: 4.30 running: test_argparse (1.0 sec), test_tcl (1.0 sec)
0:00:15 load avg: 4.30 [ 13/419] test_tcl passed -- running: test_argparse (1.2 sec)
exp: 4.3 -- mean: 3.93 -- last 10 values: 0, 0, 6, 1, 0, 0, 0, 11, 10, 5
0:00:15 load avg: 4.31 [ 14/419] test_codecmaps_kr passed -- running: test_argparse (1.8 sec)
0:00:16 load avg: 4.31 [ 15/419] test_imghdr passed -- running: test_argparse (2.5 sec)
exp: 4.4 -- mean: 4.20 -- last 10 values: 0, 6, 1, 0, 0, 0, 11, 10, 5, 8
0:00:16 load avg: 4.37 [ 16/419] test_netrc passed -- running: test_argparse (3.1 sec)
0:00:17 load avg: 4.37 [ 17/419] test_check_c_globals passed -- running: test_argparse (3.8 sec)
exp: 4.3 -- mean: 4.12 -- last 10 values: 6, 1, 0, 0, 0, 11, 10, 5, 8, 3
0:00:18 load avg: 4.35 [ 18/419] test_turtle passed -- running: test_argparse (4.5 sec)
exp: 4.4 -- mean: 4.24 -- last 10 values: 1, 0, 0, 0, 11, 10, 5, 8, 3, 6
0:00:18 load avg: 4.38 [ 19/419] test_argparse passed (4.6 sec)
0:00:18 load avg: 4.38 [ 20/419] test_format passed
0:00:19 load avg: 4.38 [ 21/419] test_kqueue skipped
test_kqueue skipped -- test works only on BSD
0:00:19 load avg: 4.38 [ 22/419] test_doctest2 passed
exp: 4.5 -- mean: 4.78 -- last 10 values: 0, 0, 0, 11, 10, 5, 8, 3, 6, 14
0:00:20 load avg: 4.54 [ 23/419] test_bigaddrspace passed
0:00:20 load avg: 4.54 [ 24/419] test_fork1 skipped
test_fork1 skipped -- object <module 'os' from 'C:\\vstinner\\python\\master\\\\lib\\os.py'> has no attribute 'fork'
0:00:20 load avg: 4.54 [ 25/419] test_defaultdict passed
exp: 4.6 -- mean: 4.84 -- last 10 values: 0, 0, 11, 10, 5, 8, 3, 6, 14, 6
0:00:21 load avg: 4.56 [ 26/419] test_genericclass passed -- running: test_idle (1.3 sec)
exp: 4.6 -- mean: 5.00 -- last 10 values: 0, 11, 10, 5, 8, 3, 6, 14, 6, 8
0:00:22 load avg: 4.62 running: test_idle (2.3 sec), test_set (1.0 sec)
0:00:22 load avg: 4.62 [ 27/419] test_idle passed (2.2 sec) -- running: test_set (1.4 sec)
exp: 4.6 -- mean: 5.05 -- last 10 values: 11, 10, 5, 8, 3, 6, 14, 6, 8, 6
0:00:23 load avg: 4.64 running: test_runpy (1.0 sec), test_set (2.4 sec)
exp: 4.7 -- mean: 5.23 -- last 10 values: 10, 5, 8, 3, 6, 14, 6, 8, 6, 9
0:00:24 load avg: 4.71 running: test_runpy (2.1 sec), test_set (3.5 sec)
exp: 4.7 -- mean: 5.22 -- last 10 values: 5, 8, 3, 6, 14, 6, 8, 6, 9, 5
0:00:26 load avg: 4.72 running: test_runpy (3.2 sec), test_set (4.6 sec)
exp: 4.7 -- mean: 5.25 -- last 10 values: 8, 3, 6, 14, 6, 8, 6, 9, 5, 6
0:00:26 load avg: 4.74 [ 28/419] test_set passed (5.1 sec) -- running: test_runpy (4.2 sec)
exp: 4.8 -- mean: 5.40 -- last 10 values: 3, 6, 14, 6, 8, 6, 9, 5, 6, 9
0:00:27 load avg: 4.81 [ 29/419] test_runpy passed (4.7 sec)
exp: 4.8 -- mean: 5.38 -- last 10 values: 6, 14, 6, 8, 6, 9, 5, 6, 9, 5
0:00:28 load avg: 4.81 running: test_csv (1.0 sec), test_tokenize (2.0 sec)
0:00:29 load avg: 4.81 [ 30/419] test_csv passed -- running: test_tokenize (2.2 sec)
exp: 4.8 -- mean: 5.41 -- last 10 values: 14, 6, 8, 6, 9, 5, 6, 9, 5, 6
0:00:30 load avg: 4.83 running: test_compileall (1.0 sec), test_tokenize (3.2 sec)
exp: 4.9 -- mean: 5.54 -- last 10 values: 6, 8, 6, 9, 5, 6, 9, 5, 6, 9
0:00:31 load avg: 4.90 running: test_compileall (2.1 sec), test_tokenize (4.3 sec)
exp: 4.9 -- mean: 5.59 -- last 10 values: 8, 6, 9, 5, 6, 9, 5, 6, 9, 7
0:00:32 load avg: 4.93 running: test_compileall (3.2 sec), test_tokenize (5.4 sec)
exp: 4.9 -- mean: 5.57 -- last 10 values: 6, 9, 5, 6, 9, 5, 6, 9, 7, 5
0:00:33 load avg: 4.94 [ 31/419] test_tokenize passed (5.5 sec) -- running: test_compileall (3.9 sec)
exp: 5.0 -- mean: 5.58 -- last 10 values: 9, 5, 6, 9, 5, 6, 9, 7, 5, 6
0:00:34 load avg: 4.95 running: test_compileall (4.9 sec), test_gettext (1.0 sec)
0:00:34 load avg: 4.95 [ 32/419] test_gettext passed (1.0 sec) -- running: test_compileall (5.5 sec)
exp: 5.0 -- mean: 5.66 -- last 10 values: 5, 6, 9, 5, 6, 9, 7, 5, 6, 8
0:00:35 load avg: 5.00 [ 33/419] test_c_locale_coercion passed -- running: test_compileall (6.4 sec)
exp: 5.0 -- mean: 5.67 -- last 10 values: 6, 9, 5, 6, 9, 7, 5, 6, 8, 6
0:00:36 load avg: 5.02 [ 34/419] test_with passed -- running: test_compileall (7.1 sec)
0:00:36 load avg: 5.02 [ 35/419] test_binhex passed -- running: test_compileall (7.7 sec)
exp: 5.1 -- mean: 5.71 -- last 10 values: 9, 5, 6, 9, 7, 5, 6, 8, 6, 7
0:00:37 load avg: 5.05 [ 36/419] test_generators passed -- running: test_compileall (8.5 sec)
exp: 5.3 -- mean: 6.06 -- last 10 values: 5, 6, 9, 7, 5, 6, 8, 6, 7, 18
0:00:38 load avg: 5.27 running: test_compileall (9.7 sec), test_smtpnet (1.2 sec)
0:00:39 load avg: 5.27 [ 37/419] test_smtpnet skipped (resource denied) -- running: test_compileall (9.9 sec)
test_smtpnet skipped -- Use of the 'network' resource not enabled
exp: 5.4 -- mean: 6.22 -- last 10 values: 6, 9, 7, 5, 6, 8, 6, 7, 18, 12
0:00:40 load avg: 5.38 running: test_compileall (10.9 sec), test_inspect (1.0 sec)
exp: 5.5 -- mean: 6.32 -- last 10 values: 9, 7, 5, 6, 8, 6, 7, 18, 12, 10
0:00:41 load avg: 5.45 running: test_compileall (12.1 sec), test_inspect (2.2 sec)
exp: 5.6 -- mean: 6.47 -- last 10 values: 7, 5, 6, 8, 6, 7, 18, 12, 10, 12
0:00:42 load avg: 5.56 running: test_compileall (13.3 sec), test_inspect (3.4 sec)
0:00:42 load avg: 5.56 [ 38/419] test_inspect passed (3.0 sec) -- running: test_compileall (13.7 sec)
exp: 5.6 -- mean: 6.54 -- last 10 values: 5, 6, 8, 6, 7, 18, 12, 10, 12, 9
0:00:43 load avg: 5.62 running: test_compileall (14.7 sec), test_xmlrpc (1.0 sec)
exp: 5.5 -- mean: 6.38 -- last 10 values: 6, 8, 6, 7, 18, 12, 10, 12, 9, 0
0:00:44 load avg: 5.53 running: test_compileall (15.7 sec), test_xmlrpc (2.0 sec)
exp: 5.4 -- mean: 6.22 -- last 10 values: 8, 6, 7, 18, 12, 10, 12, 9, 0, 0
0:00:45 load avg: 5.44 running: test_compileall (16.7 sec), test_xmlrpc (3.0 sec)
exp: 5.3 -- mean: 6.07 -- last 10 values: 6, 7, 18, 12, 10, 12, 9, 0, 0, 0
0:00:46 load avg: 5.35 running: test_compileall (17.7 sec), test_xmlrpc (4.0 sec)
exp: 5.3 -- mean: 5.93 -- last 10 values: 7, 18, 12, 10, 12, 9, 0, 0, 0, 0
0:00:47 load avg: 5.26 running: test_compileall (18.7 sec), test_xmlrpc (5.0 sec)
exp: 5.3 -- mean: 6.02 -- last 10 values: 18, 12, 10, 12, 9, 0, 0, 0, 0, 10
0:00:48 load avg: 5.34 running: test_compileall (19.8 sec), test_xmlrpc (6.1 sec)
exp: 5.2 -- mean: 5.89 -- last 10 values: 12, 10, 12, 9, 0, 0, 0, 0, 10, 0
0:00:49 load avg: 5.25 running: test_compileall (20.8 sec), test_xmlrpc (7.2 sec)
exp: 5.2 -- mean: 5.76 -- last 10 values: 10, 12, 9, 0, 0, 0, 0, 10, 0, 0
0:00:50 load avg: 5.16 running: test_compileall (21.8 sec), test_xmlrpc (8.2 sec)
exp: 5.1 -- mean: 5.64 -- last 10 values: 12, 9, 0, 0, 0, 0, 10, 0, 0, 0
0:00:51 load avg: 5.08 running: test_compileall (22.8 sec), test_xmlrpc (9.2 sec)
0:00:52 load avg: 5.08 [ 39/419] test_xmlrpc passed (9.2 sec) -- running: test_compileall (23.3 sec)
exp: 5.1 -- mean: 5.62 -- last 10 values: 9, 0, 0, 0, 0, 10, 0, 0, 0, 5
0:00:53 load avg: 5.07 running: test_compileall (24.4 sec), test_decimal (1.0 sec)
exp: 5.1 -- mean: 5.61 -- last 10 values: 0, 0, 0, 0, 10, 0, 0, 0, 5, 5
0:00:54 load avg: 5.07 running: test_compileall (25.5 sec), test_decimal (2.1 sec)
exp: 5.2 -- mean: 5.70 -- last 10 values: 0, 0, 0, 10, 0, 0, 0, 5, 5, 10
0:00:55 load avg: 5.15 running: test_compileall (26.6 sec), test_decimal (3.2 sec)
exp: 5.2 -- mean: 5.71 -- last 10 values: 0, 0, 10, 0, 0, 0, 5, 5, 10, 6
0:00:56 load avg: 5.17 running: test_compileall (27.7 sec), test_decimal (4.3 sec)
exp: 5.2 -- mean: 5.77 -- last 10 values: 0, 10, 0, 0, 0, 5, 5, 10, 6, 9
0:00:57 load avg: 5.23 running: test_compileall (28.8 sec), test_decimal (5.5 sec)
exp: 5.3 -- mean: 5.83 -- last 10 values: 10, 0, 0, 0, 5, 5, 10, 6, 9, 9
0:00:59 load avg: 5.29 running: test_compileall (29.9 sec), test_decimal (6.6 sec)
exp: 5.4 -- mean: 5.89 -- last 10 values: 0, 0, 0, 5, 5, 10, 6, 9, 9, 9
0:01:00 load avg: 5.36 running: test_compileall (31.0 sec), test_decimal (7.7 sec)
exp: 5.3 -- mean: 5.87 -- last 10 values: 0, 0, 5, 5, 10, 6, 9, 9, 9, 5
0:01:01 load avg: 5.35 running: test_compileall (32.0 sec), test_decimal (8.7 sec)
exp: 5.4 -- mean: 5.88 -- last 10 values: 0, 5, 5, 10, 6, 9, 9, 9, 5, 6
0:01:02 load avg: 5.36 running: test_compileall (33.1 sec), test_decimal (9.8 sec)
exp: 5.5 -- mean: 6.05 -- last 10 values: 5, 5, 10, 6, 9, 9, 9, 5, 6, 16
0:01:03 load avg: 5.54 running: test_compileall (34.2 sec), test_decimal (10.8 sec)
exp: 5.6 -- mean: 6.16 -- last 10 values: 5, 10, 6, 9, 9, 9, 5, 6, 16, 12
0:01:04 load avg: 5.64 running: test_compileall (35.2 sec), test_decimal (11.9 sec)
exp: 5.8 -- mean: 6.29 -- last 10 values: 10, 6, 9, 9, 9, 5, 6, 16, 12, 14
0:01:04 load avg: 5.78 [ 40/419] test_compileall passed (34.9 sec) -- running: test_decimal (12.1 sec)