bpo-36044: Reduce number of unit tests run for PGO build. by nascheme · Pull Request #14702 · python/cpython (original) (raw)
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Conversation11 Commits7 Checks0 Files changed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
[ Show hidden characters]({{ revealButtonHref }})
This speeds up the PGO profile generation by a factor of about 15X.
Running the full unit test suite is slow. This may result in a
slightly less optimized build since not as many code branches will
be executed. If you are willing to wait for the slower build, the
old behavior can be enabled as follows:
./configure [...] --with-profile-task='-m test --pgo'
https://bugs.python.org/issue36044
This speeds up the PGO profile generation by a factor of about 15X. Running the full unit test suite is slow. This may result in a slightly less optimized build since not as many code branches will be executed. If you are willing to wait for the slower build, the old behavior can be enabled as follows:
./configure [...] --with-profile-task='-m test --pgo'
Add --pgo-extended option to regrtest.
| @@ -264,7 +264,9 @@ def _create_parser(): |
|---|
| help='only write the name of test cases that will be run' |
| ' , don\'t execute them') |
| group.add_argument('-P', '--pgo', dest='pgo', action='store_true', |
| help='enable Profile Guided Optimization training') |
| help='enable Profile Guided Optimization (PGO) training') |
| group.add_argument('--pgo-extended', action='store_true', |
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand what "extended" means. I suggest to rename the option to "--pgo-all-tests" and rewrite the help to explain that it runs all tests by default.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Think list.extend(). I.e. run more tests. I didn't want to use --pgo-all because we don't actually run all the tests. Some are excluded using the support.PGO flag. However, I don't feel strongly so if you really like --pgo-all, I will change it.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, --pgo-harder? 😄
| # chosen based on the following criteria: either they exercise a commonly used |
|---|
| # C extension module or type, or they run some relatively typical Python code. |
| # Long running tests should be avoided because the PGO instrumented executable |
| # runs slowly. |
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would prefer to keep main.py small. Would you mind to create libregrtest/pgo.py for this list please?
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok.
more detail and a performance disclaimer.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i updated the news entry wording a bit.
| help='enable Profile Guided Optimization training') |
|---|
| help='enable Profile Guided Optimization (PGO) training') |
| group.add_argument('--pgo-extended', action='store_true', |
| help='enable extended PGO training (slower training)') |
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd add to the description of this that "User are encouraged to benchmark the interpreter resulting from a PROFILE_TASK with this flag to decide if it is meaningfully faster than those produced using --pgo for their use cases".
I don't care what the flag name is, --pgo-extended is fine by me. as is --pgo-very-long-training-run (a flag who's length matches the build time seems... appropriate). ;)
The configure flags --with-something are intended for enabling optional external software. Just use a variable instead.
Thanks @nascheme for the PR 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒⛏🤖
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request
Reduce the number of unit tests run for the PGO generation task. This speeds up the task by a factor of about 15x. Running the full unit test suite is slow. This change may result in a slightly less optimized build since not as many code branches will be executed. If you are willing to wait for the much slower build, the old behavior can be restored using './configure [..] PROFILE_TASK="-m test --pgo-extended"'. We make no guarantees as to which PGO task set produces a faster build. Users who care should run their own relevant benchmarks as results can depend on the environment, workload, and compiler tool chain. (cherry picked from commit 4e16a4a)
Co-authored-by: Neil Schemenauer nas-github@arctrix.com
miss-islington added a commit that referenced this pull request
Reduce the number of unit tests run for the PGO generation task. This speeds up the task by a factor of about 15x. Running the full unit test suite is slow. This change may result in a slightly less optimized build since not as many code branches will be executed. If you are willing to wait for the much slower build, the old behavior can be restored using './configure [..] PROFILE_TASK="-m test --pgo-extended"'. We make no guarantees as to which PGO task set produces a faster build. Users who care should run their own relevant benchmarks as results can depend on the environment, workload, and compiler tool chain. (cherry picked from commit 4e16a4a)
Co-authored-by: Neil Schemenauer nas-github@arctrix.com
lisroach pushed a commit to lisroach/cpython that referenced this pull request
Reduce the number of unit tests run for the PGO generation task. This speeds up the task by a factor of about 15x. Running the full unit test suite is slow. This change may result in a slightly less optimized build since not as many code branches will be executed. If you are willing to wait for the much slower build, the old behavior can be restored using './configure [..] PROFILE_TASK="-m test --pgo-extended"'. We make no guarantees as to which PGO task set produces a faster build. Users who care should run their own relevant benchmarks as results can depend on the environment, workload, and compiler tool chain.
DinoV pushed a commit to DinoV/cpython that referenced this pull request
Reduce the number of unit tests run for the PGO generation task. This speeds up the task by a factor of about 15x. Running the full unit test suite is slow. This change may result in a slightly less optimized build since not as many code branches will be executed. If you are willing to wait for the much slower build, the old behavior can be restored using './configure [..] PROFILE_TASK="-m test --pgo-extended"'. We make no guarantees as to which PGO task set produces a faster build. Users who care should run their own relevant benchmarks as results can depend on the environment, workload, and compiler tool chain.
websurfer5 pushed a commit to websurfer5/cpython that referenced this pull request
Reduce the number of unit tests run for the PGO generation task. This speeds up the task by a factor of about 15x. Running the full unit test suite is slow. This change may result in a slightly less optimized build since not as many code branches will be executed. If you are willing to wait for the much slower build, the old behavior can be restored using './configure [..] PROFILE_TASK="-m test --pgo-extended"'. We make no guarantees as to which PGO task set produces a faster build. Users who care should run their own relevant benchmarks as results can depend on the environment, workload, and compiler tool chain.