introduce PGO on the dist-x86_64-msvc builder · rust-lang/rust@9027f82 (original) (raw)
`@@ -14,13 +14,26 @@ DOWNLOADED_LLVM=/rustroot
`
14
14
`# The main directory where the build occurs, which can be different between linux and windows
`
15
15
`BUILD_ROOT=$CHECKOUT/obj
`
16
16
``
``
17
`+
if isWindows; then
`
``
18
`+
CHECKOUT=$(pwd)
`
``
19
`+
DOWNLOADED_LLVM=$CHECKOUT/citools/clang-rust
`
``
20
`+
BUILD_ROOT=$CHECKOUT
`
``
21
`+
fi
`
``
22
+
17
23
`# The various build artifacts used in other commands: to launch rustc builds, build the perf
`
18
24
`# collector, and run benchmarks to gather profiling data
`
19
25
`BUILD_ARTIFACTS=$BUILD_ROOT/build/$PGO_HOST
`
20
26
`RUSTC_STAGE_0=$BUILD_ARTIFACTS/stage0/bin/rustc
`
21
27
`CARGO_STAGE_0=$BUILD_ARTIFACTS/stage0/bin/cargo
`
22
28
`RUSTC_STAGE_2=$BUILD_ARTIFACTS/stage2/bin/rustc
`
23
29
``
``
30
`+
Windows needs these to have the .exe extension
`
``
31
`+
if isWindows; then
`
``
32
`+
RUSTC_STAGE_0="${RUSTC_STAGE_0}.exe"
`
``
33
`+
CARGO_STAGE_0="${CARGO_STAGE_0}.exe"
`
``
34
`+
RUSTC_STAGE_2="${RUSTC_STAGE_2}.exe"
`
``
35
`+
fi
`
``
36
+
24
37
`# Make sure we have a temporary PGO work folder
`
25
38
`PGO_TMP=/tmp/tmp-pgo
`
26
39
`mkdir -p $PGO_TMP
`
`@@ -80,13 +93,26 @@ LLVM_PROFILE_DIR=${LLVM_PROFILE_DIRECTORY_ROOT}/prof-%p python3 $CHECKOUT/x.py b
`
80
93
` --stage 2 library/std \
`
81
94
` --llvm-profile-generate
`
82
95
``
83
``
`-
Compile rustc perf
`
84
``
`-
cp -r /tmp/rustc-perf $RUSTC_PERF
`
85
``
`-
chown -R (whoami):(whoami): (whoami):RUSTC_PERF
`
``
96
`+
Compile rustc-perf:
`
``
97
`+
- get the expected commit source code: on linux, the Dockerfile downloads a source archive before
`
``
98
`+
running this script. On Windows, we do that here.
`
``
99
`+
if isLinux; then
`
``
100
`+
cp -r /tmp/rustc-perf $RUSTC_PERF
`
``
101
`+
chown -R (whoami):(whoami): (whoami):RUSTC_PERF
`
``
102
`+
else
`
``
103
`+
rustc-perf version from 2022-05-18
`
``
104
`+
PERF_COMMIT=f66cc8f3e04392b0e2fd811f21fd1ece6ebaded3
`
``
105
`+
retry curl -LS -o $PGO_TMP/perf.zip \
`
``
106
`+
https://github.com/rust-lang/rustc-perf/archive/$PERF_COMMIT.zip && \
`
``
107
`+
cd $PGO_TMP && unzip -q perf.zip && \
`
``
108
`+
mv rustc-perf-$PERF_COMMIT $RUSTC_PERF && \
`
``
109
`+
rm perf.zip
`
``
110
`+
fi
`
``
111
+
``
112
`+
- build rustc-perf's collector ahead of time, which is needed to make sure the rustc-fake binary
`
``
113
`+
used by the collector is present.
`
86
114
`cd $RUSTC_PERF
`
87
115
``
88
``
`-
Build rustc-perf's collector ahead of time, which is needed to make sure the rustc-fake
`
89
``
`-
binary used by the collector is present.
`
90
116
`RUSTC=$RUSTC_STAGE_0 \
`
91
117
`RUSTC_BOOTSTRAP=1 \
`
92
118
`$CARGO_STAGE_0 build -p collector
`
`@@ -128,13 +154,21 @@ python3 CHECKOUT/x.pybuild−−target=CHECKOUT/x.py build --target=CHECKOUT/x.pybuild−−target=PGO_HOST --host=$PGO_HOST \
`
128
154
``
129
155
`` # Here we're profiling the rustc frontend, so we also include Check.
``
130
156
`# The benchmark set includes various stress tests that put the frontend under pressure.
`
131
``
`-
The profile data is written into a single filepath that is being repeatedly merged when each
`
132
``
`-
rustc invocation ends. Empirically, this can result in some profiling data being lost.
`
133
``
`-
That's why we override the profile path to include the PID. This will produce many more profiling
`
134
``
`-
files, but the resulting profile will produce a slightly faster rustc binary.
`
135
``
`-
LLVM_PROFILE_FILE=${RUSTC_PROFILE_DIRECTORY_ROOT}/default_%m_%p.profraw gather_profiles \
`
136
``
`-
"Check,Debug,Opt" "All" \
`
137
``
`-
"externs,ctfe-stress-5,cargo-0.60.0,token-stream-stress,match-stress,tuple-stress,diesel-1.4.8,bitmaps-3.1.0"
`
``
157
`+
if isLinux; then
`
``
158
`+
The profile data is written into a single filepath that is being repeatedly merged when each
`
``
159
`+
rustc invocation ends. Empirically, this can result in some profiling data being lost. That's
`
``
160
`+
why we override the profile path to include the PID. This will produce many more profiling
`
``
161
`+
files, but the resulting profile will produce a slightly faster rustc binary.
`
``
162
`+
LLVM_PROFILE_FILE=${RUSTC_PROFILE_DIRECTORY_ROOT}/default_%m_%p.profraw gather_profiles \
`
``
163
`+
"Check,Debug,Opt" "All" \
`
``
164
`+
"externs,ctfe-stress-5,cargo-0.60.0,token-stream-stress,match-stress,tuple-stress,diesel-1.4.8,bitmaps-3.1.0"
`
``
165
`+
else
`
``
166
`+
On windows, we don't do that yet (because it generates a lot of data, hitting disk space
`
``
167
`+
limits on the builder), and use the default profraw merging behavior.
`
``
168
`+
gather_profiles \
`
``
169
`+
"Check,Debug,Opt" "All" \
`
``
170
`+
"externs,ctfe-stress-5,cargo-0.60.0,token-stream-stress,match-stress,tuple-stress,diesel-1.4.8,bitmaps-3.1.0"
`
``
171
`+
fi
`
138
172
``
139
173
`RUSTC_PROFILE_MERGED_FILE=$PGO_TMP/rustc-pgo.profdata
`
140
174
``