Dom update 2024-01-11 by sandersn · Pull Request #57027 · microsoft/TypeScript (original) (raw)

@sandersn
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr

Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,508k (± 0.01%) 295,633k (± 0.01%) +125k (+ 0.04%) 295,601k 295,703k p=0.005 n=6
Parse Time 2.65s (± 0.15%) 2.66s (± 0.31%) ~ 2.65s 2.67s p=0.527 n=6
Bind Time 0.82s (± 0.66%) 0.82s (± 0.63%) ~ 0.82s 0.83s p=0.640 n=6
Check Time 8.13s (± 0.38%) 8.12s (± 0.33%) ~ 8.08s 8.16s p=1.000 n=6
Emit Time 7.09s (± 0.36%) 7.09s (± 0.55%) ~ 7.04s 7.13s p=0.686 n=6
Total Time 18.70s (± 0.20%) 18.68s (± 0.30%) ~ 18.61s 18.76s p=0.872 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 193,924k (± 1.44%) 192,480k (± 1.21%) ~ 191,501k 197,247k p=0.230 n=6
Parse Time 1.35s (± 0.56%) 1.36s (± 0.60%) ~ 1.35s 1.37s p=0.383 n=6
Bind Time 0.72s (± 0.00%) 0.72s (± 0.00%) ~ 0.72s 0.72s p=1.000 n=6
Check Time 9.36s (± 0.37%) 9.34s (± 0.18%) ~ 9.32s 9.36s p=0.333 n=6
Emit Time 2.64s (± 0.31%) 2.63s (± 0.32%) -0.01s (- 0.44%) 2.62s 2.64s p=0.045 n=6
Total Time 14.07s (± 0.22%) 14.04s (± 0.13%) ~ 14.03s 14.07s p=0.102 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,399k (± 0.01%) 347,393k (± 0.01%) ~ 347,363k 347,431k p=0.810 n=6
Parse Time 2.47s (± 0.47%) 2.46s (± 0.47%) ~ 2.44s 2.47s p=0.357 n=6
Bind Time 0.93s (± 0.00%) 0.93s (± 0.56%) ~ 0.92s 0.93s p=0.174 n=6
Check Time 6.88s (± 0.63%) 6.88s (± 0.47%) ~ 6.83s 6.91s p=0.746 n=6
Emit Time 4.05s (± 0.20%) 4.05s (± 0.72%) ~ 4.02s 4.10s p=0.411 n=6
Total Time 14.33s (± 0.26%) 14.32s (± 0.36%) ~ 14.25s 14.41s p=1.000 n=6
TFS - node (v18.15.0, x64)
Memory used 302,774k (± 0.01%) 302,787k (± 0.00%) ~ 302,774k 302,809k p=0.065 n=6
Parse Time 2.01s (± 0.98%) 2.00s (± 1.17%) ~ 1.96s 2.02s p=0.567 n=6
Bind Time 1.01s (± 1.04%) 1.01s (± 1.49%) ~ 0.99s 1.03s p=1.000 n=6
Check Time 6.31s (± 0.47%) 6.29s (± 0.31%) ~ 6.26s 6.31s p=0.360 n=6
Emit Time 3.58s (± 0.49%) 3.57s (± 0.50%) ~ 3.55s 3.59s p=0.209 n=6
Total Time 12.89s (± 0.38%) 12.88s (± 0.21%) ~ 12.83s 12.91s p=0.376 n=6
material-ui - node (v18.15.0, x64)
Memory used 508,279k (± 0.00%) 508,406k (± 0.00%) +128k (+ 0.03%) 508,385k 508,447k p=0.005 n=6
Parse Time 2.59s (± 0.40%) 2.60s (± 0.72%) ~ 2.57s 2.62s p=0.933 n=6
Bind Time 1.00s (± 1.17%) 0.99s (± 0.82%) ~ 0.98s 1.00s p=0.401 n=6
Check Time 17.07s (± 0.54%) 17.10s (± 0.61%) ~ 16.97s 17.25s p=0.688 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.66s (± 0.45%) 20.69s (± 0.56%) ~ 20.54s 20.85s p=0.521 n=6
xstate - node (v18.15.0, x64)
Memory used 512,970k (± 0.01%) 513,336k (± 0.01%) +366k (+ 0.07%) 513,286k 513,366k p=0.005 n=6
Parse Time 3.28s (± 0.23%) 3.28s (± 0.27%) ~ 3.27s 3.29s p=0.798 n=6
Bind Time 1.54s (± 0.34%) 1.54s (± 0.00%) ~ 1.54s 1.54s p=0.174 n=6
Check Time 2.82s (± 0.48%) 2.84s (± 0.82%) ~ 2.81s 2.87s p=0.460 n=6
Emit Time 0.07s (± 5.69%) 0.07s (± 0.00%) ~ 0.07s 0.07s p=0.405 n=6
Total Time 7.71s (± 0.23%) 7.73s (± 0.30%) ~ 7.70s 7.76s p=0.258 n=6

System info unknown

Hosts

Scenarios

Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr

Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,338ms (± 0.42%) 2,345ms (± 0.89%) ~ 2,310ms 2,369ms p=0.470 n=6
Req 2 - geterr 5,555ms (± 1.23%) 5,495ms (± 1.39%) ~ 5,405ms 5,606ms p=0.230 n=6
Req 3 - references 324ms (± 0.23%) 325ms (± 1.27%) ~ 321ms 332ms p=1.000 n=6
Req 4 - navto 271ms (± 0.60%) 275ms (± 1.33%) +5ms (+ 1.66%) 270ms 278ms p=0.026 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 93ms (± 1.26%) 86ms (± 6.65%) ~ 79ms 93ms p=0.118 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,488ms (± 0.47%) 2,482ms (± 0.67%) ~ 2,465ms 2,509ms p=0.470 n=6
Req 2 - geterr 4,184ms (± 1.89%) 4,162ms (± 1.83%) ~ 4,097ms 4,264ms p=0.936 n=6
Req 3 - references 330ms (± 0.37%) 332ms (± 1.20%) ~ 328ms 339ms p=0.871 n=6
Req 4 - navto 286ms (± 0.97%) 285ms (± 0.53%) ~ 283ms 286ms p=0.615 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 76ms (± 3.66%) 77ms (± 7.65%) ~ 74ms 89ms p=1.000 n=6
xstateTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,602ms (± 0.38%) 2,615ms (± 0.66%) ~ 2,584ms 2,630ms p=0.128 n=6
Req 2 - geterr 1,711ms (± 2.73%) 1,701ms (± 2.73%) ~ 1,650ms 1,759ms p=0.810 n=6
Req 3 - references 113ms (± 9.29%) 116ms (± 9.47%) ~ 105ms 126ms p=0.865 n=6
Req 4 - navto 366ms (± 0.62%) 366ms (± 0.68%) ~ 364ms 371ms p=0.807 n=6
Req 5 - completionInfo count 2,073 (± 0.00%) 2,078 (± 0.00%) +5 (+ 0.24%) 2,078 2,078 p=0.001 n=6
Req 5 - completionInfo 308ms (± 2.26%) 309ms (± 1.57%) ~ 303ms 316ms p=0.871 n=6

System info unknown

Hosts

Scenarios

Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr

Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 154.85ms (± 0.19%) 154.77ms (± 0.19%) -0.08ms (- 0.05%) 153.70ms 157.95ms p=0.012 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 230.68ms (± 0.15%) 230.57ms (± 0.17%) -0.11ms (- 0.05%) 229.30ms 235.18ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 230.97ms (± 0.20%) 231.12ms (± 0.19%) +0.15ms (+ 0.07%) 229.59ms 236.65ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 230.78ms (± 0.19%) 230.71ms (± 0.18%) ~ 229.16ms 233.54ms p=0.173 n=600

System info unknown

Hosts

Scenarios

Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks