Fix approved stub ignore, remove normpath by hauntsaninja · Pull Request #18045 · python/mypy (original) (raw)

Benchmark on an earlier version of this PR. Looks like 0.3s saving on long and 0.7s saving on openai

+ /tmp/mypy_primer/timer_mypy_ecf5391/venv/bin/python -m mypy --version
mypy 1.14.0+dev.ecf539147dfebcbe0d1b3817729bcf618ac5c286 (compiled: yes)
+ hyperfine -w 1 -M 5 --export-json benchjson/ecf5391_clean.json '/tmp/mypy_primer/timer_mypy_ecf5391/venv/bin/python -m mypy -c '\''import torch'\'' --no-incremental --python-executable clean/bin/python'
Benchmark 1: /tmp/mypy_primer/timer_mypy_ecf5391/venv/bin/python -m mypy -c 'import torch' --no-incremental --python-executable clean/bin/python
  Time (mean ± σ):     19.751 s ±  0.322 s    [User: 18.340 s, System: 1.397 s]
  Range (min … max):   19.522 s … 20.312 s    5 runs
 
+ hyperfine -w 1 -M 5 --export-json benchjson/ecf5391_long.json '/tmp/mypy_primer/timer_mypy_ecf5391/venv/bin/python -m mypy -c '\''import torch'\'' --no-incremental --python-executable long/bin/python'
Benchmark 1: /tmp/mypy_primer/timer_mypy_ecf5391/venv/bin/python -m mypy -c 'import torch' --no-incremental --python-executable long/bin/python
  Time (mean ± σ):     19.929 s ±  0.342 s    [User: 18.567 s, System: 1.349 s]
  Range (min … max):   19.533 s … 20.461 s    5 runs
 
+ hyperfine -w 1 -M 5 --export-json benchjson/ecf5391_oai.json '/tmp/mypy_primer/timer_mypy_ecf5391/venv/bin/python -m mypy -c '\''import torch'\'' --no-incremental --python-executable /opt/oai/bin/python'
Benchmark 1: /tmp/mypy_primer/timer_mypy_ecf5391/venv/bin/python -m mypy -c 'import torch' --no-incremental --python-executable /opt/oai/bin/python
  Time (mean ± σ):     26.182 s ±  0.300 s    [User: 24.031 s, System: 2.130 s]
  Range (min … max):   25.862 s … 26.671 s    5 runs
 
+ hyperfine -w 2 -M 5 --export-json benchjson/ecf5391_clean_inc.json '/tmp/mypy_primer/timer_mypy_ecf5391/venv/bin/python -m mypy -c '\''import torch'\'' --python-executable clean/bin/python'
Benchmark 1: /tmp/mypy_primer/timer_mypy_ecf5391/venv/bin/python -m mypy -c 'import torch' --python-executable clean/bin/python
  Time (mean ± σ):      2.065 s ±  0.012 s    [User: 1.696 s, System: 0.369 s]
  Range (min … max):    2.051 s …  2.084 s    5 runs
 
+ hyperfine -w 2 -M 5 --export-json benchjson/ecf5391_long_inc.json '/tmp/mypy_primer/timer_mypy_ecf5391/venv/bin/python -m mypy -c '\''import torch'\'' --python-executable long/bin/python'
Benchmark 1: /tmp/mypy_primer/timer_mypy_ecf5391/venv/bin/python -m mypy -c 'import torch' --python-executable long/bin/python
  Time (mean ± σ):      2.093 s ±  0.012 s    [User: 1.733 s, System: 0.359 s]
  Range (min … max):    2.079 s …  2.106 s    5 runs
 
+ hyperfine -w 2 -M 5 --export-json benchjson/ecf5391_oai_inc.json '/tmp/mypy_primer/timer_mypy_ecf5391/venv/bin/python -m mypy -c '\''import torch'\'' --python-executable /opt/oai/bin/python'
Benchmark 1: /tmp/mypy_primer/timer_mypy_ecf5391/venv/bin/python -m mypy -c 'import torch' --python-executable /opt/oai/bin/python
  Time (mean ± σ):      3.239 s ±  0.075 s    [User: 2.685 s, System: 0.553 s]
  Range (min … max):    3.179 s …  3.344 s    5 runs