GitHub - sharkdp/hyperfine at 094b76d23a74341de2e03e34eb15285895517b2b (original) (raw)
hyperfine
A command-line benchmarking tool (inspired by bench).
Demo: Benchmarking fd andfind:
Features
- Statistical analysis across multiple runs.
- Support for arbitrary shell commands.
- Constant feedback about the benchmark progress and current estimates.
- Warmup runs can be executed before the actual benchmark.
- Cache-clearing commands can be set up before each timing run.
- Statistical outlier detection.
- Export results to various formats: CSV, JSON, Markdown.
- Parameterized benchmarks.
- Cross-platform
Usage
Basic benchmark
To run a benchmark, you can simply call hyperfine <command>...
. The argument(s) can be any shell command. For example:
Hyperfine will automatically determine the number of runs to perform for each command. By default, it will perform at least 10 benchmarking runs. To change this, you can use the -m
/--min-runs
option:
hyperfine --min-runs 5 'sleep 0.2' 'sleep 3.2'
I/O-heavy programs
If the program execution time is limited by disk I/O, the benchmarking results can be heavily influenced by disk caches and whether they are cold or warm.
If you want to run the benchmark on a warm cache, you can use the -w
/--warmup
option to perform a certain number of program executions before the actual benchmark:
hyperfine --warmup 3 'grep -R TODO *'
Conversely, if you want to run the benchmark for a cold cache, you can use the -p
/--prepare
option to run a special command before each timing run. For example, to clear harddisk caches on Linux, you can run
sync; echo 3 | sudo tee /proc/sys/vm/drop_caches
To use this specific command with Hyperfine, call sudo -v
to temporarily gain sudo permissions and then call:
hyperfine --prepare 'sync; echo 3 | sudo tee /proc/sys/vm/drop_caches' 'grep -R TODO *'
Parameterized benchmarks
If you want to run a benchmark where only a single parameter is varied (say, the number of threads), you can use the -P
/--parameter-scan
option and call:
hyperfine --prepare 'make clean' --parameter-scan num_threads 1 12 'make -j {num_threads}'
Export results
Hyperfine has multiple options for exporting benchmark results: CSV, JSON, Markdown (see --help
text for details). To export results to Markdown, for example, you can use the --export-markdown
option that will create tables like this:
Command | Mean [ms] | Min…Max [ms] |
---|---|---|
find . -iregex '.*[0-9]\.jpg$' | 506.0 ± 8.1 | 495.4…518.6 |
find . -iname '*[0-9].jpg' | 304.9 ± 3.1 | 299.8…309.3 |
fd -HI '.*[0-9]\.jpg$' | 66.2 ± 5.8 | 62.5…86.3 |
Installation
On macOS
Hyperfine can be installed via Homebrew:
On Ubuntu
Download the appropriate .deb
package from the Release pageand install it via dpkg
:
wget https://github.com/sharkdp/hyperfine/releases/download/v1.3.0/hyperfine_1.3.0_amd64.deb
sudo dpkg -i hyperfine_1.3.0_amd64.deb
On Arch Linux
On Arch Linux, hyperfine can be installed from the AUR:
On Void Linux
Hyperfine can be installed via xbps
xbps-install -S hyperfine
With cargo (Linux, macOS, Windows)
Hyperfine can be installed via cargo:
Make sure that you use Rust 1.24 or higher.
From binaries (Linux, macOS, Windows)
Download the corresponding archive from the Release page.
Origin of the name
The name hyperfine was chosen in reference to the hyperfine levels of caesium 133 which play a crucial role in thedefinition of our base unit of time— the second.