Extremely fast non-cryptographic hash algorithm (original) (raw)

• Documentation
• Index
• Latest Release
• Issue Tracker
• GitHub

Benchmarks ------------------------- The reference system uses an Intel i7-9700K cpu, and runs Ubuntu x64 20.04. The [open source benchmark program] is compiled with `clang` v10.0 using `-O3` flag. | Hash Name | Width | Bandwidth (GB/s) | Small Data Velocity | Quality | Comment | | --------- | ----- | ---------------- | ----- | --- | --- | | __XXH3__ (SSE2) | 64 | 31.5 GB/s | 133.1 | 10 | | __XXH128__ (SSE2) | 128 | 29.6 GB/s | 118.1 | 10 | | _memcpy_ | N/A | 28.0 GB/s | N/A | N/A | _for reference_ | City64 | 64 | 22.0 GB/s | 76.6 | 10 | | T1ha2 | 64 | 22.0 GB/s | 99.0 | 9 | Slightly worse [collisions] | City128 | 128 | 21.7 GB/s | 57.7 | 10 | | __XXH64__ | 64 | 19.4 GB/s | 71.0 | 10 | | SpookyHash | 64 | 19.3 GB/s | 53.2 | 10 | | Mum | 64 | 18.0 GB/s | 67.0 | 9 | Slightly worse [collisions] | __XXH32__ | 32 | 9.7 GB/s | 71.9 | 10 | | City32 | 32 | 9.1 GB/s | 66.0 | 10 | | Murmur3 | 32 | 3.9 GB/s | 56.1 | 10 | | SipHash | 64 | 3.0 GB/s | 43.2 | 10 | | FNV64 | 64 | 1.2 GB/s | 62.7 | 5 | Poor avalanche properties | Blake2 | 256 | 1.1 GB/s | 5.1 | 10 | Cryptographic | SHA1 | 160 | 0.8 GB/s | 5.6 | 10 | Cryptographic but broken | MD5 | 128 | 0.6 GB/s | 7.8 | 10 | Cryptographic but broken [open source benchmark program]: https://github.com/Cyan4973/xxHash/tree/release/tests/bench [collisions]: https://github.com/Cyan4973/xxHash/wiki/Collision-ratio-comparison#collision-study note 1: Small data velocity is a _rough_ evaluation of algorithm's efficiency on small data. For more detailed analysis, please refer to [the wiki](https://github.com/Cyan4973/xxHash/wiki/Performance-comparison#benchmarks-concentrating-on-small-data-). note 2: On 32-bit applications, the ranking between hashes differs drastically, strongly favoring 32-bit hashes, See [this section of the wiki](https://github.com/Cyan4973/xxHash/wiki/Performance-comparison#bandwidth-on-x86-with-various-data-size-of-len-2n-32-bit-friendliness) for illustration. note 3: some algorithms feature _faster than RAM_ speed. In which case, they can only reach their full speed when input data is already in CPU cache (L3 or better). Otherwise, they max out at RAM speed limit.
## Multiple languages The following versions produce xxHash-compatible results in different languages. |Language |Author |URL |-- |-- |-- |__Python__ |Yue Du |https://pypi.python.org/pypi/xxhash/ |__C++__ (simple) |Stefan Brumme |https://create.stephan-brumme.com/xxhash/ |__C++ 17__ |Red Gavin |https://github.com/RedSpah/xxhash\_cpp |__C++__ constexpr (XXH64) |Daniel Kirchner |https://github.com/ekpyron/xxhashct |__C++__ constexpr (XXH32) |Takayuki Matsuoka |https://github.com/Cyan4973/xxHash/issues/496 |__C++__ constexpr (XXH3) |chys87 |https://github.com/chys87/constexpr-xxh3 |__C__ multithreaded |Shawn Bayern |https://github.com/shawnbayern/xxHash |__Java__ |Adrien Grand |https://github.com/lz4/lz4-java/tree/master/src/java/net/jpountz/xxhash |__Java__ (XXH3, XXH128) |James Z.M. Gao |https://github.com/OpenHFT/Zero-Allocation-Hashing/blob/master/src/main/java/net/openhft/hashing/XXH3.java |__Java__ (XXH3, port) |Otmar Ertl |https://github.com/dynatrace-oss/hash4j/blob/main/src/main/java/com/dynatrace/hash4j/hashing/XXH3\_64.java |__C#__ (std pkg) |Microsoft |https://learn.microsoft.com/en-us/dotnet/api/system.io.hashing?view=net-9.0-pp |__C#__ (port) |Melnik Alexander |https://github.com/uranium62/xxHash |__C#__ (.net std 2.0) |Sedat Kapanoğlu |https://github.com/ssg/HashDepot#xxhash |__JavaScript__ (WebAssembly)|Michael Jungo |https://www.npmjs.com/package/xxhash-wasm |__JavaScript__ (port) |Pierre Curto |https://npmjs.org/package/xxhashjs |__JavaScript__ (nodeJS) |Brian White |https://npmjs.org/package/xxhash |__JavaScript__ (nodeJS, xxh3)|Nhan Khong |https://github.com/ktrongnhan/xxhash-addon |__JavaScript__ (React Native)|Alex Shumihin |https://github.com/pioner92/react-native-xxhash |__JSX__ (static JavaScript)|Yoshiki Shibukawa |https://www.npmjs.org/package/xxhash.jsx |__Go__ (XXH64) |Ahmed Waheed |https://github.com/OneOfOne/xxhash |__Go + ASM__ |Caleb Spare |https://github.com/cespare/xxhash |__Go__ (XXH3) |Jeff Wendling |https://github.com/zeebo/xxh3 |__Pascal__ |Vojtěch Čihák |http://sourceforge.net/projects/xxhashfpc |__PHP8__ |Anatol Belski |https://php.watch/versions/8.1/xxHash |__PHP7__ |Craig R Megasaxon |https://github.com/Megasaxon/php-xxhash |__PHP__ |Nir Heimann |https://github.com/nheimann1/php-xxhash |__PHP__ (port) |Scott Dutton |https://github.com/exussum12/xxhash |__Rust__ |Jake Goulding |https://libraries.io/cargo/twox-hash |__Rust__ (const xxh3) |Arthur Martirosyan |https://crates.io/crates/xxhash-rust |__Ruby__ (port) |Justin W Smith |http://rubygems.org/gems/ruby-xxHash |__Ruby__ (wrapper) |Vasiliy Ermolovich |https://rubygems.org/gems/xxhash |__Ruby__ (digest::class) |konsolebox |https://rubygems.org/gems/digest-xxhash |__Swift__ |Daisuke T |https://github.com/daisuke-t-jp/xxHash-Swift |__Kotlin__ |Matthew Dolan |https://github.com/appmattus/crypto/tree/main/cryptohash |__R__ |Dirk Eddelbuettel |https://github.com/eddelbuettel/digest |__R__ (XXH3) |mikefc |https://github.com/coolbutuseless/xxhashlite |__Perl__ |Sanko Robinson |https://metacpan.org/module/Digest::xxHash |__Perl__ (streaming) |Bela Bodecs |https://github.com/DoubleBB/digest-xxhash64 |__D__ |Masahiro Nakagawa |https://github.com/repeatedly/xxhash-d |__Julia__ |Hanan Rosemarin |https://github.com/hros/XXhash.jl |__Scala__ |Desmond Yeung |https://github.com/desmondyeung/scala-hashing |__Lua__ (binding) |Masatoshi Teruya |https://github.com/mah0x211/lua-xxhash |__Lua__ (jit, XXH32) |szensk |https://github.com/szensk/luaxxhash |__Lua__ (jit, XXH64) |Soojin Nam |https://github.com/sjnam/luajit-xxHash |__OCaml__ |Pieter Goetschalckx|http://opam.ocaml.org/packages/xxhash/ |__Haskell__ |Henri Verroken |http://hackage.haskell.org/package/xxhash-ffi |__Haskell__ (port) |Christian Marie |http://hackage.haskell.org/package/xxhash |__Dart__ (XXH3) |SamJakob |https://pub.dev/packages/xxh3 |__Erlang__ |Pierre Matri |https://github.com/pierresforge/erlang-xxhash |__Erlang__ (XXH3) |Ali Farhadi |https://github.com/farhadi/xxh3 |__PicoLisp__ |mpech |https://git.envs.net/mpech/xxhash-picolisp |__Elixir__ (nif) |Ali Farhadi |https://github.com/farhadi/xxh3 |__Elixir__ (port) |Mykola Konyk |https://github.com/ttvd/elixir-xxhash |__Tcl__ (XXH32, port) |D. Bohdan |https://wiki.tcl-lang.org/48790 |__Bash__ (port) |Devin Hussey |https://github.com/easyaspi314/xxbash |__sh__ (port) |Jan Chren |https://gitlab.com/rindeal/xxHashish |__Crystal__ |Lucjan Suski |https://github.com/methyl/xxhash |__Zig__ |ziglang |https://github.com/Cyan4973/xxHash/issues/1001
## xxHash is used by #### Databases

• PrestoDB
• RocksDB
• MySQL
• ClickHouse
• ArangoDB
• PGroonga
• Spark
• MariaDB
• Groonga

#### Games

• CoD Black Ops
• Mafia
• Minecraft
• PPSSPP
• Dolphin
• Cxbx-reloaded
• Cocos2D
• LWJGL
• Freecell Solver

#### Filters

• HAProxy
• Rspamd
• pfSense
• fio
• bloomxx
• C & Python Bloom Filter
• LUA Bloom Filter

#### File Transfer

• Netflix
• rsync
• LZ4
• Silverstack
• Rapidcopy
• Hedge
• fastcopy
• TeraCopy
• Hammer 2
• CHK

#### Other

• Linux
• Microsoft Azure
• Visual Studio
• NSight compute
• Dorado
• PKG
• Qemu
• btrfs
• Xpra
• TeamViewer
• Factor
• nVBio
• dvisvgm
• FastBuild
• Keypirinha
• QuickHash

### Special Thanks to - [Takayuki Matsuoka](https://github.com/t-mat/) for creating `xxhsum -c` and the canvas of this webpage. - [Mathias Westerdahl](https://github.com/JCash) for introducing the first version of XXH64. - [Devin Hussey](https://github.com/easyaspi314) for the excellent low-level optimizations on XXH3 and XXH128. ---