zstd: Improve match speed of fastest level. by klauspost · Pull Request #241 · klauspost/compress (original) (raw)

Inline matchlen code and slightly simplify it. This loses a small amount of compression but gives a big speedup. This seems reasonable at the fastest level.

Before/after, best of 3 runs each:

file	out	level	insize	outsize	millis	mb/s
enwik9	zskp	1	1000000000	343831004	4202	226.91
enwik9	zskp	1	1000000000	343848582	3682	258.97

github-june-2days-2019.json	zskp	1	6273951764	698824137	10787	554.67
github-june-2days-2019.json	zskp	1	6273951764	699045015	10474	571.23

github-ranks-backup.bin	zskp	1	1862623243	454018274	4833	367.54
github-ranks-backup.bin	zskp	1	1862623243	454072815	4568	388.82

rawstudio-mint14.tar	zskp	1	8558382592	3667295557	21060	387.55
rawstudio-mint14.tar	zskp	1	8558382592	3667489370	20207	403.90

nyc-taxi-data-10M.csv	zskp	1	3325605752	641244049	10954	289.53
nyc-taxi-data-10M.csv	zskp	1	3325605752	641339945	9668	328.01

gob-stream	zskp	1	1911399616	234947276	3514	518.62
gob-stream	zskp	1	1911399616	235022249	3354	543.36