Implement minimal implementation of HybridCache by mgravell · Pull Request #55147 · dotnet/aspnetcore (original) (raw)
Implement minimal implementation of HybridCache
This is the first "make something that works" iteration of HybridCache, that builds on the abstract API proposed in #55084 (epic: #53255 also: #54647); it will be re-targeted at main as soon as #55084 is merged
Broad design overview in src/Caching/Hybrid/src/Internal/readme.md
Implemented this wave:
- stampede protection (concurrent request combining)
- serialization (including configuration)
- underlying service invocation
- L2 (distributed cache) DI, read, and write-back
- L1 (memory cache) DI, read, and write-back
- immutable vs mutable type support (defensive copies for mutable)
- support for legacy (
byte[]) and "buffer" (ReadOnlySequence<byte>,IBufferWriter<byte>) L2 backends - flags detection to disable L1, L2, underlying
- direct set-value with L1/L2 write
- Redis and SQL Server support for "buffer" L2 backend
- supports net9.0; ns2.0; ns2.1; netfx
- benchmarks for
IDistributedCachevsIBufferDistributedCache - extensive tests for Redis and SQL Server implementations of
IBufferDistributedCache(and we found a SqlClient glitch: Inconsistent handling of empty BLOB slices as parameters SqlClient#2465)
Explicitly not implemented yet:
- tagging
- L2-assisted cache invalidation
- additional storage metadata
- metrics
- logging
- compression
- support for L3+
Here's the basic headline numbers comparing HybridCache to a class IDistributedCache "get, test, deserialize | fetch+serialize+set" loop using a local Redis backend and a simple POCO type (the fetch is artificial, but the test is fundamentally a 100% hit scenario):
BenchmarkDotNet=v0.13.0, OS=Windows 10.0.26100 AMD Ryzen 9 7900X, 1 CPU, 24 logical and 12 physical cores .NET SDK=9.0.100-preview.4.24215.2 [Host] : .NET 9.0.0 (9.0.24.20603), X64 RyuJIT DefaultJob : .NET 9.0.0 (9.0.24.21104), X64 RyuJIT
| Method | Mean | Error | StdDev | Ratio | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|
| HitDistributedCache | 553,492.01 ns | 5,439.496 ns | 4,821.972 ns | 1.000 | - | - | - | 1,464 B |
| HitHybridCache | 608.02 ns | 12.098 ns | 29.218 ns | 0.001 | 0.0105 | - | - | 176 B |
| HitHybridCacheImmutable | 33.02 ns | 0.664 ns | 1.145 ns | 0.000 | - | - | - | - |
(yes, callers could add L1 support, but many don't because it is awkward; stampede support is very tricky to get right, so is almost never implemented; the point of HybridCache is to make it easy to get all the right behaviors)