Propose vector-set API by mgravell · Pull Request #2939 · StackExchange/StackExchange.Redis (original) (raw)

Brilliant, glad to see this landing as we look into vector sets in Garnet as well.

In this vein, as I'm working on Garnet's Vector Set api support, I want to have a discussion about likely extensions.

It seems quite possible that Redis will add new quantization options (to join NOQUANT, B8, and BIN). Garnet is definitely going to have a few (I've sketched out a XPREQ8, though that name is a place holder, for "NOQUANT but unsigned bytes"), and I expect more to come.

It also seems reasonable for there to be new vector data formats for the commands (to join FP32 and VALUES), given how common reduced precision storage is in ML workloads (FP16 and FP8 for example). Garnet is going to have at least one (which I'm calling XB8 right now, for "each dimension is one byte, [0, 255]").

Obviously all these extensions could be handled with the .ExecuteXXX methods just fine. But I wonder, given the likelihood of extensions from both Redis proper and other RESP implementers, if a something for these specific options might be merited.

Just spit-balling, but a struct (which wraps some Span<byte>/Memory<byte>) instead of an enum for VectorSetQuantization and a new type for values rather than a simple ReadOnlyMemory<float> with an accompanying Span<byte>/Memory<byte> for the flag. With some conversions, this could also accommodate users who have their vectors elements as strings w/o forcing them to parse them client side before sending.

/cc @NickCraver Since we discuss this a little bit this morning.