Paper page - Exploring ell_0 Sparsification for Inference-free Sparse Retrievers (original) (raw)

Published on Apr 21, 2025

Abstract

Inference-free retrieval achieves state-of-the-art performance using an ell_0\ell_0ell_0 inspired sparsification method, offering a balance between effectiveness and efficiency.

With increasing demands for efficiency, information retrieval has developed a branch of sparse retrieval, further advancing towards inference-free retrievalwhere the documents are encoded during indexing time and there is no model-inference for queries. Existing sparse retrieval models rely on FLOPS regularization for sparsification, while this mechanism was originally designed for Siamese encoders, it is considered to be suboptimal in inference-free scenarios which is asymmetric. Previous attempts to adapt FLOPS for inference-free scenarios have been limited to rule-based methods, leaving the potential of sparsification approaches for inference-free retrieval models largely unexplored. In this paper, we explore ell_0 inspired sparsification manner for inference-free retrievers. Through comprehensive out-of-domain evaluation on the BEIR benchmark, our method achieves state-of-the-art performance among inference-free sparse retrieval models and is comparable to leading Siamese sparse retrieval models. Furthermore, we provide insights into the trade-off between retrieval effectiveness and computational efficiency, demonstrating practical value for real-world applications.

View arXiv page View PDF Project page GitHub 24 Add to collection

Get this paper in your agent:

hf papers read 2504.14839

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash