Paper page - Drowning in Documents: Consequences of Scaling Reranker Inference (original) (raw)
Published on Nov 18, 2024
Abstract
Rerankers, typically cross-encoders, offer diminishing returns and degrade quality when scoring a large number of documents, challenging the assumption that they are consistently more effective.
Rerankers, typically cross-encoders, are often used to re-score the documents retrieved by cheaper initial IR systems. This is because, though expensive,rerankers are assumed to be more effective. We challenge this assumption by measuring reranker performance for full retrieval, not just re-scoringfirst-stage retrieval. Our experiments reveal a surprising trend: the best existing rerankers provide diminishing returns when scoring progressively more documents and actually degrade quality beyond a certain limit. In fact, in this setting, rerankers can frequently assign high scores to documents with no lexical or semantic overlap with the query. We hope that our findings will spur future research to improve reranking.
View arXiv page View PDF Add to collection
Get this paper in your agent:
hf papers read 2411.11767
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2411.11767 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2411.11767 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2411.11767 in a Space README.md to link it from this page.