What Is Reranking? – Definition and How It Works
Learn what reranking is, how cross-encoders differ from bi-encoders, why it is the cheapest retrieval improvement, and how to configure it.
Reranking reorders a shortlist of retrieved candidates using a more accurate, more expensive model than the one that produced the shortlist. It is usually the largest quality gain available for the least architectural change.
1. The Two-Stage Shape
- Retrieve broadly. A fast method returns perhaps fifty candidates — imprecise but high recall.
- Rerank precisely. A stronger model scores each candidate against the query and reorders.
- Keep the top few that fit the context budget.
The division exists because the accurate model is far too slow to run over a whole corpus, and the fast method is not precise enough to pick the final three.
2. Bi-Encoder Versus Cross-Encoder
| Aspect | Bi-encoder (retrieval) | Cross-encoder (reranking) |
|---|---|---|
| Encoding | Query and document separately | Query and document together |
| Precomputation | Documents embedded in advance | Nothing can be precomputed |
| Cost per pair | Effectively free at query time | One model pass per candidate |
| Accuracy | Good | Substantially better |
| Scale | Millions of documents | Tens of candidates |
Reading query and passage together is what allows the cross-encoder to notice that a passage mentions the query's terms in the wrong relationship — something separate encodings cannot represent.
3. Why It Helps So Much
First-stage retrieval optimises for not missing the right passage. That tolerance admits marginal candidates, and with only a few slots in the context window, marginal candidates displace good ones.
Reranking fixes the ordering exactly where it matters: the final selection.
4. Configuring It
- Candidate count. Enough for recall, few enough for latency — a few dozen is typical.
- Final count. How many passages reach the prompt, bounded by the context budget.
- Latency budget. Each candidate costs a model pass; batching helps considerably.
- Model choice. Smaller rerankers are often adequate, and much faster.
5. Where It Fits in Hybrid Retrieval
With lexical and vector retrieval fused by rank, reranking runs on the merged list. This is the standard modern shape: retrieve twice, fuse, rerank, select.
It also reduces sensitivity to fusion weights, since a good reranker can recover from an imperfect merge.
6. Alternatives and Complements
- Maximal marginal relevance — diversifies results rather than scoring relevance, useful when candidates are near-duplicates.
- Model-based filtering — asking a model to discard irrelevant passages instead of ordering them.
- Metadata boosting — favouring recency or authority, which is orthogonal to semantic relevance.
These compose: diversify, rerank, then boost.
7. When to Skip It
Very small corpora where retrieval already returns everything relevant, latency-critical paths with no room for a second stage, and cases where a large context window makes selection less consequential.
Even then, the win is usually about answer quality rather than fit — fewer, better passages produce better answers than more, weaker ones.
8. Summary
Reranking reorders retrieved candidates with a cross-encoder that reads query and passage together, correcting the ordering at the point where only a few passages will be used.
It is the cheapest large improvement in most retrieval systems: retrieve broadly, fuse if hybrid, rerank precisely, and keep only what the context budget genuinely deserves.