What Is a Retrieval Pipeline? – Definition and How It Works
Learn what a retrieval pipeline contains end to end, which stage owns which failure, and how to instrument it so problems are attributable.
A retrieval pipeline is the path from source documents to the passages that reach a model. It has an offline half that prepares the corpus and an online half that answers questions, and almost every quality problem is attributable to a specific stage.
1. The Offline Half
| Stage | Produces | Owns the failure when |
|---|---|---|
| Collect | Raw documents with fetch metadata | Sources are missing or stale |
| Extract | Clean text, structure preserved | Boilerplate is indexed as content |
| Chunk | Passages with headings carried in | The answer straddles a boundary |
| Embed | Vectors, one per passage | Query and corpus models mismatch |
| Store | Vectors, text, metadata | Filters cannot express what queries need |
2. The Online Half
| Stage | Produces | Owns the failure when |
|---|---|---|
| Query processing | A cleaned, possibly expanded query | Intent is misread |
| Retrieve | Candidates from vector and lexical search | The right passage never surfaces |
| Fuse | One merged ranking | One retriever dominates unhelpfully |
| Rerank | A precise ordering | Marginal passages fill the context |
| Select | Passages that fit the budget | Too many or too few are kept |
| Generate | The grounded answer | The model ignores or misreads the passages |
3. Instrument Each Stage
Without per-stage visibility, every failure looks like "the model gave a bad answer". Log:
- Candidates returned by each retriever, with scores.
- The fused and reranked orderings.
- Which passages entered the prompt, with their identifiers.
- Whether the model refused, and whether it should have.
Most disappointing answers turn out to be retrieval failures, and this is how that becomes visible rather than assumed.
4. Evaluate Retrieval Separately From Generation
Build a small set of questions with known supporting passages, then measure whether retrieval surfaces the right passage at all, and at what rank.
That metric is independent of the model and improves the pipeline directly. Optimising prompts against a broken retriever is the most common wasted effort in this work.
5. Where Effort Pays Off
- Extraction quality — removing boilerplate raises everything downstream.
- Chunk boundaries — structural splitting beats fixed size on real documents.
- Reranking — the cheapest large improvement once retrieval works.
- Hybrid retrieval — covers exact tokens that vectors miss.
- Refresh cadence — a stale index cannot be fixed by better prompting.
6. Common Anti-Patterns
- Indexing rendered HTML without extraction.
- One global recrawl interval regardless of how fast sources change.
- Retrieving many passages and hoping the model sorts them out.
- Changing the embedding model without re-indexing.
- No refusal instruction, so gaps are filled from model memory.
7. Keeping It Maintainable
Store raw documents separately from extracted text, so extraction can be improved and the corpus rebuilt without refetching. Version the pipeline configuration — model, chunking parameters, prompt — alongside the index, because comparing results across changes is impossible otherwise.
8. Summary
A retrieval pipeline spans collection, extraction, chunking, embedding, storage, retrieval, fusion, reranking, selection, and generation, and each stage owns a distinct class of failure.
Instrument every stage, evaluate retrieval independently of the model, and invest in extraction and chunking first — the generation step is rarely where the problem actually is.