What Is Haystack? – Definition and How It Works

Learn what the Haystack framework provides, how its component pipelines work, and where it suits retrieval systems.

Haystack is an open-source Python framework for building search and retrieval-augmented generation systems, from deepset. Its organising idea is an explicit pipeline of connected components.

1. The Pipeline Model

Rather than a chain of calls, Haystack asks you to declare components and the connections between them:

from haystack import Pipeline
from haystack.components.embedders import SentenceTransformersTextEmbedder
from haystack.components.retrievers import InMemoryEmbeddingRetriever
from haystack.components.builders import PromptBuilder
from haystack.components.generators import OpenAIGenerator

p = Pipeline()
p.add_component("embedder", SentenceTransformersTextEmbedder())
p.add_component("retriever", InMemoryEmbeddingRetriever(document_store=store))
p.add_component("prompt", PromptBuilder(template=TEMPLATE))
p.add_component("llm", OpenAIGenerator())

p.connect("embedder.embedding", "retriever.query_embedding")
p.connect("retriever.documents", "prompt.documents")
p.connect("prompt.prompt", "llm.prompt")

p.run({"embedder": {"text": "how do I stop getting blocked"}})

The explicit wiring is verbose and buys something real: the graph is inspectable, serialisable to YAML, and validated at connection time rather than failing at runtime on a shape mismatch.

2. Core Concepts

3. Indexing Is a Pipeline Too

The same model covers ingestion, which is where collected pages enter:

idx = Pipeline()
idx.add_component("converter", HTMLToDocument())
idx.add_component("cleaner", DocumentCleaner())
idx.add_component("splitter", DocumentSplitter(split_by="word", split_length=200, split_overlap=30))
idx.add_component("embedder", SentenceTransformersDocumentEmbedder())
idx.add_component("writer", DocumentWriter(document_store=store))
idx.connect("converter", "cleaner"); idx.connect("cleaner", "splitter")
idx.connect("splitter", "embedder"); idx.connect("embedder", "writer")

That converter-cleaner-splitter sequence is exactly the boilerplate-removal-then-chunking discipline that determines retrieval quality, expressed as reusable components rather than ad-hoc code.

4. Retrieval Options

Haystack supports the retrieval strategies that matter:

• Embedding retrieval — dense vector similarity.
• BM25 retrieval — lexical, for exact identifiers and rare terms.
• Hybrid — both, with a joiner and optional reranker.
• Metadata filtering alongside either.

Having BM25 as a first-class retriever rather than an afterthought is worth noting, because it is what handles the queries dense retrieval gets wrong.

5. Where It Fits

Good fit: a retrieval system with a defined shape, where you want the pipeline explicit and serialisable; teams that value a declared graph over implicit chaining; deployment where the same YAML describes what runs.

Poor fit: highly dynamic agent loops where the graph is discovered at runtime; very simple single-call use, where the ceremony exceeds the benefit.

6. Compared With the Alternatives

For a fixed retrieval pipeline, the explicit graph is an advantage. For a dynamic agent, a framework built around state machines fits better.

7. Working With Collected Data

1. Extract main content before converting. Feeding raw HTML in means indexing navigation.
2. Carry provenance into Document.meta — source URL, fetch date, heading path.
3. Use metadata filters to scope retrieval by source, language, or recency.
4. Record the embedding model version in metadata, so a model change is traceable.
5. Add BM25 alongside dense retrieval, for identifiers and error codes.
6. Keep the indexing pipeline separate from the query pipeline, since they run on different schedules.

8. Summary

Haystack builds search and RAG systems from explicitly connected typed components, with the pipeline graph inspectable, validated at connection time, and serialisable to YAML.

Its indexing pipelines express the content-extraction-then-chunking sequence that determines retrieval quality, and BM25 is a first-class retriever alongside dense search — so carry provenance and the embedding model version into document metadata, and keep indexing and query pipelines separate.

Potential Related Glossary

• Unstructured Data
• Parsing
• Data Structure
• Data Fusion