What Is an Embedding? – Definition and How It Works

Learn what an embedding is, how text becomes a vector, why cosine similarity measures meaning, and what embeddings can and cannot represent.

An embedding is a list of numbers that positions a piece of content in a space where distance corresponds to similarity of meaning. It is the mechanism that lets software compare two sentences without sharing a single word.

1. What Is an Embedding?

An embedding is a fixed-length vector produced by a model from an input — a sentence, a paragraph, an image, or a product record.

• Key idea: meaning becomes geometry. Related content lands near related content.
• Mechanism: a trained encoder maps input to coordinates in a space of several hundred to several thousand dimensions.
• Goal: make semantic comparison a numeric operation that scales to millions of items.

Example of the Shape

embed("residential proxy")   # → [0.021, -0.114, 0.087, ...]  (e.g. 1536 numbers)
embed("home IP address")     # → [0.019, -0.102, 0.091, ...]  nearby
embed("banana bread recipe") # → [-0.233, 0.418, -0.052, ...] far away

The individual numbers are not interpretable. Only the relationships between vectors carry information.

2. How Similarity Is Measured

Cosine similarity is the usual metric. It compares the angle between two vectors and ignores their length.

• 1.0 — the vectors point the same way; the texts mean approximately the same thing.
• around 0 — unrelated.
• negative — pointing in opposing directions, which for text embeddings is uncommon.

Angle is preferred over straight-line distance because vector length tends to track text length rather than topic, and a long paragraph should still match a short question about it.

3. What Embeddings Capture

Embeddings encode usage patterns learned from training data, which produces some useful properties and some limits.

The negation weakness is the one that surprises people most often, and it is why retrieval systems usually combine vector search with keyword matching.

4. Why the Same Model Must Be Used

Vectors from different models are not comparable, even when their dimensions match. Each model organises its space differently, so a question embedded with model A cannot be searched against chunks embedded with model B.

This has a practical consequence: changing the embedding model means re-embedding the entire corpus, not just new documents. Treat the model as part of the index's schema.

5. Where Embeddings Are Used

• Semantic search — rank documents by meaning rather than shared keywords.
• Retrieval-augmented generation — select passages to place in a model's context window.
• Clustering and deduplication — group near-identical records that differ in wording.
• Classification — embed once, then train a small, cheap classifier on the vectors.
• Recommendation — treat proximity in the space as relatedness between items.

6. Real-World Examples

• A help centre search where "my card was declined" finds an article titled "Failed payments", despite no shared vocabulary.
• Product catalogue deduplication where two listings for the same item, written by different sellers, cluster together.
• Support ticket routing where incoming text is embedded and matched to the queue whose historical tickets sit closest.
• Scraped-page filtering where a crawl keeps only pages whose embedding sits near a topic definition, discarding the rest before storage.

7. Common Misconceptions

• "A higher-dimensional model is always better." Larger vectors cost more to store and search, and the gain flattens quickly. Measure on your own queries.
• "Similar scores mean similar quality." Cosine values are not comparable across models, and a 0.82 in one space is not a 0.82 in another. Only ranking within one model is meaningful.
• "Embeddings understand the text." They encode statistical usage, which is why negation and exact numbers survive so poorly.
• "One embedding per document is fine." A long document averages into a vague point in the space; passages retrieve far better than whole files.

8. Summary

An embedding turns content into coordinates so that similarity of meaning becomes a distance calculation. It is the foundation of semantic search and of the retrieval half of RAG.

Its weaknesses are as important as its strengths: negation, exact figures, and unseen proper nouns are all handled badly, so production systems pair vector similarity with keyword search rather than trusting either alone.

Potential Related Glossary

• Data Structure
• Unstructured Data
• Data Fusion
• Parsing