What Is Fuzzy Matching? – Definition and How It Works
Learn what fuzzy matching is, the main similarity measures, how blocking makes it scale, and how to set thresholds without merging wrong records.
Fuzzy matching decides whether two records refer to the same thing when no shared identifier exists. It is the standard problem when combining data from sources that never agreed on a key.
1. Why It Is Needed
The same product, company, or person is written differently everywhere: abbreviations, punctuation, word order, transliteration, and simple typing errors.
Exact matching finds none of these, so matching has to be based on similarity with a decision threshold.
2. The Common Measures
| Measure | Compares | Good for |
|---|---|---|
| Levenshtein distance | Edits to transform one string to another | Typing errors and small variants |
| Jaro-Winkler | Character agreement, favouring shared prefixes | Names |
| Token set ratio | Word overlap, ignoring order | Reordered multi-word names |
| Phonetic codes | How a string sounds | Transliteration and spelling variants |
| Embedding similarity | Meaning | Descriptions rather than names |
Choosing badly is the usual first mistake: character distance on reordered company names performs poorly where token overlap performs well.
3. Normalise Before Comparing
Lowercase, strip punctuation, expand or remove common suffixes, and remove filler words before measuring. A large share of apparent fuzziness disappears at this step, and what remains is genuinely ambiguous.
4. Blocking Makes It Scale
Comparing every record against every other is quadratic and impossible beyond modest sizes.
Blocking groups records by a coarse key — first letters, a phonetic code, a postcode, a category — and compares only within groups. The trade is that a true match split across blocks is never found, which is why several blocking keys are often used together.
5. Setting a Threshold
A threshold trades false matches against missed ones, and the right balance depends on consequences.
- Merging wrongly creates a record describing two different things, which is hard to detect afterwards.
- Missing a match leaves duplicates, which is visible and fixable.
For most work the asymmetry favours caution: set the automatic threshold high, and route the middle band to manual review rather than guessing.
6. Use Several Signals
Names alone are weak. Combining name similarity with a second field — location, category, size, a partial identifier — raises accuracy sharply, because two records agreeing on several dimensions are far less likely to coincide by chance.
7. Record the Decision
Store the match score, the method, and the fields that agreed. Later questions about why two records were merged are common, and an unexplained merge cannot be audited.
Keep the originals so a merge can be reversed.
8. Summary
Fuzzy matching links records without a shared key using similarity measures, blocked into comparable groups to make the work tractable.
Normalise first, choose a measure that suits the data, combine several fields, set a cautious threshold with a manual review band, and record every decision with its score so merges remain auditable and reversible.