What Is Record Linkage? – Definition and How It Works
Learn how record linkage matches entities across sources without shared identifiers, using blocking, comparison, and thresholds.
Record linkage decides which records from different sources refer to the same real-world entity. It is the problem that appears the moment data comes from more than one place and no shared key exists.
1. Why It Is Hard
Two records for the same business:
A: "Radiant Plumbing, Air Conditioning, & Electrical" | 512-690-4935 | Austin TX
B: "Radiant Plumbing A/C & Electric" | (512) 690-4935 | Austin, Texas
Obvious to a person, and not equal by any string comparison. The names differ in abbreviation, punctuation, and ampersand style; the phone differs in formatting; the state differs in form. Every one of those is a normal, correct representation.
2. The Pipeline
| Stage | Purpose |
|---|---|
| Normalisation | Reduce formatting variance |
| Blocking | Avoid comparing every pair |
| Comparison | Score field-level similarity |
| Classification | Decide match, non-match, or review |
| Clustering | Resolve groups consistently |
3. Blocking Is the Practical Constraint
Comparing all pairs is quadratic: 100,000 records against 100,000 is 10 billion comparisons. Blocking restricts comparison to records sharing a cheap key.
Common blocking keys are a postcode, the first few characters of a normalised name, a phonetic encoding, or the last seven digits of a phone number. Several blocking passes with different keys, unioned, recovers most pairs a single key would miss — which is the standard fix for blocking's inherent weakness: a true match whose block key differs is never even compared.
4. Comparison Uses Field-Appropriate Measures
- Names — Jaro-Winkler or token-set ratio, which tolerate word order and abbreviation.
- Addresses — parse into components first, then compare per component.
- Phones — normalise to digits, compare the national portion.
- Dates — compare as dates, permitting small offsets.
- Coordinates — distance, with a threshold.
- Identifiers — exact, after canonicalisation.
Comparing a parsed address field by field beats comparing whole address strings, because a single missing unit number otherwise sinks an obvious match.
5. Turning Scores Into Decisions
Weight fields by how much they discriminate. A matching phone number is powerful evidence; a matching city is almost none, since most records in a block share it.
Then use two thresholds rather than one:
- Above the upper threshold → automatic match.
- Below the lower → automatic non-match.
- Between → human review.
A single threshold forces every uncertain pair into a wrong answer. A review band makes the uncertainty visible, and lets the cost of an error decide how wide it should be.
6. Clustering Must Be Transitive
Pairwise decisions can contradict: A matches B, B matches C, A does not match C.
Resolving them by connected components merges everything transitively, which chains unrelated entities together through one bad link. Stricter clustering that requires higher internal similarity avoids the chaining at the cost of leaving some duplicates.
Neither is free, and the choice depends on whether a false merge or a false split is worse for the use case.
7. Evaluation Requires Labels
Precision and recall need ground truth: a hand-labelled sample of pairs, including hard negatives — different entities that look similar — because a set of easy negatives makes any method look excellent.
Record the threshold and the field weights alongside the results, since a linkage result is meaningless without the parameters that produced it.
8. Summary
Record linkage matches records to real-world entities across sources by normalising, blocking to keep comparisons tractable, scoring fields with appropriate similarity measures, and classifying with thresholds.
Use several blocking passes so a single key does not hide true matches, weight fields by discriminating power, keep a human-review band between the two thresholds, and choose clustering knowing that connected components chains entities together through one bad link.