What Is Data Extraction? β Definition and How It Works
Learn what data extraction is, how it differs from content extraction, which sources to prefer, and how to validate extracted fields.
Data extraction turns a source into typed fields β a price, a rating, a date, an identifier. It differs from content extraction, which isolates readable prose; here the output is a record, not a passage.
1. The Output Is a Record
{
"url": "https://example.com/product/4192",
"title": "Widget",
"price": 49.90,
"currency": "USD",
"in_stock": true,
"rating": 4.6,
"collected_at": "2026-08-18T09:00:00Z"
}
Typed fields, explicit nulls, and provenance. Anything less is a string blob that a consumer must re-parse.
2. Prefer Structured Sources
| Source | Stability | Effort |
|---|---|---|
| An official API | Highest | Lowest |
| A JSON endpoint the page calls | High | Low |
| Embedded state in a script tag | High | Low |
| JSON-LD or microdata | Moderate | Low |
| Rendered markup and selectors | Lowest | Highest |
The order matters. Layout changes constantly; data contracts change far less. Checking the four options above before writing selectors is the single largest maintenance saving available.
3. When Selectors Are Unavoidable
Anchor on stable attributes β id, data-*, name β rather than generated class names. Keep selectors short and relative to a meaningful container, and use text anchors where the label is the only stable landmark.
Extract to explicit None when an element is missing, so absence is recorded rather than crashing or silently becoming an empty string.
4. Normalise at Extraction
Trim whitespace, strip currency symbols, coerce types, convert units, and standardise dates as the field is read β not later.
The reason is that every downstream operation compares values, and a trailing space or a currency symbol defeats equality. Retrofitting normalisation means reprocessing everything already stored.
5. Validate Immediately
Declare a schema and check against it at extraction, where a failure can be attributed to a specific page:
- Types and required fields.
- Plausible ranges β a zero price or a rating of 47 is a parsing failure.
- Enumerated values for currency and status.
- Cross-field consistency.
Add set-level checks too: minimum record count, fill rates, and duplicate rate. Per-record validation cannot see that a page returned three products instead of sixty.
6. Record Provenance
Store the source URL, the fetch timestamp, the egress region where content varies by location, and β where possible β the raw payload.
Raw retention is what makes extraction improvable. For web sources the original cannot be re-read later, so discarding it forecloses every future correction.
7. Extraction With Language Models
A schema plus a model can extract fields from prose without per-site rules, which changes the economics for long-tail sources.
It costs more per page and requires the same validation as any other method β arguably more, since a model will produce a well-formed wrong value confidently. Structured output plus range checks is what makes it usable.
8. Summary
Data extraction produces typed records with provenance, and its quality is decided by which source is used: an API or embedded JSON beats selectors over rendered markup every time.
Normalise and validate at extraction, add set-level checks that catch collection failures, and keep the raw payload β because that is the only thing that makes a wrong extraction fixable later.