What Is a Data Quality Check? – Definition and How It Works

Informational P2 what is a data quality check

Learn the dimensions of data quality, which automated checks catch which failures, how to set thresholds, and how to act on results.

A data quality check is an automated assertion about a dataset, run as part of the pipeline. Its purpose is to make a problem visible on the run that caused it, rather than in a report weeks later.

1. The Dimensions

Dimension	Question	Typical check
Completeness	Is everything here?	Record count, required-field fill rate
Validity	Are values well-formed?	Types, ranges, formats, allowed values
Consistency	Do fields agree?	Cross-field rules, referential integrity
Uniqueness	Any duplicates?	Distinct count against total
Timeliness	Is it current?	Age of newest record, freshness of source
Accuracy	Is it correct?	Comparison to a trusted reference

Accuracy is the hardest and the only one that cannot be checked internally — it needs an external truth.

2. Checks That Catch Collection Failures

Structural checks catch parsing errors. These catch the quieter class:

Record count against expectation, per source.
Run-over-run comparison — a 90% drop is a failure, not a market event.
Fill rate for critical fields, not merely presence of the column.
Value distribution — all prices identical, or all ratings zero, indicates extraction failure.
Duplicate rate, which usually reveals a pagination bug.

3. Setting Thresholds

Thresholds should come from history rather than intuition. Collect a few weeks of normal runs, then set bounds relative to what actually occurs, per source.

A single global threshold produces constant false alarms on small sources and misses genuine failures on large ones.

4. Warn Versus Fail

Not every check should stop a pipeline.

Fail on checks that mean the data is unusable — zero records, schema mismatch, required fields empty.
Warn on checks that indicate drift worth investigating — a modest count change, a rising null rate.

Failing everything trains people to override; warning on everything trains them to ignore. The split is what keeps signals meaningful.

5. Where to Run Them

After extraction, before storage, to prevent bad data landing.
After transformation, to catch logic errors.
On the published dataset, to catch anything the earlier stages missed.

Checks at the boundary are the most valuable, because they prevent contamination rather than reporting it.

6. Acting on Failures

A failed check needs three things to be actionable: what was expected, what was observed, and an example record. Without the example, investigation starts from scratch.

Quarantine failing batches with their raw payloads. The most common bad response to a failing check is relaxing the threshold, and having the evidence to hand makes that harder to do casually.

7. Track Quality Over Time

Store check results as a time series, not just pass or fail. A fill rate drifting from 99% to 91% over a month is a real degradation that never trips a threshold on any single run.

8. Summary

Data quality checks assert completeness, validity, consistency, uniqueness, timeliness, and — where a reference exists — accuracy, run at pipeline boundaries.

Set thresholds from observed history per source, distinguish failing conditions from warnings, keep raw evidence for every failure, and track results over time so slow degradation becomes visible.