What Is Property-Based Testing? – Definition and How It Works

Learn how property-based testing generates inputs to falsify invariants, and which properties suit parsers and pipelines.

Property-based testing states a property that should hold for all inputs, then generates many inputs trying to falsify it. Where example-based tests check cases you thought of, this searches for the ones you did not.

1. The Shift in What You Write

Example-based: given this input, expect that output.

Property-based: for any input of this shape, this relationship holds.

from hypothesis import given, strategies as st

@given(st.text())
def test_normalise_is_idempotent(s):
    once = normalise(s)
    assert normalise(once) == once

You never name an input. The framework generates hundreds, including the ones nobody would think to write: empty strings, lone combining characters, zero-width spaces, strings of only whitespace.

2. Shrinking Is What Makes It Practical

When a property fails, the framework shrinks the failing input to the smallest example that still fails.

That is the feature that turns a random failure into a usable bug report. Instead of a 400-character string of mixed scripts, you get "́" — a lone combining accent — and the cause is immediately obvious.

3. Properties Worth Testing in Collection Code

4. The Highest-Value Property for Parsers

For anything parsing untrusted input, this one:

@given(st.binary())
def test_parser_never_crashes(data):
    try:
        parse_page(data)
    except ParseError:
        pass                      # a defined failure is fine

A parser fed arbitrary bytes must not raise an undefined exception. Real collection meets truncated responses, wrong encodings, binary served as HTML, and challenge pages — and a crash on any of them takes down a pipeline.

Property-based testing finds those inputs far faster than production does.

5. Equivalence Testing Is Underused

Where you replace a parser or optimise a normaliser, assert the new implementation agrees with the old:

@given(st.text())
def test_new_matches_old(s):
    assert normalise_fast(s) == normalise_reference(s)

That converts "I think this refactor is safe" into evidence, and it is one of the cheapest ways to gain confidence in a rewrite.

6. Custom Strategies for Realistic Inputs

Generic text finds encoding bugs. Domain-shaped input finds domain bugs:

prices = st.one_of(
    st.just(""), st.just("—"), st.just("Free"),
    st.from_regex(r"\$\d{1,4}\.\d{2}", fullmatch=True),
    st.from_regex(r"\d{1,3}(,\d{3})*,\d{2} €", fullmatch=True),
    st.from_regex(r"(?:USD|EUR|GBP) ?\d+(\.\d+)?", fullmatch=True),
)

@given(prices)
def test_price_parse_never_lies(s):
    out = parse_price(s)
    assert out is None or (isinstance(out, int) and out >= 0)

out is None or valid is the useful shape: the function may decline to parse, but it must never return a wrong-typed or negative value.

7. Limitations

• It tests properties, not correctness. A property can hold while the output is wrong.
• Properties are hard to find for some code; not everything has a clean invariant.
• Slower than example tests, so it belongs in CI rather than in a tight edit loop.
• Flaky-looking if the code is nondeterministic — fix that first.
• It does not replace fixtures. Real captured pages still test real structure.

8. Summary

Property-based testing states invariants and generates inputs to falsify them, shrinking any failure to a minimal example that makes the cause obvious.

For collection code the highest-value property is that a parser fed arbitrary bytes never raises an undefined error, since truncated, misencoded, and binary responses are routine — and equivalence properties turn a risky refactor into a verified one, alongside rather than instead of fixture tests.

Potential Related Glossary

• Parsing
• Unstructured Data
• Data Wrangling
• Data Structure