What Is a Data Lake? – Definition and How It Works

Learn what a data lake is, how schema-on-read works, why zones and partitioning matter, and how a lake becomes a swamp without governance.

A data lake stores raw data in its original form at low cost, deferring structure until something reads it. It suits collected data well, because web sources cannot be re-read later.

1. The Defining Property

Schema-on-read. Data lands as it arrived — JSON, HTML, CSV, images, logs — and structure is imposed when it is queried rather than when it is written.

The benefit is that nothing is discarded because it did not fit a table definition decided in advance.

2. Zones

Mature lakes separate data by processing stage:

The rule that keeps this working: raw is immutable, and every downstream zone is reproducible from it.

3. Why Raw Retention Matters for Web Data

A page fetched today is not retrievable as it was tomorrow. If extraction proves wrong and the original was discarded, that history is gone.

Landing raw payloads is therefore not conservatism — it is the only way to improve extraction retroactively for sources that cannot be re-read.

4. Partitioning

Storing files partitioned by date and source lets queries skip everything irrelevant.

raw/source=example/date=2026-08-18/part-0001.json.gz

Query engines read partition values from the path, so a filter on date touches only matching directories. Without partitioning, every query scans everything — the most common reason a lake is slow.

5. File Format and Size

Store cleaned and curated zones as a columnar format such as Parquet: smaller, typed, and prunable. Keep raw in its original form, compressed.

Avoid very small files. Thousands of tiny objects are far slower to query than a few large ones, and consolidation jobs exist specifically to fix that.

6. How a Lake Becomes a Swamp

• No catalogue, so nobody knows what exists.
• No ownership, so nothing is maintained.
• No partitioning, so nothing is fast.
• No documented schemas, so consumers guess.
• No retention policy, so cost grows without bound.

Every one of these is a governance failure rather than a technical one, which is why the swamp outcome is so common.

7. Table Formats

Modern table formats add transactions, schema evolution, and time travel over lake storage, giving warehouse-like guarantees without moving the data.

They are what makes "lakehouse" a meaningful term rather than a marketing one.

8. Summary

A data lake stores raw data cheaply with structure applied on read, organised into raw, cleaned, and curated zones with raw kept immutable.

For collected web data it is the right default, because sources cannot be re-read. Partitioning, sensible file sizes, a catalogue, and documented ownership are what keep it usable rather than merely large.

Potential Related Glossary

• Data Structure
• Unstructured Data
• Data Wrangling
• Web Crawling