What Is a Schema? – Definition and How It Works
Learn what a schema defines, how schema-on-write differs from schema-on-read, how migrations work safely, and why evolution needs planning.
A schema is the declared structure of data: which fields exist, what types they hold, which are required, and how records relate. It is a contract between whoever writes data and whoever reads it.
1. What It Declares
- Fields and types — names and what each holds.
- Constraints — required, unique, within a range, referencing another record.
- Relationships — keys linking records across tables.
- Defaults — what happens when a value is absent.
2. Schema-on-Write Versus Schema-on-Read
| Aspect | On write | On read |
|---|---|---|
| Enforced | At insertion | When queried |
| Bad data | Rejected immediately | Stored, discovered later |
| Flexibility | Changes require migration | Writers can vary freely |
| Typical of | Relational databases | Data lakes and document stores |
Collection pipelines often use both: land raw payloads schema-on-read, then enforce schema-on-write when loading the cleaned table. That keeps flexibility at the boundary and guarantees downstream.
3. Why Enforcement Matters Downstream
Without enforcement, every consumer must handle every possible shape. One writer changing a field's type silently breaks readers that assumed it, and the failure appears far from its cause.
A schema moves that failure to the write, where the responsible change is obvious.
4. Migrations
Changing a schema on live data requires an ordering that never leaves readers broken:
- Add the new field as optional, defaulted.
- Write both old and new fields.
- Backfill history.
- Move readers to the new field.
- Stop writing the old field.
- Remove it.
Skipping to the end is what causes outages. Each step is individually safe, and the sequence tolerates readers being deployed at different times.
5. Additive Changes Are Safe; Others Are Not
| Change | Safety |
|---|---|
| Add an optional field | Safe |
| Add a required field with a default | Usually safe |
| Rename a field | Breaking — do it as add, migrate, remove |
| Narrow a type | Breaking |
| Widen a type | Usually safe |
| Remove a field | Breaking until every reader stops using it |
6. Schemas for Collected Data
Web sources have no obligation to keep their structure stable, so a collection schema is your declaration of what you expect — and the point at which a source's change becomes visible.
Version it, and record which version produced each stored batch. Without that, comparing data across a schema change is guesswork.
7. Documenting It
A schema that exists only in code is discovered by reading the code. Publishing it — field names, types, meanings, and null semantics — is what lets consumers use the data without asking.
Include what null means per field: unknown, not applicable, and empty are different, and only documentation can distinguish them.
8. Summary
A schema declares the structure and constraints of data, enforced either at write time or interpreted at read time.
Enforce it where data enters a queryable store, treat migrations as a sequence of individually safe steps, keep changes additive where possible, version it alongside the data, and document what null means for every field.