What Is Apache Avro? – Definition and How It Works

Learn how Avro embeds its schema, why that enables schema evolution, and where it fits against Parquet and JSON.

Avro is a row-oriented binary serialisation format with the schema carried alongside the data. Its defining property is disciplined schema evolution.

1. Schema-First, Schema-Embedded

An Avro schema is JSON:

{
  "type": "record",
  "name": "Listing",
  "fields": [
    {"name": "source_url", "type": "string"},
    {"name": "title",      "type": ["null", "string"], "default": null},
    {"name": "price_cents","type": ["null", "int"],    "default": null},
    {"name": "currency",   "type": {"type": "enum", "name": "Ccy", "symbols": ["USD","EUR","GBP"]}},
    {"name": "fetched_at", "type": {"type": "long", "logicalType": "timestamp-millis"}}
  ]
}

In an Avro file, that schema is written in the header. So the file is self-describing — a reader needs nothing external to interpret it, which is a real advantage over headerless formats.

2. Schema Evolution Is the Point

Avro defines precise compatibility rules for changing a schema:

The default value is what makes it work: an old reader encountering a new field it does not know skips it, and a new reader encountering old data without the field uses the default.

That is directly useful for collected data, where sources add fields without warning. A schema with defaults absorbs the change without breaking existing readers.

3. Row-Oriented, Unlike Parquet

The distinction that decides which to use:

The common architecture uses both: Avro for the ingestion and streaming path where records arrive one at a time and schemas drift, Parquet for the analytical store where queries touch a few columns across many rows.

4. Schema Registry

In streaming systems, embedding the full schema in every message would be wasteful. A schema registry stores schemas centrally and messages carry a small schema id.

That adds a genuine benefit beyond size: the registry can enforce compatibility, rejecting a schema change that would break existing consumers. Compatibility stops being a convention and becomes a gate.

5. Where It Fits Collection Work

Good fit: a streaming ingestion path where records flow one at a time; sources whose shape changes; a pipeline where producers and consumers deploy independently and must not break each other.

Poor fit: analytical queries over columns — use Parquet; small ad-hoc datasets where JSON's readability wins; anything a human needs to read directly.

6. Logical Types Matter for Correctness

Avro's primitives are minimal — no native date or decimal. Logical types annotate a primitive with meaning:

{"type": "long",  "logicalType": "timestamp-millis"}
{"type": "bytes", "logicalType": "decimal", "precision": 12, "scale": 2}

Using the decimal logical type for money rather than a float is the correctness point. Prices stored as floating point accumulate representation errors, which is exactly the class of silent data fault that surfaces months later in a total that does not reconcile.

7. Practical Notes

1. Give every optional field a default, or you cannot evolve the schema compatibly.
2. Use unions with null for optionality — ["null","string"] with a null default.
3. Use logical types for timestamps and money.
4. Use aliases rather than renaming fields.
5. Register schemas where a registry exists, so compatibility is enforced.
6. Version the schema alongside the data it wrote.

8. Summary

Avro is a row-oriented binary format carrying its schema in the file header, with formal compatibility rules that make schema evolution safe when optional fields have defaults.

That suits ingestion and streaming paths where source shapes drift and producers and consumers deploy independently, while Parquet suits the analytical store — and using the decimal logical type for money rather than a float avoids a silent correctness fault that surfaces much later.

Potential Related Glossary

• Data Structure
• Data Wrangling
• Unstructured Data
• Parsing