Scrapeless Wiki

Parquet vs. CSV – Definition and Differences

Comparison P2 parquet vs csv

Compare columnar Parquet with delimited CSV on size, speed, type fidelity, and tooling β€” and see when each is genuinely the right choice.

CSV is text laid out row by row; Parquet is binary laid out column by column with embedded types and statistics. Nearly every practical difference follows from those two facts.

1. Key Differences

Dimension CSV Parquet
Layout Row-oriented text Column-oriented binary
Types None β€” everything is text Stored in the file
Nulls Ambiguous Explicit
Size Large Typically several times smaller
Column selection Reads the whole row Reads only needed columns
Filter pushdown None Row-group statistics enable skipping
Human-readable Yes No
Spreadsheet support Native None
Nested data Not supported Supported

2. Why Parquet Is Smaller

Values in a column share a type and often repeat. Dictionary encoding replaces repeated values with references, run-length encoding collapses repetition, and delta encoding compresses sequences β€” then general compression applies on top.

CSV cannot do any of this, because adjacent bytes belong to different columns.

3. Why Parquet Is Faster to Query

Two mechanisms compound. Reading three columns from a fifty-column dataset touches only those three, and per-row-group statistics let a filtered query skip groups whose value range cannot match.

Combined with partitioned directories, a well-targeted query can read a small fraction of the dataset β€” where CSV must read every byte to find anything.

4. Type Fidelity Is the Quiet Win

CSV loses types. Identifiers with leading zeros become numbers, long numeric codes become scientific notation, dates get reinterpreted by whatever opened the file, and empty versus null becomes indistinguishable.

Parquet stores types in the file, so a round trip returns what went in. For collected data with identifiers and mixed types, this eliminates an entire class of silent corruption.

5. Where CSV Still Wins

  • A person needs to open it. Spreadsheets are the universal interface.
  • Small, flat, one-off exchanges.
  • Command-line text tooling on modest files.
  • Systems that accept nothing else, which remains common.

Those are real needs, and they are why CSV has not gone away.

6. The Practical Pattern

Keep Parquet as the stored format and generate CSV as an export when a person asks for one. The reverse β€” storing CSV and converting for analysis β€” pays the conversion cost repeatedly and risks type loss at every step.

7. Migration Notes

When converting existing CSV, declare the schema explicitly rather than letting types be inferred. Inference is where identifiers become floats, and doing it once wrongly bakes the error into the Parquet file too.

8. Summary

Parquet is smaller, faster to filter, and type-preserving; CSV is readable, universally supported, and opens in a spreadsheet.

Store Parquet, export CSV. The only common mistake is treating CSV as a storage format, where its lack of types quietly corrupts identifiers and its size makes every query expensive.