Definition
Apache Parquet stores data by column rather than row, so queries that need a few fields read less data. It compresses well and is a default for data lakes and ML feature dumps.
It pairs commonly with Arrow, Spark, pandas, and DuckDB.
In simple terms
A row file is a binder of full student records. Parquet is separate drawers for grades, names, and ages — grab only the drawer you need.
Where you see it
- Exporting training sets from warehouses to object storage.
- Sharing large NLP metadata tables efficiently.
How it works
1.Write columnar chunks
Tools serialize tables to .parquet files.
2.Read needed columns
Engines push down projections and filters.
3.Partition files
Organize by date/key for scalable lakes.
Why it matters
- Parquet is the practical lingua franca for large structured ML and analytics data on disk.
Often confused
Parquet replaces databases.
It is a file format — great for interchange and lakes, not a full transactional DB.