Data engineering¶
All topics · Alphabetical index
Concepts¶
- ETL; ELT; batch processing; stream processing
- Event time; processing time; watermarks; late events
- Tumbling windows; hopping windows; sliding windows; session windows
- Stateful processing; stream joins; checkpoints
- Data warehouse; data lake; lakehouse; OLTP; OLAP
- Columnar storage; Parquet; Avro; ORC; compression
- Star schema; snowflake schema; dimensional modeling
- Slowly changing dimensions
- Data contracts; schema evolution; lineage; catalogs
- Freshness; completeness; reconciliation; backfills; incremental processing
Further reading¶
Reference links for this topic.