Delta Lake

Delta Lake is an open-source storage layer that brings reliability to data lakes. It provides ACID transactions, scalable metadata handling, and unifies streaming and batch data processing.

Delta Lake runs on top of your existing data lake and is fully compatible with Apache Spark APIs. It is the core storage technology behind the Databricks platform but is also available as an open-source project. By providing features like time travel (data versioning) and schema enforcement, Delta Lake ensures that data remains high-quality and consistent for downstream analytics and machine learning.