The Data Lakehouse: One Data Layer for Modern Data and AI

Data lake, data warehouse and data lakehouse

Traditional data warehouses are strong where structured data is used for reporting and analytics. The data lake approach lets large volumes of structured and unstructured data sit in more flexible, more economical storage layers.

But keeping files on object storage is not on its own enough to make an enterprise data platform. The lakehouse approach sets out to combine the advantages of both:

One shared data layer, rather than copying data into separate analytics and AI platforms again and again.

The layers of a lakehouse

A modern lakehouse architecture is roughly made of these layers:

In this architecture storage and compute can be separated. The data sits on object storage while different compute technologies run different workloads against it.

The same data might be used by Trino for SQL queries, by Apache Spark for large-scale data processing and machine learning, and by other AI services for model or agent scenarios.

Parquet files alone do not make a lakehouse

Keeping Parquet files on object storage does not on its own make a lakehouse. At enterprise scale, tables need the following managed:

• Schema changes
• Partition layouts
• Snapshots
• Metadata
• Transaction state

Modern lakehouse architectures therefore use open table formats such as Apache Iceberg.

Apache Iceberg is an open table format built for large analytic datasets; it supports schema evolution, partition evolution and snapshot-based table state. Data can therefore sit in open formats on object storage while still being managed as enterprise tables.

Apache Iceberg

Apache Iceberg can be used in the Kauzas Data Platform's lakehouse architecture.

Rather than closing the data itself inside a proprietary database format, Iceberg builds a table metadata layer over data files such as Parquet, ORC or Avro. That architecture brings real capabilities with it.

These are the properties that stop a lakehouse being a file store.

The data catalog

A lakehouse may hold thousands of tables, and it is not enough for them merely to exist on storage. Systems need to know:

• Which tables exist
• Their schemas
• Their metadata
• Their current table state

The catalog is therefore a critical component of a lakehouse architecture.

In the Kauzas architecture, a catalog approach that works alongside open table formats makes it possible to build architectures where different compute engines use the same enterprise data layer.

Separating storage from compute

In traditional data platforms, storage and compute are usually tightly bound inside the same system. In a lakehouse architecture the two layers can be separated.

This reduces the need to move the same data into another platform for every new use, and it lets compute capacity scale independently against the workload.

Querying lakehouse data with distributed SQL

Data in the lakehouse has to be queryable in SQL, quickly and at scale. In the Kauzas architecture, distributed SQL query engines such as Trino can be used.

Trino can query data in a range of table formats, Iceberg among them, and across a range of storage systems including S3 and S3-compatible object storage. Lakehouse data is therefore available to:

• BI
• Ad-hoc analytics
• Data applications
• Other data services

Apache Spark and data processing

A lakehouse is not only a query layer. Data has to be processed, cleaned, transformed, enriched and prepared for machine learning.

In the Kauzas Data Platform, distributed data processing technologies such as Apache Spark serve these workloads, so SQL analytics and large-scale processing run against the same lakehouse data layer.

The data lakehouse and AI

An AI model is worth what the enterprise data it can reach is worth. When an organization's data sits disconnected across a data warehouse, a data lake, operational databases, document systems and applications, every AI project starts by building new data integrations again.

A lakehouse can become one of the shared data layers that AI works against.

A modern lakehouse is not only business-intelligence infrastructure; it is the data foundation of enterprise AI.

The lakehouse and enterprise RAG

Enterprise RAG systems do not have to draw their knowledge only from documents. Structured enterprise data in the lakehouse is another source an AI application can use.

When a user asks “which products grew most in sales last month?”, the answer can come from structured data. When they then ask “what are those products' technical specifications?”, the answer can come from documents.

The AI system can use structured and unstructured sources within the same conversation.

More on enterprise RAG →

The lakehouse and AI agents

Carrying out enterprise tasks, AI agents may need current operational and analytical data as much as documents. An agent might query data in the lakehouse, analyse the results, use a machine learning model and then act through the enterprise tools it has been given.

But an agent need not reach every piece of data in the lakehouse. Data access policy should be weighed against the agent's identity alongside the user's permissions.

More on AI agent governance →

An open lakehouse, without vendor lock-in

Holding data in open file and table formats helps separate the storage and compute layers. Open table formats such as Apache Iceberg let the same data layer be used by different query and processing engines.

This does not remove vendor lock-in altogether, but it can reduce how tightly enterprise data is bound to one compute technology's or one platform's proprietary data layer.

The Kauzas Data Platform takes a lakehouse approach built on open, replaceable technology layers.

More on the open data lakehouse and vendor lock-in →

The lakehouse on the Kauzas Data Platform

The Kauzas Data Platform treats the lakehouse not as a single proprietary technology but as open, replaceable technology layers working together.

This architecture runs on Kubernetes, so organizations can take the same data platform approach on-premise, in a private cloud and in suitable public cloud environments.

The layered Data & AI architecture →

The on-premise lakehouse

A modern lakehouse architecture need not be tied to a particular public cloud provider. With S3-compatible object storage, open table formats and compute technologies running on Kubernetes, a lakehouse can be built on the organization's own infrastructure.

Trino's documentation likewise shows lakehouse workloads working with S3-compatible storage and HDFS as well as cloud object storage. The same architectural approach therefore holds across different infrastructure models.

More on the on-premise AI platform →

Who a lakehouse suits

The lakehouse approach is worth weighing particularly for organizations that:

• Have many data sources
• Process high volumes of data
• Are reconsidering the cost or scalability of a data warehouse
• Want analytics and machine learning running against shared data
• Are building the enterprise data foundation for generative AI projects
• Have on-premise or hybrid cloud requirements
• Do not want their data dependent on proprietary storage formats

Frequently asked questions

What is a data lakehouse?

A data lakehouse is a modern data architecture that brings together the flexible, scalable storage of a data lake and the table management and analytical capability of a data warehouse.

What is the difference between a data lake and a data lakehouse?

A data lake is mainly about storing data of different kinds at scale. A lakehouse adds table management, metadata, transactions and richer analytics on top of that.

What is Apache Iceberg?

Apache Iceberg is an open table format built for large analytic datasets. It offers schema evolution, partition evolution, hidden partitioning and snapshots.

Can a lakehouse be used for AI?

Yes. Structured and unstructured data held in a lakehouse can serve as the enterprise data source for machine learning, generative AI, RAG and AI agent scenarios.

Can a lakehouse be built on-premise?

Yes. On-premise lakehouse architectures can be built with object storage, open table formats and Kubernetes-based compute technologies.

Is the Kauzas Data Platform a lakehouse?

The lakehouse is one of the Kauzas Data Platform's core architectural components. Kauzas provides a wider platform layer on top of it, covering data processing, distributed SQL, catalog, governance, orchestration and Data & AI use cases.