How can you distinguish between data warehouse and data lake?

Understanding a data warehouse as a centralized repository is crucial. It collects various data into one place, aiding in accessibility and analysis. Structured, curated data supports business intelligence, analytics, and reporting, saving analysts' time. Predefined schemas, like star or snowflake models, provide an organized structure, simplifying data navigation. Lastly, data integration, transformation, and quality processes ensure data consistency and reliability, emphasizing the importance of rigorous data management.

Fabrizio said it best. The only thing I would add is that there is also some overlap between Data Lake and Data Warehouse technologies. For example, with Snowflake you can have unstructured - or semi structured data like CSV, XML and JSON and use their Lateral and Flatten operators to turn that data into structured. One very interesting idea that I have seen is to use the Data Lake as a "load bay" and data discovery area - and then use the Data Warehouse once you have curated it and defined it in a structured way. I see the two technologies as complimentary - not competing.

After what has been said by the others, I would like to add that Data Warehouses are usually built from three approaches: Enterprise Data Warehouses, Data Marts, and Virtual Warehouses. An Enterprise Data Warehouse collects all the information about subjects stretching across the organization. It provides corporate-wide data integration, usually from one or more corporate systems, and is cross-functional in its scope. A data mart usually contains a subset of corporate-wise data that is of value to a specific group of users. The scope is confined to particular selected subjects. A virtual warehouse is a set of views over operational databases that are easy to build but require excess capacity.

Adding further, in my experience, the data warehouse is now essential solution for modern business intelligence and analytics. It enables organizations to gain valuable business insights and improve operations, customer service, and gain a competitive edge.

Data warehouse projects cost up to 2 million and up to 80% of them fail. Why this? The main reason could be the definition! To say, what is a data warehouse? Originally, begining 90's, DW environments were used to separate BI from production. This was necessary because BI required transformed and integrated data from distributed sources to generate insights for business continuity and decision-making processes. Because of this, DWs used different forms of data modeling called dimensional. This is where the concept of a data mart, or OLAP, that leverages star schemas and snowflakes as the core modeling technology arose. I would define a data warehouse as an integrated repository only to be used for statistical data analysis and BI purposes.

One insight I've gained from understanding a data lake is the inherent flexibility it provides. Being a distributed system of raw and unstructured data, it enables data exploration and innovation by storing data in its original form. Unlike a data warehouse, a data lake uses a schema-on-read approach. This means the data doesn’t need to be cleaned or transformed before it's stored. Instead, it's structured at the time of analysis or usage, offering flexibility in accommodating various data types and uses. The ability of a data lake to ingest, store, and provide access to data from various sources and types enhances its versatility.

A key differentiator that the Data Lake introduces is the separation of storage and compute resources, possible with the recent technologies of cloud computing. With this approach storage cost have lowered since the data no longer needs to be duplicated in various transformation layers like staging, normalised and dimensional and the storage format is open and accesible to a variety of processing languages that no longer need to be one-size-fits all for the entire enterprise.

Typically, the tagging and cataloging of video, image, sound and other files is done to describe such data. Some image data - such as that from modern digital cameras have EXIF Metadata which helps greatly in at least understanding the "schema" of that data. But its content or contextual description is much harder to infer - typically a human is needed for this. A Data Catalog is very much needed as its easy to quickly get overwhelmed with the volume of such data.

Data Lake is a place where you can store all your electronic data sources that is yours or you gained access to with no limits to format and size of them

We can understand a data lake as a pond where a wide variety of data types, whether it's structured, semi-structured or unstructured, at any scale, can be stored and analyzed. It works fine either with relational or non-relational data from a variety of sources—business applications, mobile apps, IoT devices, social media, or streaming.

Easy to catalog and query using standard SQL. Easy to apply data security frameworks and provide contextual documentation of this data. Easier to infer based on standardized naming.

In my experience, data warehouses provide a stable, centralized repository for large amounts of historical data, improve data quality and trustworthiness, and, together with a federated data governance strategy, can improve business processes and decision-making with actionable insights.

In my opinion, the key benefit of having a data warehouse is capability to store a huge data volumne in a structured, relational key-pair basis, and can be consumed for business and data intelligence, big data queries and analytics, got better data based decisionand actions.

I once had to use a Python OCR library to scan old scanned legal documents and then apply a risk score for that file. I kept a Data Catalog in a Data Warehouse - so I used both a Data Lake and a Data Warehouse. The Data Warehouse was the "card catalog" of this unstructured data. It was great to have both technologies - and not try to shoehorn both approaches using just one of the two technologies.

Data lake is like a big unorganised library with all books but not structured/organised. You will basically will have all kinds of books. Data warehouse is just a science section of library with all organised. Easy to find any book with proper cataloguing and organised. Having a library(data lake) and then keeping it section wise organised (data warehouse) are needed for good eco system

Biggest advantage of a data lake is can store raw data at any scale - relational and not-relational - low cost solution, mainly useful for machine learniner, predictive and real-time analytics applications.

A data warehouse is like a tidy bookshelf for structured knowledge, while a data lake is a sprawling reservoir for unsorted exploration. Combining both brings the best of organized insights and limitless discovery to your data strategy.

For a data lake, no need to define any structure or schema of data until it is accesed, as it supports schema-on-read functionality. Also, can be leveraged for ML, AI based use cases. In contrary, a data warehouse is relational in nature. The schema need to be pre-defined to load the data in a strcutured manner. Mainly, used for BI and big data analytics.

Think of a data warehouse as your reliable hub for structured data analysis and streamlined reporting. It excels at handling well-organized information and is your go-to for historical insights. On the other hand, a data lake offers an open playground where you can accommodate diverse data types, from structured to unstructured. It’s a space for creativity, experimentation, and diving into raw data to uncover hidden gems. Interestingly, the real magic happens when you combine the strengths of both. By integrating a data warehouse and a data lake, you're setting up what experts refer to as a 'lambda' or 'kappa' architecture. This powerful combination empowers you to harness the full spectrum of data possibilities.

Hybrid architectures where data lakes and data warehouses are combined, offering a punch-out from the warehouse to search for detailed information in the data lake not present in the warehouse schema, or vice versa, curating information flowing into the data lake into clean, usable, structured information that can be a source for the data warehouse is an effective strategy in many cases. Sometimes referred to as a "lake house", it does require careful consideration of access controls, data quality and data semantics as these are often less mature in the data lake component of such an architecture

It's no longer sufficient for organizations to confine themselves to standard data warehouses solely for structured data, at the same time data lake by itself is not a strategy. With the diverse range of decision intelligence encompassing ( BI, ML, AI and GenAI) alongside the varied data types including structured, semi-structured, and unstructured data, adaptation to Data Lakehouses is imperative. The expectation now is that any type and source of data should be readily available to business for consumption, boosting business agility and accommodating the growing variety of requirements and use case development.