OneLake in Microsoft Fabric

In a world where data is an asset, having a simple way to manage it is essential. That’s where OneLake comes in a single, easy-to-use data lake for your entire organization, built right into Microsoft Fabric.

How do organizations leverage data?

• One Data Lake: Say goodbye to data silos! OneLake provides a central place for all your analytics data, making teamwork easier.
• One Copy of Data: Use the same data across different tools, ensuring everyone stays on the same page without duplication.
• No Hassle: With every Microsoft Fabric account, OneLake is automatically set up for you no extra work required!

Before OneLake, many companies created multiple data lakes for different teams, leading to confusion and extra management work. OneLake solves that by bringing everything together, ensuring that every Fabric account has just one data lake.

Hierarchy:

• OneLake
• Workspaces
• Lakehouses (within workspaces)
• Delta Tables (within lakehouses)

OneLake is a smart storage solution built on Azure Data Lake Storage (ADLS) Gen2, designed to handle all kinds of data both structured (like spreadsheets) and unstructured (like videos). When you use Microsoft Fabric, any data you work with automatically goes into OneLake.

For example, if a data engineer uploads customer purchase records and a SQL developer adds sales updates, both types of data are stored in the same place. All this data is kept in a special format called Delta Parquet, which helps keep it organized and easy to use. I’ll share more about Delta Parquet in a separate article soon!

Think of OneLake as a large storage room for your organization. It works with the same tools you might already use in ADLS Gen2, making it compatible with apps like Azure Databricks. Each workspace you create acts like a separate container in this room, with different data organized into folders.

For example, the Sales team could have a workspace with folders for reports, customer data, and marketing materials. This setup makes it easy for everyone in the organization to access and collaborate on data!

One Copy of Data OneLake allows you to use the same copy of data with different analytical tools, making it easier to work across various applications. Typically, data is set up for one specific tool, which can make it hard to share and reuse. But with Microsoft Fabric, all the different engines like T-SQL, Apache Spark, and Analysis Services store data in a common format called Delta Parquet. This means you don’t have to make copies of the data just to use it with another tool.

For example, imagine a team of SQL engineers working on a transactional data warehouse using the T-SQL engine to create tables and load data. If a data scientist wants to analyze this data, they can do so directly using the Spark engine without needing any special setup. Since all the data is stored in Delta Parquet format in OneLake, the data scientist can easily access it and leverage Spark’s powerful libraries for their analysis.

Governance OneLake automatically governs your data, establishing clear security and compliance boundaries within your organization. A tenant defines your organization’s data limits, managed by a tenant admin who sets governance rules. While the admin oversees governance, they cannot block other teams from accessing OneLake, fostering collaboration across departments.

Example: Imagine a large company, XYZ Tech Corp.

• Tenant: XYZ Tech Corp has one tenant that encapsulates all its data operations.
• Workspaces: Within this tenant, the admin creates separate workspaces for the Sales and Marketing teams. Each team can set its own access policies while still adhering to the overall governance rules.
• Collaboration: The Sales team can easily share data with the Marketing team without waiting for approval from the admin, fostering a collaborative environment.