Microsoft Fabric

Today, at the Build Conference, Microsoft unveiled its latest Data and Analytics solution named Microsoft Fabric. Microsoft Fabric integrates existing services such as Data Factory, Synapse, and Power BI into a single unified platform for all data and analytics tasks. It consists of diverse components and experiences, designed to meet various data needs. My exploration today will focus on a key part of this new solution - OneLake.

OneLake, as the name suggests, is the unified data foundation for Microsoft Fabric. It functions as a universal Software as a Service (SaaS) Data Lake, comparable to OneDrive but dedicated to data.

This powerful tool provides a single, logically unified SaaS data lake to serve your entire organization, eliminating data silos. It's an automatic provision with every Fabric tenant and doesn't require additional management efforts. The platform takes care of that. OneLake implements data governance features automatically on all data entering the lake, including data protection, certification, and catalog integration, to name a few.

Workspaces within OneLake allow different sectors of the organization to work independently while contributing to the same data lake. Each workspace can have individual administrators, access control, region, and billing capacity. Tenant admin rules are inherited by the workspaces, reducing repetitive governance tasks and resources coordination. Workspace admins are in control of the data within their workspace.

OneLake has been designed to allow workspaces in different regions, implying that data in those workspaces will also be located in those regions. It is a virtual manifestation of multiple Azure Data Lake Gen2 storage accounts.

OneLake promotes the data mesh pattern, allowing different business units to function independently within the same data lake, without the burden of managing different storage resources. OneLake takes it a step further by introducing the concept of domains, grouping workspaces to represent a single business domain like Sales, providing admins with a higher level of control.

All data in OneLake is part of a Fabric data item, and many of these are already designed to store their data in OneLake using open file formats. These data items include Power BI datasets, a fully transactional data warehouse for T-SQL developers, and a Lakehouse for Data engineers or Data Scientists. The beauty of Microsoft Fabric lies in the fact that it allows developers with various skillsets to interact with the same data using different technologies.

OneLake introduces data shortcuts, symbolic links that point from one data location to another. This feature eliminates the need to physically copy data from one source to another. Instead, you create shortcuts in the lakehouse pointing to the warehouse (or other locations). This mechanism ensures that any changes in the source data are immediately reflected in the lakehouse. It also allows you to consolidate data across workspaces and domains without altering data ownership.

In terms of security, OneLake ensures secure data access at both the item and workspace levels. Additional, engine-specific security can be defined, such as object-level security using T-SQL. Power BI reports that use DirectQuery mode will respect these security settings transparently.

In conclusion, Microsoft Fabric, with its fundamental component OneLake, paves the way for the future of data platform construction. As it transitions into Public Preview, I eagerly anticipate its rapid maturation into a platform that empowers various roles, and above all, delivers exceptional value for our customers.

DLytica - Data Analytics and AI does the implementation of this Architecture.

You can schedule a call for free Architecture consulting.