Azure Data and Power BI News (October 2023)

And so, we are in November with beautiful golden Polish autumn (see the picture above) and exciting times ahead! Don't miss Microsoft Ignite 2023 and some cool announcements at this conference!

But you know what? October was also packed with some important updates and new features announced to Microsoft Fabric and Power BI. I'll get to that in a minute.

October was also the last month in my role on the Fabric community team. Since November 1st I'm a member of the Fabric Customer Advisory Team (CAT) working with the biggest customers using Microsoft Fabric, Power BI and Azure Synapse Analytics. This awesome team is full of people with high recognition in the community ( Matthew Roche , Adam Saxton , Patrick LeBlanc , Chris Webb , Kasper de Jonge , Benni De Jagere , Gabi Münster , Phil Seamark and Alex Powers - to name a few). I'm looking forward to learning tons of new things here! Of course, the community has a permanent place in my heart and I will continue to actively support it ??

But hey, let's get back to the major topic of this newsletter and see what Microsoft and the community shared last month.

Microsoft

• Finally! The roadmap for Microsoft Fabric is publicly available. Check it out and add to your favorites to keep an eye on it: https://aka.ms/FabricRoadmap .
• Speaking about Microsoft Fabric, we got the Microsoft Fabric October 2023 update with some new features and improvements including: V-order write optimization for warehouses, data warehouse SKU guardrails for burstable capacity , KQL database capacity reporting , improvements to KQL database autoscaling algorithm , Azure Machine Learning activity in pipelines. Also, we got several things announced in public preview: Microsoft OneLake shortcut for Power Platform and Dynamics 365 , Data Activator (the 7th experience in Fabric for automatically taking actions when patterns or conditions are detected in changing data - see a demo video below), Semantic link (a feature that allows you to establish a connection between Power BI datasets and Data Science and Data Engineering experiences in Microsoft Fabric). Last but not least - in a separate announcement we got Fabric warehouse publishing full DML to Delta Lake logs .

• One more small update to Fabric - now we have colorful icons for Fabric items. Read more here: Fabric’s New Item Icon System . You can download old and new icons as one ZIP file here: https://aka.ms/fabricicons .

• Do not miss the latest videos in the Fabric Espresso series recorded by Estera Kot, PhD , Stijn Wynants , Filip Popovi? and other members of the Fabric product team. Some recent videos I recommend: Spark Data Engineering Patterns Optimizing Delta Tables for Power Bi in Microsoft Fabric , Row-Level security in Fabric Warehouse & SQL Endpoint , Microsoft Fabric Notebooks - Showcase with advanced features and Microsoft Fabric Spark integration and VS Code .

• For Power BI we got Power BI October 2023 update with some nice new features, including Model explorer public preview with calculation group authoring and creating relationships in the properties pane, customized deployment pipelines (now you can have up to 10 stages), Power BI Desktop OneDrive and SharePoint integration (now you can even share reports stored in OneDrive and SharePoint directly from Power BI Desktop), and deduplication rules for composite models on Power BI Datasets and Analysis Services.

• If you have been using automatic aggregations in Power BI there is one blog post on the official Power BI blog that you should read - Troubleshooting automatic aggregations to optimize DirectQuery performance .
• One news for Azure SQL fans - the Transparent data encryption (TDE) with database level customer-managed keys is now generally available for all editions of Azure SQL Database (it's not supported for SQLMI and SQL Server, Azure VMs and Azure Synapse).

Community

• Let me start this list of community contributions with a useful tool I discovered last month - Azure Diagrams developed by three Microsoft'ers: Shane Ochotny , Harsha Konduri and Bret Myers . This web-based tool lets you draw high-level architectural diagrams of your solutions using Microsoft technologies, including Microsoft Fabric.

• Let me continue with more contributions from Microsoft employees: an article Medallion architecture: best practices for managing Bronze, Silver and Gold by Piethein Strengholt and two blog posts - What does it mean to refresh a Direct Lake Power BI dataset in Fabric? and Getting report visual IDs with Power BI Desktop Developer Mode (find out which specific visual in the Power BI report generated the query) - by Chris Webb .
• In October we got tons of community content on new features in Fabric, specifically on Semantic Link and Data Activator. Here's a list of my favorites: a blog post Connect Power BI and Spark notebooks with Microsoft Fabric Semantic Link – Seequality by Adrian Chodkowski (Microsoft MVP), a blog post Exploring Direct Lake Framing and warm-up data using Semantic Link in Fabric Notebooks by Marc Lelijveld (Microsoft MVP), a blog post YOUR NEW ACTION HERO! What is Data Activator? by ?? Johnny Winter ?? , a video Advancing Fabric - Power BI meets Spark with Semantic-Link! by Simon Whiteley (Microsoft MVP), a video Fabric Fridays: Data Activator in Microsoft Fabric #22 by Christopher Wagner, MBA, MVP and Kevin Arnold (both Microsoft MVPs), a video Semantic link in Microsoft Fabric by Amit Chandak (Microsoft MVP), a blog post Augmenting Gold Layer with Semantic Link and SparkSQL in Microsoft Fabric (how to query your Power BI datasets using Spark SQL) by Nikola Ilic (Microsoft MVP), three blog posts - Fabric Semantic Link and Use Cases , Measure Maze: Visualizing Measure Dependencies Using Semantic Link & Network Analysis (this one I enjoyed the most!) and Formatting DAX Expression Returned By SemPy in Fabric - by Sandeep Pawar .
• One of hot topics in the Fabric community is OneLake. Check out the following content: a blog post & video What is OneLake in Microsoft Fabric, and Why You Should Care? by Reza Rad (Microsoft MVP) and a blog post What is OneLake in Microsoft Fabric? by Marthe Moengen (Microsoft MVP).
• Also, in October we got some blogs and videos on more general topics for Fabric, like data governance, adoption/learning, using Fabric with other platforms for analytics. My favorite content in this area: a blog post How do you set up your Data Governance in Microsoft Fabric? by Marthe Moengen (Microsoft MVP), a blog post Tear down walls, no data silos any longer using Microsoft Fabric, and finally, export to Excel will become a breeze by Thomas Martens (Microsoft MVP), two blog posts - Does it feel like too much? (how to approach learning Fabric without feeling overwhelmed) and How to use Tableau with Power BI and Fabric - by Kurt Buhler (Microsoft MVP), a video Do the Thing : Microsoft Fabric and Databricks - Practical Examples by Alex Rostan and Gaston Cruz (both Microsoft MVPs).
• Reza Rad (Microsoft MVP) shared to blog posts supported by videos covering the basics of two experiences of Fabric: Data Science in Microsoft Fabric and Fabric Real-Time Analytics .
• Andy Cutler (Microsoft MVP) presented his own custom scenario for loading data into the Fabric lakehouse: Fabric Lakehouse Loading using Data Pipelines & Notebooks – Inspired by MS End-to-End Tutorials .
• Reid Havens ????♂? (Microsoft MVP) shared a 10-minute video to showcase new capabilities of Model explorer in Power BI Desktop: Exploring the NEW Semantic Model View + Native Calculation Groups! .
• Two interesting blog posts by Sam Debruyn : Is Microsoft Fabric just a rebranding? (spoiler: NO, it's not a rebranding :-)) and Fabric: Lakehouse or Data Warehouse? (a guide on when to use what in Fabric).
• Mimoune Djouallah came up with an interesting blog post on the usage cost of every compute engine Fabric: Fabric : Engines Resource Consumption .
• Kevin Chant (Microsoft MVP) shared three blog posts on one of his favorite topics - CI/CD for Fabric: Git integration and deployment pipeline updates on the Microsoft Fabric roadmap , CI/CD for Microsoft Fabric Data Warehouses using Azure DevOps and CI/CD for Microsoft Fabric Data Warehouses using YAML Pipelines .
• Mathias Halkj?r Petersen (Microsoft MVP) shared two interesting Fabric-related articles on LinkedIn - Deep-dive into Direct Lake and Warping through Data pipelines .
• Two new videos in the Learn Fabric series by Michael Carlo and Tommy Puglia (both Microsoft MVPs): Writing Data into Tables - Let's Learn Fabric - ep.18 and Intro to GIT Integration - Let's Learn Fabric - ep.19 .

My 3 cents for the community last month were: 1) the presentation titled First look at Microsoft Fabric for Microsoft 365 users delivered at Excel Virtually Global online event, 2) Microsoft Fabric MVP Corner – October 2023 - a curated list of content published recently by MVPs, 3) participating in building the agenda for the Cloud Based Solutions track of Data Science Summit .

You can find all the official announcements and updates on the?Azure Updates website ,?Microsoft Fabric blog ?and the?Power BI blog .

That's it from me this time! Traditionally, please share your favorite new features/improvements in Fabric and Power BI, and the community content I missed last month. Thanks for reading and until next time.