Azure Data and Power BI News (October 2022)

Boom! October has just passed. Pretty intensive month, I must say. We're in the middle of the conference season! And in October one event was especially important people passionate about Microsoft technologies - Microsoft Ignite 2022. You can check out all the announcements in the Microsoft Ignite 2022 Book of News .

But there was much more last month than one event and its announcements. Let me go through my October findings related to Azure data services and Power BI.

Azure Data

• The October update of Azure Synapse Analytics was purely related to Ignite announcements. Three new features were highlighted and demoed in the video: new Microsoft 365 Pipeline Template Dataflows (Public Preview) , SAP Change Data Capture (CDC) Connector (General Availability) , and R language support in Synapse Spark (Public Preview) .
• Azure Synapse blog was full of great posts in October. Let me share some of my favorite blog posts published last month: Synapse Spark - Encryption, Decryption and Data Masking , Choosing Between Spark Notebook vs Spark Job Definition , Monitoring serverless SQL ended requests by using Log Analytics , How To - Advanced Properties of Linked Services .
• Also, last month we started a new blog series Azure Synapse MVP Corner. In this series we highlight a selected set of content created by Microsoft Most Valuable Professionals (MVPs). Read the first blog post of the series: Azure Synapse MVP Corner - September 2022 .
• Some exciting updates in Azure Data Factory! First, you can now track billing of the service by individual pipelines . Then we got the announcement of Azure Data Factory Community Templates (community members can share their pipeline templates to be published in the template gallery). Finally, the new Azure Data Factory Learning Center is now available, providing lots of learning resources on ADF, like the?latest features updates, getting started tutorials, best practice videos, training modules, product links, and more.
• Last month was full of riddles for Azure Data Explorer (ADX, aka Kusto) fans! If you haven't had the opportunity to test yourself in Kusto Detective Agency, don't waste your time and join the game: https://detective.kusto.io . It's a great way to learn Kusto and KQL, and also achieve some cool badges and - if you're fast - win some prizes too! I'm right after the 4th mission and I must admit they were all super fun and engaging.
• October brought a long-awaited update to Microsoft Purview. Manual data lineage bridges the gap in coverage of end-to-end lineage, where automated lineage of the data system is not natively supported in?Purview. Also, for Purview do not miss two blog posts co-authored by a good friend of mine, Bartlomiej Graczyk : Data curation: Discover more with data estate insights in Microsoft Purview and Catalog Adoption: Discover more with Data estate insights in Microsoft Purview . What good is data catalog if the data is not classified or if its assets are not being used by the users? Must read!
• Nice update to the Form Recognizer v3.0 . Its document layout analysis model extracts new structural insights like paragraphs, titles, subheadings, footnotes, page headers, page footers, and page numbers. The latest model also enhances existing features like extraction of text (OCR), tables and selection marks. Check it out with your documents!
• Any Azure SQL fans here? You got some nice updates too: new TempDB configurations (GA), auto-scale compute to higher limits with up to 80 vCores in selected regions using Azure SQL Database serverless (GA), Import/Export Service on Azure SQL Hyperscale Database (GA), Approximate Percentile Functions for Azure SQL DB and Azure SQL MI (Public Preview), add table constraints as online and resumable operations in Azure SQL Database and Azure Managed Instance (Public Preview).
• Let's stay in SQL world. Big news from Ignite: the distributed PostgreSQL comes to Azure Cosmos DB . With that you get: true PostgreSQL with latest versions, truly managed cloud database (high availability configurations across availability zones, fork or restore your cluster to a particular point in time, or one-click upgrade your PostgreSQL & database extensions), global scalability thanks to Citus .
• Lots of goodies from the Azure Data community in October! Some of the resources I discovered last month: a blog post Deploy Synapse workspaces via DevOps - Setup by Joost van Rossum (Microsoft MVP), two blog posts Update both ends of Azure Synapse Link for SQL Server 2022 using Azure DevOps and A complete CI/CD experience for Azure Synapse Link for SQL Server 2022 by Kevin Chant (Microsoft MVP), a blog post Connecting to Azure Storage from Synapse Analytics using Private Endpoint by Adrian Chodkowski (Microsoft MVP), a blog post What are Synapse Analytics Database Templates and why should you use them? by Barry Smart, two blog posts User Permissions in Serverless SQL Pools: External Tables vs Views and Dedicated SQL Pools DMVs: Show Name of Synapse Studio Script Being Executed by Andy Cutler (Microsoft MVP), solution accelerator Azure Synapse Lakehouse Sync by Shane Ochotny and Bret Myers (both Microsoft), a video Locking Down Azure Synapse Analytics by Kevin Feasel (Microsoft MVP), a blog post Mastering DP-500: Score ML models in Synapse with PREDICT function! by Nikola Ilic (Microsoft MVP), a blog post Power BI using Synapse Serverless Pool with Delta Lake format by Dan English (Microsoft), a video Azure Synapse News - August & September 2022 by Simon Whiteley (Microsoft MVP).
• Talking about community... Do not miss the Azure Data Community Newsletter Sept/Oct 2022 by Rie Merritt (Microsoft) to find out what's going on in the community (events, meetups, ...) and learn some best practices related to community leadership.
• To wrap-up Azure Data news - this month I have a customer story from Japan for you. NTT TechnoCross (a part of NTT Group) uses Azure Synapse Analytics and Power BI to enable budget control for all departments. Read more: Building DWH and Analysis System Using Azure Synapse Analytics — “Future Data Utilization” Vision of NTT TechnoCross Corporation .

Power BI

• Of course, number one topic last month for Power BI were Ignite announcements . For me the biggest one was the one about automated migration tool from AAS to Power BI Premium. Learn more on this topic by watching a webinar Roadmap for Semantic Models: from Azure AS to Power BI Premium delivered by Christian Wade and Ogbemi Ekwejunor-Etchie from the Power BI product team. Also, Power BI becomes a part of the Office 365 installer, which will reduce deployment effort for IT. Another nice feature announced (in public preview) was the Cross-tenant Power BI Dataset Sharing . This capability allows customers to share their Power BI datasets with external users (consumers) in a way that allows these external users to access the datasets in their own Power BI tenant. Last but not least, we got an announcement of the new Optimize ribbon in Power BI Desktop which will enable better experience for reports working with large data models (for example by providing an option to refresh individual visuals instead of the whole report page).
• The October update for Power BI also contained several nice features. I'd highlight the following ones: query performance improvements (TOP N filter pushed down to DirectQuery sources!), quick measure suggestions (you write English - Power BI generates DAX!), cool improvements to modeling (DAX editor in model view + relationship editing in the properties pane), more options for .pbix download in the Power BI service.
• But there was even more than that in October. We got several improvements to chart sharing and copy as image (share visual using a link or send it directly to chat in Teams, multiple copy visual options). Also, last month we got submetric roll ups in Power BI Metrics . With this feature it’s easier to aggregate metric values so you can spend less time crafting multiple scorecards and connections, and more time analyzing your performance. Last but not least, we got some improvements of Power BI Ideas site (improved search during new idea creation).
• Finally, do not miss this video: Power Platform – 2022 Release Wave 2 Launch - YouTube , containing demos of all the cool features (in-place visual editing!) coming to Power BI in the next 5 months (until March 2023).
• The Power BI community was super active in October. Some content I'd like to highlight this month: a blog post No More Auto Aggregations in Dimension Tables (on a hidden feature of data modeling) by Matt Allington (Microsoft MVP), a blog post Mastering DP-500: Identify Data Loading Bottlenecks in Power BI! by Nikola Ilic (Microsoft MVP), an important blog post Why Power BI totals might seem inaccurate (there is a kind of a "religious war" around this topic!) by Marco Russo and Alberto Ferrari (both Microsoft MVPs), a blog post (in "live update" mode) A look at automatic partition maintenance by Power BI Service for datasets with Incremental Refresh by Shabnam Watson (Microsoft MVP), a blog post Most needed ideas from ideas.powerbi.com (yes, the infamous one on table/matrix totals is there ??) by Adrian Chodkowski (Microsoft MVP), three blog posts - What Is The Difference Between A Live Connection And DirectQuery Mode In Power BI? (this one is super important, as many people still do not distinguish these modes!), Diagnosing Calculation Group-Related Performance Problems In Power BI Using EvaluateAndLog , and Speed Up Power BI Dataset Refresh Performance In Premium Or PPU By Changing The “Parallel Loading Of Tables” Setting by Chris Webb (Microsoft), and a blog post Visualise Filters Applied by Kerry Kolosko (Microsoft MVP).
• Customer story alert! Find out how Walmart Finance standardizes on Power BI for reporting and analytics . The scale of data in case of this customer is huge - 10,500 stores and clubs and 2.3 million associates worldwide mean trillions of transactions.

Traditionally, you can find all the official August announcements for Azure and Power BI on the?Azure Updates website ?and the?Power BI blog . Also, you can be up to date with Azure Synapse by tracking?monthly update blog ?and with Power BI thanks to the?Power BI Release Plan ?report.

That's all folks! I hope you enjoyed this month edition of my newsletter. As always, please share your findings and thoughts. Looking forward to reading your comments! Thanks, and we catch up on the next one.