Azure Synapse Reference

Azure Synapse Reference

?New to Synapse : Start here

?Azure Synapse Playlist - https://www.youtube.com/channel/UCsZ4IlYjjVxqe5OZ14tyh5g

?Synapse SQL Architecture

?Azure Data Store, Criteria & Decision tree and Synapse Access, Disaster Recovery

?New Query Optimization Techniques in the spark engine of Azure Synapse (2022) - https://vldb.org/pvldb/vol15/p936-rajan.pdf

?Implementation Success Methodology - https://learn.microsoft.com/en-us/azure/synapse-analytics/guidance/implementation-success-overview

?Learn: Proof of concept playbook - https://learn.microsoft.com/en-us/azure/synapse-analytics/guidance/proof-of-concept-playbook-overview

?Azure Synapse Security White Paper - https://learn.microsoft.com/en-us/azure/synapse-analytics/guidance/security-white-paper-introduction

?Demo: Industry Aware Analytics Using Azure Synapse Database Templates

?Analytics end-to-end with Azure Synapse

?Great set of links, solutions, accelerators at Awesome Synapse

?Synapse Analytics Blog

?Frequently asked questions?

?Synapse Toolbox & Monitor

?Serverless SQL pool QPI – Query Performance Insight - https://www.dhirubhai.net/pulse/troubleshooting-performance-serverless-synapse-sql-pool-jovan-popovic/

?Serverless SQL pool self-help - Azure Synapse Analytics | Microsoft Learn

?Q&A – Technical questions about Azure Synapse Analytics

?Dedicated Vs Serverless SQL Pool Blog & on-demand

?Synapse ML Blog & Videos & dotnet & Samples

?Pricing?and?Memory and concurrency limits

?Introduction & Full course

?30-day Azure Synapse analytics learning challenge for data engineers – Limitless analytics with unmatched time to insights

?Synapse Sessions from Simon?@ advancinganalytics.co.uk

?Azure Synapse and Power BI Datamart – Better together

?Parquet & Delta & MLFlow

?Azure based Data Mesh, Data Domains, Data Products, Data Contracts & Data Marketplace

?Data product canvas

?Unlocked: Cloud Scale Analytics – Scalable Data management, Analytics, Data Mesh & Fabric

?Choose a data store

?Northtrop Johnson Retailers ?- Customer story - Synapse Link to Dataverse, Dynamics 365

?Synapse Link to Cosmos DB Gremlin API

?Table Distribution Design, Best practices & Common performance tuning pointers

?Dedicated SQL Pool - Understand MPP, Cheatsheet, Difference and all about with cross-region

?Distributed SQL Engine in Azure Synapse

Synapse Spark Notebook:

?Create a spark pool using synapse studio (billing prorated per minute): https://learn.microsoft.com/en-us/azure/synapse-analytics/quickstart-create-apache-spark-pool-studio

?Spark Auto Scale: https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-autoscale

?Call Synapse Pipeline with a notebook activity (template) : https://learn.microsoft.com/en-us/azure/data-factory/solution-template-synapse-notebook

?Understand Synapse Spark Basic Configuration & Job Definition vs Notebook

?Synapse Notebook Activity: https://learn.microsoft.com/en-us/azure/synapse-analytics/synapse-notebook-activity

?Microsoft SparkUtils : https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/microsoft-spark-utilities

?Manage Spark Pool Packages: https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-manage-pool-packages

?Interpret Error Codes in Synapse Spark: https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-handle-livy-error

?Python logging in Notebook: https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-development-using-notebooks#python-logging-in-notebook

?Samples: https://github.com/Azure-Samples/Synapse/tree/main/Notebooks/PySpark

?Optimize Write on Spark: https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/optimize-write-for-apache-spark

?Spark Log Analytics Monitoring -?https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-azure-log-analytics

?Stream Analytics - Write to Delta Lake APPEND ONLY (Ignite 2022) - https://learn.microsoft.com/en-us/azure/stream-analytics/write-to-delta-lake

?How vCores and Memory get allocated for spark pool

?How notebook settings can be tweaked to use executor cores in a spark pool

? Spark Pool Configuration Override default settings for advanced users

?Learn to Debug Package Failure

?Hyperspace - Spark Performance Optimization using Hyperspace Indexes

?Bring data from on-prem, troubleshoot, tune and features with diagnosis and monitor

People, Blogs and Videos:

?Synapse Espresso – Jovan Popovic, Filip Popovi? and Stijn Wynants

? Liliam Cristiman Leme - Blogs

? James Serra - https://www.jamesserra.com/

?Nick Salch

?Data Exposed – Anna Hoffman | Anna Hoffman @AnalyticAnna

?Ryan Majidimehr – MVP series

? Pawel Potasinski Azure Synapse and Power BI Datamart

? Vengatesh Parasuraman – Synapse Security - Outbound, Inbound, Managed VNET

?Giles Lherault – Azure Synapse CSE Blog

? Paul Andrew - https://mrpaulandrew.com/category/azure/synapse/

? Arshad Ali Channel – Azure Synapse CSE

? Nellie Gustafsson – Synapse ML Github Binder

? Mark Hamilton – Synapse ML with Open AI

? Piethein Strengholt – Data Products & Data Mesh

? Scott Hanselman 's Azure Friday?- Scott Hanselman

? Mohammad Ghodratigohar (MG) – Synapse MG cafe

?Guy in a cube – Adam and Patrick

? Sarnendu De Blog

Anuj Pandey

Senior Cloud Solution Architect - Data & AI at Microsoft

2 年
Raj Sharma

EX-FIS Global || Tredence || Consultant || Data Engineer || Data Analytics - Databricks, Python, SQL, Pyspark, ADF, Synapse

2 年

Hi Srikanth Bhakthan , I hope you are doing well. I would like to thank you for the above post it really helpful and i am going through this to these stuff to brush up my skills. But I have to learn Azure Data engineer Concepts as well for my current role. I need some assistance on this how to start & which one is good training . Please guide me on this awaiting your response

Biju K Das

Associate Principal | Data Architect | Data Modeler | Data & AI | AWS Certified Solution Architect - Associate

2 年

Thanks for sharing

Hamish MacDonald

Lead Data Engineer at Azenix

2 年

The data movement into dedicated sql pools still a friction. Interesting to see the developments in the medium term around serverless pools.

要查看或添加评论,请登录

Srikanth Bhakthan的更多文章

  • A CxOs Guide to Capturing Value using Generative AI

    A CxOs Guide to Capturing Value using Generative AI

    Views are Personal. Links to latest ~70 articles, research papers.

  • Switching LLMs - The Invisibles

    Switching LLMs - The Invisibles

    Opinions are personal, do not reflect my employer. Properitary to Open Source LLM Model switch is more than just a…

  • Azure Open AI - GPT - LLM Reference

    Azure Open AI - GPT - LLM Reference

    A set of links, videos, papers, code and surveys to learn Azure Open AI LLMs. Fundamentals: Stanford University course…

    1 条评论
  • ChatGPT Prompts : Tested

    ChatGPT Prompts : Tested

    Basics: A burger and coke costs $10 in total. The coke cost five times the cost of a burger.

  • ChatGPT Ref

    ChatGPT Ref

    Explain Code with ChatGPT ChatGPT from OpenAI - Microsoft for Startups' trusted partner Where to try it:…

  • Microsoft Purview - Why, What, When, Who & How - End to End

    Microsoft Purview - Why, What, When, Who & How - End to End

    Microsoft Purview - Data Governance, Landscape, Discovery, Classification, Lineage, Security and more..

社区洞察

其他会员也浏览了