Azure Data Lake – Microsoft stretched vision for making big data easy
The Azure Data Lake Store offers a single warehouse where you can with no trouble capture data of any size, type and speed without forcing modifications to your application as data scales. In the store, data can be securely shared for collaboration and is accessible for processing and analytics from HDFS applications and tools like HDInsight, Hortonworks, Cloudera, or MapR.
Azure Data Lake includes all the capabilities required to make it easy for developers, data scientists, and analysts to store data of any size, shape and speed, and do all types of processing and analytics across platforms and languages. It removes the complexities of ingesting and storing all of your data while building it faster to get up and running with batch, streaming, and interactive analytics. Azure Data Lake works with existing IT investments for identity, management, and security for simplified data management and governance. It also incorporates flawlessly with operational stores and data warehouses so you can enlarge current data applications.
What’s new in this expanded vision?
Azure Data Lake Analytics allows you focus on the logic of your application, not the distributed infrastructure running it. As an substitute of deploying, tuning and configuring hardware, you mark queries to modify your data and take out valuable insight. Built on Apache YARN, and designed for the cloud, the analytics service can handle jobs of any scale instantly by simply setting the dial for how much power you need. The analytics service for Azure Data Lake is cost-efficient because you only pay for your job when it is running, and support for Azure Active Directory lets you direct access and tasks simply and mingles with your on-premises identity system.
Many developers and data scientists resist to be successful with big data using existing technologies and tools. Azure Data Lake Analytics comprises U-SQL, a language that merges the benefits of SQL with the significant power of your own code. The U-SQL language is built on the same distributed runtime that influences the big data systems inside Microsoft. Millions of SQL and .NET developers can now process and analyze all of their data with the skills they already have.
The U-SQL support in Azure Data Lake Tools for Visual Studio includes state of the art support for authoring, debugging and sophisticated performance analysis features for increased output when optimizing jobs running across thousands of nodes. Visualizations of your U-SQL code allows you to see how your code runs at scale and identify performance bottlenecks and cost optimizations, making it easier to tune your queries.