Microsoft today announced a new cloud-based service for running queries on big data stored in Microsoft’s growing public cloud, named Azure Data Lake. The new service, according to the Redmond giant, will make big data processing and analytics easier and more accessible. The service tags along Azure Data Lake Store that will act as a central repository for you to easily capture data of any size, type and speed without forcing changes to your application as data scales. This is expected to be available in preview later this year. Also, Azure Data Lake utilizes a new Microsoft-built query language which is essentially a fusion between SQL and C#.The new query language that this analytics service utilizes is called U-SQL and it derives primarily, as already stated, from the widely used SQL query language and Microsoft’s C# programming language. The C# integration could be because of the fact that Microsoft wants to take advantage of its large array of programmers.
This move from Microsoft will surely help its Azure service to be distinguishable in the ever-increasing crowd of public cloud providers with data services out there.
Microsoft also announced general availability for Azure HDInsight today. This service gives you the breadth of the Apache Hadoop ecosystem in a managed service that is monitored and supported by Microsoft over a Linux based system. Also, Azure Data Lake Tools for Visual Studio will now come in with new debugging and authoring capabilities and will offer support for the Apache Hive data warehousing software.
T. K. “Ranga” Rengarajan, Microsoft Data Platform corporate vice president, will be addressing us all about big data processing and analytics on Tuesday in his keynote speech at AzureCon, Microsoft’s free virtual event about Azure. Meanwhile, here’s how he explains the working behind U-SQL today:
We know that many developers and data scientists struggle to be successful with big data using existing technologies and tools. Code-based solutions offer great power, but require significant investments to master, while SQL-based tools make it easy to get started but are difficult to extend. We’ve faced the same problems inside Microsoft and that’s why we introduced, U-SQL, a new query language that unifies the ease of use of SQL with the expressive power of C#. The U-SQL language is built on the same distributed runtime that powers the big data systems inside Microsoft. Millions of SQL and .NET developers can now process and analyze all of their data with the skills they already have. The U-SQL support in Azure Data Lake Tools for Visual Studio includes state of the art support for authoring, debugging and advanced performance analysis features for increased productivity when optimizing jobs running across thousands of nodes.
