Evolving trends in cloud data modernization – key takeaways from Snowflake Snowday

Snowflake hosted its winter Snowday event on November 1, when the company announced its latest advancements and product features across the data cloud.

?While the announcements spanned the foundational cloud data modernization tenets of Data Architecture and Engineering, Data Management and DataOps, the emphasis on machine learning-related features — and more specifically support for large language models (LLM) — clearly took center stage.

Taking note of the trends shaping the data modernization conversations within enterprises, I thought I’d share my key takeaways from Snowday 2023:

1. LLM capability infusion

?This has been the breakout year for generative artificial intelligence (AI), and Snowday clearly emphasized the game-changing applicability of LLM use cases on data. The most exciting announcement was the launch of Snowflake Cortex, a managed service comprising task-specific models and Structured Query Language (SQL) functions to invoke the models for performing tasks such as summarization, translation and extracting answers. Cortex addresses two big challenges with LLM adoption:

·?????? Data security: With Cortex, Snowflake runs models on GPU infrastructure within the perimeter and security architecture of a customer’s Snowflake account. The customer data remains within the same region as the Snowflake account like it does with virtual warehouse compute.

·?????? Need for AI talent: The ability to do retrieval augmented generation with a few lines of SQL code is very powerful. This opens the potential for every Snowflake developer to also be an LLM app developer.

?2. Data science and exploration focus

?Exploratory and predictive analytics use cases along with associated user personas such as data analysts and data scientists are gaining focus in enterprises. These user personas are predominantly driving the data modernization strategy.?

Catering to these user personas has become extremely important, and to that extent, these eagerly expected features were announced:

?·?????? Notebooks will soon be natively available

·?????? Feature engineering and model training API enhancements along with a native feature store

?Notebooks and Cortex take Snowflake’s capability for data exploration and analysis to a higher level.

?

3. Enhanced Data Governance and DataOps support

?As enterprises become data-mature on cloud, the focus on optimization and governance has grown. These featured announcements cater to this trend:

?·?????? Cost management UI to provide not only visibility but also control and manage workloads. Enterprises are now building their own dashboards to understand and manage Snowflake costs. This UI will be a huge benefit for smaller enterprises that cannot afford to build these dashboards.

·?????? CI/CD support for database management to ensure DDLs in Git are aligned with what’s defined in the database

·?????? Data lineage UI and data quality monitoring UI

?

4. Support for interoperability

?Enterprises have gone through periods of data migration from one data platform to another in the last several decades. They understand the importance of storing data in open formats such as Iceberg to avoid a similar migration in the future.

?·?????? Snowflake’s support for Iceberg storage format with similar read performance as their own format

·?????? Iceberg Catalog API to read Snowflake’s metadata

?

In conclusion, the advances in AI and machine learning will push data platforms to be easily consumable for business and will continue to drive data modernization trends.

?

The views reflected in this article are the views of the author and do not necessarily reflect the views of Ernst & Young LLP or other members of the global EY organization.