Data+AI: Snowflake releases Arctic LLM today – why you should pay attention...

It's a big day for Snowflake, as they release their new integrated Gen AI LLM called Arctic.? This has been a huge focus for Sridhar Ramaswamy after he took over as CEO from Frank Slootman in Feb 2024… less than a year after his AI company, Neeva, was acquired by Snowflake in May 2023.

This comes only a month after Databricks announced their new LLM, DBRX.? This followed their acquisition of MosaicML in July 2023.? Databricks now positions itself as not just a Data platform, but a Data+AI platform.? Their AI offering is called Mosaic AI, and it's an integral part of their Data Intelligence Platform.

Why are these cloud data companies so concerned about integrating AI into their data platforms??

Data and AI, separately, are both important and powerful in their own right, but Data+AI together is the business game changer.?

Here's some basics to get you started…

Retrieval Augmented Generation (RAG)

Adding your proprietary data to a Gen AI model is the key to unlocking business value, and this is called Retrieval Augmented Generation (RAG).? The obvious use case here is the AI chatbot that answer questions about your products based on being trained on product literature.? But what if a customer just types "are there any recalls on my car?" If the customer is logged in, your chatbot shouldn't ask which model and year, because it can look up the actual VIN, then look up any recall information for that specific car, look at that car's service records to determine which ones this person has already taken care of, and then explain (in plain English) which ones may still need to be addressed, and offer some available appointment times at their usual dealership – this is all possible, today, if you join up Data+AI.

Governance

As you open your data to Gen AI, you need to make sure users don't get access to data they shouldn't.? Two users that ask your Gen AI the same question should get different answers based on the corporate data they personally have access to or are restricted from – very important, and difficult to do if you're pulling together Data and AI solutions from different vendors.? Databricks governs this with their Unity Catalog, which is a unified access model that ensures the AI models in Databricks are only augmenting their answers with data that specific user has access to – pretty clever.

Which AI Model to use?

OpenAI's ChatGPT is fun to play with on your own, but it can quickly get expensive for high scale commercial use (because they charge by the word).? And do you really need the entire world's training data in your AI model… allowing your customers to ask your support chatbot about any random topic… like politics... or the winning attributes of your competitor's product?? (Chevy dealership’s AI chatbot suggests Ford F-150 when asked for best truck (msn.com))

There are currently over 600,000 different AI models to choose from on Hugging Face, and you can plug into whichever one you choose within Snowflake or Databricks.? Will you use an open source AI model that you can host for a lower "server-based" cost model, or do you really need that powerful, general, and proprietary LLM (like ChatGPT) that operates more like LLM-as-a-service with a "pay-per-request" model?? Chances are, for most business use cases, you actually don't want or need ChatGPT.? You can even chain together different models to perform specific tasks in a workflow: one to turn an image into text, one to find data in the text, yet another to summarize the data into a coherent written summary, and maybe even another to translate that into another language.

?

Fine Tuning vs. Pre-Training

Fine Tuning is when you take an existing pre-trained foundation model and further train it with your proprietary data.? Pre-training is when you start with a more "bare bones" model and train it with your proprietary data from the ground up.? Fine Tuning is typically faster and cheaper to start, but may have a higher chance of resulting in unintended consequences if it's given too much freedom.? Pre-training is typically slower and more expensive to start, but ultimately results in a smaller footprint that can run on fewer servers and give faster responses, and with a more limited/focused knowledge base.

?

Gen AI Fundamentals

If you want to explore these topics further, I'd recommend the 1 hour Gen AI Fundamentals course from Databricks. If you pass the assessment at the end, you even get a badge you can add to your LinkedIn profile.

?

P.S. This article was written by me, a person, without any AI assistance... except for that fancy banner graphic... ;-)