SAP Generative AI Hub - Part 2 of Business AI services on BTP

The generative AI hub gives you instant access to a broad range of large language models (LLMs) from Microsoft Azure -OpenAI or Google Cloud -Vertex AI (PaLM 2 and Gemini models). Generative AI Hub is not available as a separate service but as part of SAP AI Core with the Extended service plan.

The service can only be deplyed via Core AI services runtime on BTP and Models from Azure OpenAI are accessed through a private instance of the chat-completions API. The access points are not publicly accessible and can only be accessed through SAP AI Core. Users can deploy the model by creating a resource group with the respective model’s name and version to create a deployment URL which can be used by a User facing App.

As you can see in the below reference architecture, the deployment uses "Retrieval-Augmented Generation or RAG", which is a neural architecture that combines the strengths of large pre-trained language models with external retrieval or search mechanisms. The main goal of the RAG architecture is to improve the capability of language models by allowing them to pull relevant information from a vast corpus, much like how search engines retrieve relevant web pages based on queries. RAG is used for various tasks such as question answering and knowledge-intensive NLP tasks. The architecture represents an interesting fusion of retrieval-based and generation-based approaches to NLP.

Key Benefit of the RAG Architecture is even if the base LLM has not been trained your organization’s information, if that information exists in the corpus base used for retrieval, RAG can still provide relevant answers.

You can configure the trust and access settings in AI core and the model itself is hosted on either SAP AI Core runtime or Azure-OpenAI or GCP instance. You can use foundational OpenAI models like gpt-35-turbo or gpt-4 – point to note here is models have deprecation dates and will stop working after deprecation date. You can deploy the model in Auto Upgrade or Manual Upgrade mode based on your use case.

You can apply tenant segregation and zero data retention rules including no 3rd party training, privacy settings, de-identification, and re-identification (of personal data) on the use of the models.

The generative AI hub is available as consumption-based pricing within Cloud Platform Enterprise Agreement (CPEA). Customers with existing cloud credits can directly consume LLMs upon entitling SAP AI Core in their BTP subaccount, without the need to manage different commercial frameworks from various LLM providers.

One interesting deployment option in the Generative AI Hub is content filtering. You can filter out Hate, Sexual, Self-Harm, violence, and Jailbreak risk!