SAP AI Ambitions (Part III-Final)

In my three-series blog about AI ambitions with SAP, I described SAP's recent pieces of information around AI with the intent to give more details on the subject but very focused on SAP and Generative AI.

I tried to go beyond buzzwords aiming to push away from existing articles and entrepreneurs claiming to use AI in any form because it looks innovative and cutting edge.

In the first blog, I talked about what Generative AI can do for SAP, what SAP can do with Generative AI, and the release of SAP Business AI (Generative or not)

The second blog is about SAP announcing its collaboration with Cohere, Anthropic, and Aleph Alpha; Foundation Models, and LLMs for SAP.

In this third and last blog, I will talk about BTP and SAP Platforms for Generative AI.

I will use the last Christian Klein statements and add opinions as I did in the two previous blogs.

Christian Klein statement 5; Reliable AI hinges on applying the wide data to the wide model.

By using SAP DataSphere to leverage substantial contact switch industry-specific data, business AI system can drastically improve accuracy, generate more relevant content, and minimize AI hallucinations. Our customers trust us with their most critical data and can confidently deploy our AI offerings, knowing we prioritize the highest levels of security, privacy, compliance, and ethics. We comply with the highest standards when it comes to customer consent, security, GDPR regulations. We will continue to innovate and deliver by creating an AI ecosystem for the future, combining SAP and partner innovation built on the SAP business technology platform.

DataSphere and Business Technology Platform. How do they participate in all this?

An AI platform is where the machine learning teams collaborate on models, datasets, and applications. Tasks of an AI platform include

• Browsing and selecting Foundation Models
• Inference APIs
• Train and consume / Build Pytorch, and Tensorflow models
• Dataset Management
• Actions like the ability to train and deploy Transformers & Diffusers on chipmakers, like cloud-consumed versions AWS Trainium / AWS Inferentia or train on Amazon Sagemaker / Hugging Face / Vertex AI

I will lead with the AWS example, probably the most prominent cloud Machine Learning platform available, providing customers the ability to build their machine learning models,?Amazon SageMaker?provides all the tools necessary to build, train, and deploy ML models at scale. Sagemaker has been around for several years and is already the platform of choice for thousands of customers. More specifically, to Generative AI Foundation Models and LLMs, Amazon released Amazon Bedrock?in April 2023, still not General Availability, as an easy way to build and scale generative AI applications with foundation models (FMs) from the worlds leading third-party providers. Bedrock can use a wide range of services that allow you to add AI capabilities like image recognition, forecasting, and intelligent search to applications with a simple API call. Many customers and startups have already been using a superb platform called Hugging Face.

Hugging Face is a collaborative Machine Learning platform in which the community has shared over 150,000 models, 25,000 datasets, and 30,000 ML apps.

One of the most critical topics about Foundation Models is that many of them are highly reusable and require little training. Some prompt engineering might be enough, making Hugging Face a fantastic collaborative platform used by many vendors.

What SAP has, and SAP doesn't have on BTP.

Do we have a Machine Learning Platform? Yes, that is BTP. Is BTP providing the full lifecycle for Generative AI?

Let's roll back to March 2023. SAP "released" DataSphere. Initially, as an evolution of SAP Data Warehouse Cloud, combined with SAP Data Intelligence Cloud, DataSphere is a complete data service acting as the foundation for the business data fabric; DataSphere is designed to help businesses deliver business data with all its context and logic intact to the data consumers within an organization.

With that release, SAP announced four strategic new partnerships to support the idea of a Business Data Fabric and better integrate non-SAP data in a unified usage context.

Databricks –?Integrates SAP Data, with its complete context, with Data Bricks Data Lakehouse platform

Collibra –?Data Governance, privacy, and compliance.

Confluent –?Sets data in motion with its data streaming platform

DataRobot –?An AI Data Platform.

I will focus on DataRobot for the importance it has for this blog.

DataRobot provides enterprises with a data science platform that helps data scientists experiment with and deploy ML models in production to drive better business outcomes. DataRobot is one of the leader AI/ML Platforms .

This partnership between SAP and DataRobot enables customers to take advantage of multimodal AutoML capabilities of DataRobot in conjunction with SAP DataSphere, wherein DataSphere acts as a logical source of federated ML task data.

DataRobot is an AI Platform. AI Platform features Generative AI all kind of Machine Learning tasks. There is where the communities build and share demos, use cases, models, datasets, and much more.

Accessing a plethora of Foundation Models for Computer Vision like Image Classification, Image Segmentation, Object Detection, Video Classification, or NLP features like Conversational models, Text generation, and Classification models. Text, Audio, Video... is what makes AI Platform one of the three key pillars of a Generative AI lifecycle management.

BTP allows different approaches to machine learning. it's been the case for years before BTP was called BTP and it was called Leonardo.

SAP distinguish between side-by-side and Embedded ML depending on where the Machine Learning algorithms reside; if it's Embedded, they reside on S/4HANA like PAL; if its Side-by-side, the ML algorithms reside on BTP (typically in SAP DataSphere or the SAP AI Core and SAP AI Launchpad), while the business logic stays either on SAP BTP or SAP S/4HANA depending on the application requirements.

SAP Offers AI Core and AI Launchpad, can it be used for Generative AI?

AI Core and AI Launchpad products allow us to productize and operate AI models on BTP that natively integrate with SAP applications.

With SAP AI Core we can train, deploy, and monitor ML models. SAP AI Launchpad is a Software as a Service application that provides a visual interface to manage AI scenarios from SAP and external vendors through API.

But AI Core and AI Launchpad don't allow us to execute the main technical tasks needed for Generative AI, which are Train, Validate, Tune, and Deploy.

Then, again today it's easy and feasible to deploy an LLM in an external AI Platform (like DataRobot or Hugging Face) and consume it from BTP. It's easy to deploy a GPT3 Model from Microsoft Azure and consume it from BTP, but we don't control the data transfer or the model training from SAP; we consume it.

an AI Studio/Platform is a missing piece

For Generative AI, an AI studio is where enterprise scientists and developers can build, run and deploy AI based on machine learning and generative AI using multiple libraries of high-quality, domain-specific foundation models, from the vendor itself, third party or community created if Open Source.

For example, AI Platforms from cloud vendors (IBM, Google, and Amazon) integrate with Hugging Face to provide access to thousands of open models and datasets. Customers can choose the best models and architecture depending on their business needs at the same time as they build their LLMs on the cloud platform of choice.

AI Data Platform

An AI data platform provides access to businesses to build and refine generative AI foundation models and traditional machine-learning systems.

Data originating from SAP systems contain critical information on supply chains, financial forecasting, or human resources records; for that, SAP released DataSphere to combine this mission-critical data with data from across the enterprise landscape, regardless of its origin, using a data federation model.

Beyond Analytics, the ability to use AI and ML services to train models on data originating from SAP and non-SAP will determine the success of this new era of Generative AI and NLP models.

With generative AI, we sit on a data of over 400,000 customers and the material flows, the financial flows, employee customer data. Now we are taking this data, not only with Signavio to benchmark and give business process recommendations. We see it in the first prototypes that we are going to be able to not only that the system can self-learn on this data on how to improve all these workflows. The systems itself will also drive further automation of workflows going forward. They can look into the customization of an ERP, which is huge in on-prem. They can help customers to generate code on the platform to build differentiating capabilities to fasten the time to value.

This statement probably has excessive hype; let's discuss how Data is processed.

Let us first dive into the types of data a generative AI service or application provider may process if we decide to fine-tune and prompt tune an existing foundation model, using the example reference diagram below:

Generative AI is a relatively new technology, and many customers still need more expertise to create, train and run the models. There still needs to be more information if they can use and fine-tune an existing model or if they need to deploy their models, if the current models are safe, and how they share that information, and here is where Christian Klein is going.

Enterprise clients without Generative AI experience might have many questions about the topic. How do they execute a model? How do they make the chatbot customer-facing or employee-facing? How do they scale it up? And, importantly, how do they have confidence that the outputs will be accurate?

This SAP AI Studio/Platform should include the following three main items (conceptually)

AI Model Development

The studio must allow to train, test, tune, and deploy machine learning models.

Any existing studio (from other vendors) comes with various foundation models, training, and tuning tools. SAP model library should be made up of curated and trained foundation models based on SAP enterprise data because who better than SAP knows the Data Model? Another benefit from SAP not available in existing studios is that they could be trained in language, code, time-series data, and SAP events data.

Data Management (Storage and Federation)

Building Datasets. Generative AI must be scalable. Generative AI's ability to deliver customer benefits will be limited by its capacity to be tuned to an enterprise’s unique data and domain knowledge. Businesses must control AI models and data through specialization, deployment, management, and governance, resulting in owning the value of the data they produce.?The good news is that SAP already offers exciting tools in this space with the previously mentioned DataSphere.

Sometimes, using a proprietary platform for LLMs could make sense for Enterprise customers; the training data for AIs need to come from somewhere, and we’re not always so sure what gets stuck inside the neural networks. If you use an untrusted platform, what if AIs leak enterprise information from their training data?

To make matters worse, locking down AIs is much more complicated because they’re designed to be so flexible. A relational database or analytical tool can limit access to a particular table or record with personal information, but an AI can be queried in dozens of different ways. Attackers will quickly learn how to ask the right questions in the right way to get at the sensitive data they want.

Whether the customer decides to train its proprietary foundation model or fine-tune and prompt-tune an open-source/commercial foundation model, it is critical that they take the necessary steps to mitigate potential data security and privacy risks.?

Data Governance (Privacy and Security)

Governance management aims to mitigate the risks associated with AI and protect customer security and privacy concerns, all should be managed from the platform, and Klein, Julia White, and SAP, in general, have been quite clear for this purpose that the company will provide a tool to address it.

SAP should also include data governance tools to make information more accessible to regulators and third parties. This is important because new European and US government legislation is being formulated. Data governance is necessary for every data management tool to make processes easier for stakeholders and allow customers to manage regulatory changes.

Also, the AI Studio/Platform should include mechanisms to protect customer privacy, proactively detect model bias and help organizations define and meet their ethical standards.

SAP has also been keen to show that it is?preparing for expected legislation?on AI transparency and compliance, establishing its own AI ethics policy and compliance processes.

As a summary

The extensive market disruption caused by OpenAI's ChatGPT has led to many new AI companies being formed, possibly with immature and unvetted products.

By contrast, SAP, which will be rolling out an AI platform in the coming months, is one of the world's most respected IT companies and their AI research teams have a long history of successful ML research and AI/ML product implementations in other products for Demand Planning, Material Requirements Planning or Sales and Operations planning, previously APO and now IBP.

SAP will be rolling out something sometime this fall. No one should discount or minimize the importance of Data Modeling and Governance. Clean data will keep the model "honest" and free from creating false and biased outcomes.

If an enterprise plans to use generative AI, it must have confidence in the model’s safety and its suitability to be moved into a production environment safely. Companies can gain the necessary confidence in models by having visibility into their data. That is the only way to ensure it is clean and compliant with company standards and mandatory regulations.