Transforming Customer Support with Retrieval-Augmented Generation (RAG) on SAP BTP

In today's fast-paced business environment, providing exceptional customer support is more critical than ever. Companies face numerous challenges such as manual email processing, insufficient customer insights, and ineffective data analysis. These challenges lead to slower response times, decreased customer satisfaction, and inconsistent service quality. The introduction of advanced AI technologies like Retrieval-Augmented Generation (RAG) on SAP Business Technology Platform (SAP BTP) offers a revolutionary solution to these issues.

Introduction to Retrieval-Augmented Generation (RAG)

What is RAG?

Retrieval-Augmented Generation (RAG) is a neural architecture that combines the strengths of large pre-trained language models with external retrieval or search mechanisms. The main goal of the RAG architecture is to improve the capability of language models by allowing them to pull relevant information from a vast corpus, similar to how search engines retrieve relevant web pages based on queries. RAG is used for various tasks such as question answering and knowledge-intensive NLP tasks. This architecture represents a fusion of retrieval-based and generation-based approaches to NLP.

How RAG Works

The RAG architecture enhances the capabilities of language models through a three-step process:

1. Question Encoding: The user provides a question or prompt, which is then encoded by a sequence-to-sequence model into a dense vector.
2. Document Retrieval: This dense vector is used as a query to retrieve relevant documents or passages from a large corpus. The retrieval is typically done using a dense vector space search, where documents in the corpus are pre-embedded in the same dense vector space. The top-k most relevant documents or passages are retrieved based on their proximity to the query vector.
3. Answer Generation: The retrieved documents and the original question are fed into a Large Language Model to generate an answer. The model is fine-tuned on a downstream task to generate relevant responses based on both the input question and the retrieved passages.

The SAP BTP RAG Architecture

The provided architecture diagram showcases a CAP-based multi-tenant SaaS architecture utilising RAG. This architecture is designed to seamlessly combine various Large Language Models (LLMs) using SAP AI Core. By leveraging the capabilities of LangChain in the CAP model and implementing advanced methods such as custom schema-based output parsing or Retrieval Augmented Generation (RAG) with embeddings and a vector database, businesses can maximise the benefits for their specific needs.

Components and Services in the Architecture

1. SAP HANA Cloud: This component is used to store, process, and federate data in a cloud infrastructure. It provides the necessary backend for handling large datasets required for RAG.
2. SAP AI Core: This platform builds and manages artificial intelligence solutions, providing the foundational infrastructure for deploying AI models and handling AI workloads efficiently.
3. SAP AI Launchpad: This service manages AI scenarios across all landscapes and deployment options. It provides a centralised platform for monitoring and managing AI operations.
4. SAP BTP, Cloud Foundry Runtime: This runtime environment allows the operation of polyglot applications, offering flexibility and scalability for different application components.
5. Destination Service: This service retrieves information about destinations in the Cloud Foundry environment, ensuring seamless integration and data flow.
6. SAP HTML5 Application Repository Service for SAP BTP: This service is used to develop and run HTML5 applications in a cloud environment, supporting the front-end needs of the application.
7. SAP Authorization and Trust Management Service: This component manages application authorizations and connections to identity providers, ensuring secure access and operations.
8. SAP Cloud Transport Management: This service manages the transport of development artifacts and application-specific content, facilitating efficient deployment and updates.
9. SAP Business Application Studio: This integrated development environment is used for developing, debugging, testing, and deploying SAP business applications.
10. SAP Continuous Integration and Delivery: This service configures and runs predefined pipelines for continuous integration and delivery, ensuring smooth and automated deployment processes.
11. SAP Application Logging Service for SAP BTP: This component creates, stores, accesses, and analyses application logs, providing critical insights into application performance and issues.
12. Application Autoscaler: This service automatically scales applications to meet their dynamic resource needs, ensuring optimal performance and cost-efficiency.
13. SAP BTP, Kyma Runtime: This runtime extends SAP solutions using cloud-native microservices and serverless functions, providing agility and scalability for modern application development.

Detailed Use Case: Enhancing Customer Support in a Travel Agency

Scenario Description

The reference architecture illustrates a multi-tenant application developed for SAP Business Technology Platform (SAP BTP), tailored for enhancing customer support within a travel agency. This scenario presents a comprehensive SaaS solution for improving email insights and automation.

Key Features and Benefits

1. Email Analysis: The system analyses incoming emails using Large Language Models (LLMs) to offer core insights such as categorisation, sentiment analysis, and urgency assessment. This ensures that emails are processed efficiently and appropriately prioritised.
2. Key Fact Extraction: The system goes beyond basic analysis by extracting key facts and customisable fields like location, which are managed through a dedicated configuration page.
3. Historical Email Embeddings: One innovative feature involves utilising email embeddings to identify similar historical emails. This aids in understanding how similar requests were handled previously, fostering consistent and efficient customer service.
4. Summarization and Translation: The system demonstrates capabilities of summarizing and translating both email subject and body, enabling streamlined comprehension across languages and breaking down language barriers.
5. Response Generation: The system takes automation to the next level by generating potential responses for customer inquiries. This response generation is influenced by configurable actions and services, enhancing response accuracy and speed.
6. Seamless Integration: The flexibility to connect with SAP systems like SAP Concur adds an enterprise dimension, allowing seamless integration of processes and data. This optimizes operations and ensures that customer support is aligned with business workflows.

Implementation Steps

1. Setting Up the Environment: Begin by setting up the necessary environment on SAP BTP. This includes provisioning SAP HANA Cloud, SAP AI Core, and other essential services.
2. Developing the Front-End: Use SAP HTML5 Application Repository Service and SAP Business Application Studio to develop a responsive and user-friendly front-end for the application. This involves creating interfaces for email analysis, key fact extraction, and configuration pages.
3. Implementing Email Analysis: Leverage SAP AI Core to implement LLMs for analyzing incoming emails. Fine-tune models for categorisation, sentiment analysis, and urgency assessment. Integrate these models with the email processing workflow.
4. Key Fact Extraction: Develop custom schemas for extracting key facts from emails. Use LangChain and vector databases to manage and query extracted information efficiently.
5. Embedding and Retrieval: Implement email embeddings to identify similar historical emails. Use dense vector space search to retrieve relevant emails and provide context for handling new inquiries.
6. Summarization and Translation: Integrate summarisation and translation capabilities into the email processing workflow. Use SAP AI Core and language models to provide accurate and concise summaries and translations.
7. Automating Responses: Develop a module for generating potential responses based on analyzed data and historical handling of similar inquiries. Ensure that the response generation module is configurable to adapt to different business rules and actions.
8. Integration with SAP Concur: Implement seamless integration with SAP Concur for travel-related operations. Use the Destination Service and other relevant components to retrieve and process necessary data.
9. Testing and Deployment: Conduct thorough testing of the application to ensure all components work seamlessly. Use SAP Continuous Integration and Delivery for automated deployment.
10. Monitoring and Scaling: Set up monitoring using SAP Application Logging Service. Use the Application Autoscaler to ensure the application scales dynamically based on demand.

Business Problem Solved

Addressing Customer Support Challenges

Businesses often grapple with several customer support issues that hinder their efficiency and customer satisfaction. Here’s how the RAG-based solution on SAP BTP addresses these challenges:

1. Manual Email Processing: The AI solution automates email handling, reducing the need for manual processing and enabling faster response times.
2. Insufficient Customer Insights: By leveraging advanced email analysis and key fact extraction, the system provides deep insights into customer inquiries, helping businesses understand and address customer needs more effectively.
3. Ineffective Data Analysis: The integration of LLMs and embedding-based retrieval ensures that data analysis is robust and accurate, enabling businesses to make data-driven decisions.
4. Language Barriers: Summarisation and translation capabilities break down language barriers, allowing businesses to provide support to a global customer base.
5. Slow Response Times: The automation of response generation significantly reduces response times, ensuring that customers receive timely and accurate support.
6. Lack of Automation: The system automates various aspects of customer support, from email analysis to response generation, improving efficiency and reducing the workload on support teams.
7. Adaptability Issues: The flexible architecture allows the solution to be adapted to different industries and business needs, ensuring that it can grow with the business.

Outcomes and Benefits

1. Enhanced Customer Satisfaction: Faster, personalised, and consistent customer service leads to higher customer satisfaction.
2. Increased Employee Efficiency: Automation reduces the manual workload on employees, allowing them to focus on more strategic tasks.
3. Cost Savings: Efficient automation and data-driven decision-making lead to significant cost savings.
4. Adaptability and Competitiveness: The solution's flexibility allows businesses to adapt to changing market needs and stay competitive.
5. Optimised Operations: Seamless integration with systems like SAP Concur optimizes operations and ensures that customer