Stop replicating your SAP data to hyperscalers cloud storage for your ML use cases. You're loosing time, money and don't offer the right insights !

What's the issue ?

Everybody agrees that hyperscaler platforms are able to manage huge quantities of unstructured/semi-structured data and providing analytics and AI capabilities on those data sets.

However, to lay the foundation for analytics as well as data science experiments on their platform,?the hyperscalers are creating the?need for businesses to extract their business data out?to cheap cloud storages because without business data there is no context to the analysis they will provide on the non structured/semi-structured data they have collected from other sources.

This forced data replication is due to the fact that analytics & building of?machine learning models by those hyperscalers can only work or work seamlessly when the data resides on the respective hyperscalers platform’s native cloud storages.

This inadvertently brings in the need for expensive ETL and data pipelines to move the data across systems (sometimes with CDC to satisfy realtime replication) but it’s not just about additional cost !

It?leads to data inconsistency issues because as data science people don’t know what they don’t know about SAP processes and SAP's highly normalized data structures, when they simply extract out SAP data, they often make significant mistakes when they recontextualize it in their data lake. If the data is re-contextualized incorrectly there is no coherent way to use that data and send correct insights back.

On top, this is taking away the time and focus of data scientists, as they are the ones who end?up tackling data sourcing issues.

The Solution

SAP Federated-ML or FedML is a library built to address this issue. The library applies the Data Federation architecture of SAP Data Warehouse Cloud and provides functions that enable businesses and data scientists to build, train and deploy machine learning models on hyperscalers, thereby eliminating the need for replicating or migrating data out from its original source.

By abstracting the data connection, data load and model training on these hyperscalers, the SAP FedML library provides end to end integration with just a few lines of code.

On top, SAP FedML will help organizations to avoid vendor lock-in and aids them with reduction of their hyperscaler storage costs, and adherence to GDPR policies,?as data migration is eliminated. It also enables instant access to cross-cloud data sources, combined with SAP Business data managed through SAP Data Warehouse Cloud’s unified semantic models.?

Interested to see how it works ?

Azure is your hyperscaler of choice, then check this blog on how to immediately take advantage of SAP FedML with Azure ML 2.0

Google is your hyperscaler of choice, then check this blog on how to immediately take advantage of SAP FedML with Google Cloud Vertex AI 2.0

Amazon is your hyperscaler of choice, then check this blog on how to immediately take advantage of SAP FedML with Amazon SageMaker 2.0

Only SAP data required for your advanced analysis use case ?

SAP’s Python package?hana_ml?and R package?hana.ml.r?make it easy to trigger such an advanced analysis from any Python or R environment on the data held in SAP Data Warehouse Cloud / SAP HANA Cloud. And those packages allows you also to easily take advantage of the hundred ML algorithms provided by the SAP HANA APL, PAL libraries as well as taking advantage of the SAP HANA Graph Engine and Spatial Engine.

Just install those packages in a notebook that runs in your hyperscaler’s environment (e.g install HANA ML to Databricks notebook),?trigger processing both in-HANA and Spark from that same notebook, and bring results of data processing to local environment (via .collect() statement) as needed.

Check those few examples :