SAP Data Analytics on GCP - Way Forward
Kiran Kere
Data & Analytics Leader - As a Data and Analytics leader, I'm passionate about harnessing disruptive data technologies to drive innovation and create competitive advantages
Many organizations are moving towards the cloud transformation journey for data . Analytics from this data on cloud becoming important aspect which is deriving lot of value for all of our clients.
?When organization just have SAP data getting generated out of their transaction and data warehouse SAP systems like SAP BW on HANA, SAP BW , SAP ECC, S4HANA , SAP Native HANA etc SAC (SAP Analytics cloud) becomes the?preferred solutions to get analytics out of SAP Data. SAP Analytics Cloud solution also?brings together analytics and planning in a single solution in the cloud.
As many organizations have data from disparate source systems like SAP and NON SAP obviously they think about cloud analytics solutions where SAP data can be combined with NON SAP systems data quickly at real time. This article mainly focusing on deriving the cloud analytics for SAP data from GCP – Google cloud platform which is provider of very powerful data analytics tools like Google BQ – Big query, Google data proc and Google looker and so on.
GCP platform is helping to get powerful data analytics and valuable insights for SAP data. This article talks about quickly few practical approach and architecture for staging SAP data with the help of GCP.
Building architecture and ETL flow:
With SAP HANA, the SAP Data Service is already build in so you can easily export data from SAP application or its underlying database to Google BQ . However, the combination of GCP and SAP is also possible for companies who are still living in the ERP world.
Some of the following options can be used for data extraction from ECC SAP to GCP :
·??????Extracting data via RFC (Remote Function Call) interface with data integration tools like talend/Google Dataflow/SAP BODS/Aecorsoft
·??????Writing an ABAP program
·??????3rd party connectors like five Tran
·??????Using SAP Replication Server
?SAP data can be moved from different ECC tables , SAP BW data targets like DSO, Info Cube, Bex Query, ADSO , SAP HANA views etc moved to GCP landing and raw zone through different integration patterns /tools like Talend , Aecorsoft , SAP BODS etc.
Organization can build the landing zone as per ?reporting requirement , security and compliance requirements and depending upon the workload/amount of data they want to ingest /move to GCP cloud for analytics purpose.
Once data is ingested and staged into landing and raw zone , you can apply/replicate the business logic into harmonization zone . Organizations are also replicating and building the same logic from existing data warehouse to GCP to gain the advantage of speed ,agility and scalability of GCP cloud platform. Slowly they are moving away from their old legacy data warehouse solutions which are not agile, not performance efficient and not scalable as per business growing data needs.
Post data harmonization organization can build KPI specific views in their product or publishing zone. These business KPIs can be finally published into your visualization layer where ultimately it will be consumed by end users.
Data transformation:
With GCP ?it is faster to transform SAP data and also easy to combine it with different NON SAP source system data. Lot of clients started realizing benefits in terms of faster completion of queries , faster delivery of analytics reports.
It is easy to combine data and explore the SAP and NON SAP KPIs through GCP.
领英推荐
Data is normally staged and mapped/moved 1:1 from all the SAP source systems into landing and raw buckets. As it is moving into harmonized or published layers it can be transformed as per business requirements and reporting needs.
Data transformation is also being necessary due to columnar based data warehousing tools like BQ as within SAP?there are different data types and data structures between the source and target systems. With GCP BQ it is easy to denormalise the data . Master data and transaction data can be easily combined with the help of data transfer service DTS in BQ.
Delta loads :
Different tools provides different methods to transfer the SAP delta records to GCP. Historical data can be staged as a one time as full load extraction but delta needs to be planned very carefully depending upon the integration pattern you are going to select.
With Aecorsoft ?the incremental?delta?records from SAP are merged into?Delta?Lake automatically by?AecorSoft?Data Integrator. You can also plan for superdelta with aecorsoft.
While the SAP Data Service comes already build in with a CDC wizard, other data integration approaches might need some extra work. To realize this step it’s important to get to know the SAP tables and table structure very well. Realizing the delta load on an own logic can be realized especially with the help of two tables:?CDHDR?(Changes Header) and?CDPOS?(Changes Position). These tables are tracking changes in master data or transactional data.
Data Visualization , Reporting and AI capabilities :
Google Looker Studio(Earlier known as Google Data studio) is a free, self-service business intelligence platform that lets users build and consume?data visualizations, dashboards, and reports. Looker Studio turns your?data?into fully customizable informative reports and dashboards that are easy to read and share. There are few SAP attributes like company code, controlling area can be used to restrict access of data within dashboard
Power BI, Tableau , React Js can be another set of data analytics and visualization tools choice for business reporting.
Dataproc is a Google Cloud product with Data Science/ML service for Spark and Hadoop. In comparison, Dataflow follows a batch and stream processing of data. It creates a new pipeline for data processing and resources produced or removed on-demand. Whereas Dataprep is UI-driven, scales on-demand and fully automated.
Summary:
GCP is definitely leading the analytics speed, data processing and reporting for SAP data. SAP ECC holding majority organization transactional data and master data that can be analyzed with GCP which is having many of data analytics , data science tools and services. While data from SAP systems can be ingested through GCP platform, harmonized and data products can be built on top of them , the newer SAP HANA systems can be also integrated and combined seamlessly with GCP architecture. These SAP data products can be harnessed , scaled with the help of GCP cloud platform. Bigquery for SAP has significant impact on data analytics in terms of query execution and delivery of analytics reports. GCP is definitely way forward and solution to go away from old legacy data warehouse solutions.
References :?
SAP Analytics Project Manager | SAP BW/4HANA/SAP HANA/SAC/Datasphere/Snowflake DWC | Ex-Accenture
2 年Great stuff Kiran..Thanks for sharing