Telltale signs your SAP deployment can benefit from Snowflake Data Cloud

Executive summary: Complexity builds up over time as companies attempt to resolve tactical problems in their analytics stacks. Once in a while, new technologies come along that can compress and massively simplify processes at economies of scale. Snowflake is one of these technologies that provides massive operational efficiencies, growth to your business while reducing risk and time to market. This article takes you through an SAP customer's story, from historic complexity, to business value with Snowflake, to more technical topics like extraction and ingestion, modelling, and security to a high-level target architecture.

Hey everyone, what are the telltale signs that your analytics environment has become too complicated, siloed and expensive? Are you still working weekends loading data or performing patches and upgrades?

Are your business partners complaining (more than usual) about long running queries? Do you have an SAP BW 7.x system and are dreading taking the jump to BW4/HANA because of licensing, sizing and migration effort to convert to the new BW objects (ADSO, etc) and lose your trusty Bex Analyzer tools?

Are you feeding data from SAP BW to an Enterprise Data Warehouse? Did someone start a data lake as well? Are business units just using Bex or BusinessObjects to extract data out of BW so they can put it into Excel and then push it into some other tool? Does your environment look something like this?

How did it become so complex? When you started, it was a lot simpler. BW was pretty good for SAP ERP and CRM. But then President Bill (not Clinton) went and bought SuccessFactors, Fieldglass, Concur and Ariba. Each had and has its own analytics stack. It was actually easier to put this business network data in an SQL database instead of conforming it into BW.  

And then the variety, volume and velocity of sensor, social and signal (the Nielsens of the world) data made it too expensive to put all the data in BW or the EDW, so a data lake was the cheapest option. Except no one looked at the price of fishing it out again, especially if the lake was frozen. 

Why the data marts? Tactical. To take the load off of BW and improve performance and reduce concurrency (read out of memory errors here and crashing).

Local databases grew like icicles as local markets built their own analytic solutions. They responded to immediate business needs, but no one supported them--the danger of an icicle database breaking off and injuring was too great.

Moving and hosting legacy SAP systems in the cloud will certainly shift some of the infrastructure and software operational workload to someone else, but you still pay for the complexity and the fragility of your data flows through increased costs and monitoring. Not to mention you now have vendor lock-in from SAP and your cloud provider.

Snowflake Business Value

To start, please simplify your landscape to respond to business needs and reduce capex and opex. Make your data accessible and actionable by putting it all in the same place. Make it FAIR (Findable, Accessible, Interoperable, Reusable).

This centering of data provides the data foundation for 360 customer, product, supply chain, and so on, in addition to machine learning. It is not virtualisation which makes your supply chain of data dependent on the weakest link. This is robust.

Snowflake provides share-what-you-want-when-you-want-with-who-you-want cloud computing with instant elasticity, scalability, isolation and a pay-for-what-you use pricing structure.

The first step of the 360 vision is getting the data off the SAP systems and sharing it in Snowflake mixed with real-life data (by the way, Strava is a customer of Snowflake).

Extraction and Ingestion

To achieve a common data foundation more simply than before and at a massively lower cost you need a cloud provider agnostic solution that can ingest data from your SAP operations systems, both on-premise and cloud, 3rd-party systems, and signal data. A solutionat that can ingest the data natively, whether it’s in structured or semi-structured formats. 

The solution must robustly ingest even if the data structure in the files changes. Check out our type VARIANT. Ingest into a structured, governable, easy to access and open solution--where the data can then be accessed immediately by common open interfaces such as ODBC, JDBC, Python, and Spark.

You will want storage managed automatically in terms of capacity, compression, statistics, performance. You don’t want to build indexes or do housekeeping. You want computational power that hits against the storage that is elastic and isolated from other compute clusters used by other teams. 

You want a solution that can handle volume, velocity and variety, where you can load and report at the same time. A system that has no down-time. No four-hour window every week for upgrades. No system shutdown to increase or decrease compute. 

The solution is Snowflake. Just to wax your skis, there is no resource contention in Snowflake. The nightly ETL that got longer and longer, and that elusive batch window before the first analytical user throws the first snowball becomes a thing of the past. That risk to your financial close, those sleepless nights during year end close become a thing of the past. You will sleep better and have time for other things.

But how is this different from the marketing coming out of the competition? Similar marketing Strap on the poles and skis and go ski (start for free), or talk to some of our more than 4,000 customers. Frictionlessly share data to or from those customers (check out our marketplace) through access controls, not file transfer.

We have 3 extraction patterns for classic SAP ECC, SAP S4/HANA, SAP BW, ones that either replicate the base tables from SAP or go through the application layer. 

You can use standard SAP technology such as BW extractors, SLT, ODP/ODQ, SAP Data Services, or OpenHub to pull this data. Or you can use a wide variety of partner solutions that handle pool and cluster tables, replication, differential loads and what not. I’ve seen SAP Data Services and Datavard in operation at my customers. 

Other partner solutions include HVR and Qlik Replicate. Five-tran, on our partner connect site, includes an S/4HANA connector. I tend to favour solutions that leverage the SAP stack. If you don’t want to go it alone and want an SI that can handle the migration end to end I suggest you take a look at LTI and their Polar Sled solution.

Don’t want to hit any trees? Verify your access usage licensing by checking out this document from SAP on indirect access usage.

Modelling

SAP BW is a data warehousing solution with lots of built-in tools including modelling and planning and consolidation, process chains and the like. Snowflake works differently, so most customers do not immediately replace BW. They leverage BW with Snowflake.

They provide quick business value by offloading from SAP BW corporate data and combine it with 3rd party or sensor data in Snowflake. Then in a second step they streamline their data pipeline by hitting directly against SAP ECC or S/4HANA.

Using BW to offload the data means the modelling of the SAP data has already been done before it goes into Snowflake. Then it is simply a matter of re-joining the facts and dimensions. It looks as simple as what this customer did, rebuilding their LSA++ model in Snowflake:

Business accepts this and appreciates that they can now easily combine the Financials, Sales, AR/AP, HR, etc., with other third-party data sets coming from our marketplace, such as from FactSet. This is a quick win for IT. There are over 100 data providers on our marketplace (Finance, Health, Demographics, Weather, etc.).

And with Snowflake business can now use their BI tool of choice, such as Tableau or Power BI or Web Intelligence or any tool that uses a JDBC or ODBC connection. Before, hitting against BW using MDX was like skiing on rocks (possible but dead slow). See customer Serge Gerkovich’s blog here on the improvements brought by using Snowflake over BW.

But, as a step two, how to get data directly from the ECC and S/4HANA and into Snowflake. And how to model it? 

Snowflake supports all data models, from third-normal form, to dimensional star-schema, Snowflake, and data vault. 

Here is an example of modelling SAP in data vault. Different and adjusting levels of massive parallel processing for the ingestion piece, the data vault side with satellites and hubs to structure at the lowest levels of granularity that allows you to incrementally build up and iterate your model, the business vault on the other side. Views for the business, and so on.

For more information on data vaulting check out data warrior Kent Graziano’s blog.

But let’s call a snow shovel a snow shovel. Snowflake’s modelling is SQL DML and you’ll eventually want to leverage a modelling tool once you move from SAP BW. 

There are modelling tools out there that will help with your SAP data, such as Qlik Compose and Wherescape. Other more generic modelling tools include SQLDBM. SAP PowerDesigner also has Snowflake connectivity. Check out DBT and Dataops for Snowflake from Datalytyx for devops.

Feeling alone in the vast expanse of it all? I suggest you first leverage what you know from BW (models, security, LSA), then move with a full glide to much more flexibility, elasticity and sharing. This is good for you and your company. You’ll see things you’ve never seen before and provide more value at a lower price.

So why Snowflake over SAP HANA? Price, elasticity, data sharing, isolated compute, variety and volume. Plus Snowflakers don’t have to partition, and don’t have to upgrade software.

With Snowflake you can run cloud agnostic, avoid the vendor lock-in of the cloud providers. But what about vendor lock-in to Snowflake? Although you will find the Snowflake sticky (very good for snowballs and snow persons), your data is and will always be your own and you can easily egress it. That’s what working with open-standards is about. 

Security

What’s more secure, copying data from 80 SAP ECC systems to six regional BW systems and then copying a subset of that data to one global BW system, or moving the data once into Snowflake and then sharing it, either through Role based Access Control (RBAC) or data sharing?

Our largest customers have one Snowflake account and within it they have their EDW space, their country market space, their sensor data space and so on. The data is landed once and shared out. One copy, one security model, hundreds of elastic compute clusters, with monitors on each one of them. Within the same account, they also have their Test, Development and Production environments.

Local markets, instead of using FTP to transfer their data between systems, simply open access of 200 or so views to corporate, and their reporting is done. No latency, no auditing, one governance.

What about SAP Analytic Privileges? Behind the covers, it’s no more than hidden constraints in your where clauses. A combination of RBAC and row-level security constraints do the trick.

Final Leg - Target Architecture

With 2km to go to get back to the lodge, what does success look like? Snowflake can simplify and take over your data lake workflow, your enterprise data warehouse workflow, your SAP BW workflows, and your local databases as well. It's iterative, start small, take the small loop first, then larger.

You can bring your data back under governance and provide the SLAs to the business that were promised. Here is what success could look like:

Thanks for your time. Special thanks to guest star skier, Corinne!