SAP on Cloud: Demystifying the Performance Testing Saga

SAP Performance on Cloud

SAP migrations to hyper-scalers be it homogeneous or heterogenous are a known stuff today in the SAP world. We focus by & large on the optimization techniques for the migration per se, as that's what determines the least amount of business downtime to our clients. What often however is not widely optimized is the performance and the best way to optimize the performance for SAP solutions is via the route of performance, stress and volume testing.

Although we call it hyper-scalers and yes its infinitely scalable, but every individual components - be it compute, disks, NIC's, etc. have their limits defined. So its up to you to broaden the limits and scale-up/out during the performance testing to tune the limits based on customer requirements.

Whilst for a use case where the customer SAP estate already exists, you size the new inventory on cloud by matching the SAPS and architecture/non-functional requirements, it still doesn't guarantee that the performance of all key SAP Business Processes, Transactions, Reports, etc. would be same/better. And, in certain cases it could very well be counterintuitive even though you're on the latest & greatest hyper-scaler infrastructure, CPU models and chipsets. And, certain key transactions can actually run slower - well, that's SAP for you! But of course you can tune & re-tune it through a robust performance testing framework!

Performance Testing Objective & Simulation

1. Verify that System Performance is acceptable under simulated high load with extra headroom to meet expected business volume (focus is not on individual transaction's performance testing)
2. High-Load = Peak Concurrent Users + Critical & Complex SAP T-codes
3. It is not possible to script all transactions therefore, the highest impact transactions and processes from the analysis will be selected for automation; ideally this will represent 80% of the production load however the spread of transactions varies from system to system. In order to replicate the load of the lower impact transactions the frequency of scripted transactions will be increased.
4. Take the batch job schedule/history for the peak hour from system records, in order to simplify the task compare this information to the performance statistics in order to remove jobs that run infrequently and do not produce significant load on the system.
5. Take the interface volumes from the peak hour from middleware system records, in order to simplify the task compare this information to the performance statistics in order to remove jobs that run infrequently and do not produce significant load on the system.
6. Front-end, interface and batch parallel schedules should be considered. And should mimic production scenarios

Performance Test Model

Testing Scope & Plan

1. Technical Teams - Most used T-codes, DB/Resource intensive T-codes
2. Business Inputs - Additional business critical T-codes. Important to prioritize (Tier-1, Tier-2, etc.)
3. Agree on measurement KPI's and acceptance criteria. In general the acceptance criteria specify the areas in which the system under a simulated load will not show any performance degradation from the simulated baselined load.

Test Tooling and Scripting

1. Write/Re-use Test Scripts
2. Test Data Preparation with inputs from Business
3. Ensure to set-up the test automation tools (Microfocus Loadrunner, etc.) including the requirement infrastructure and licenses

Test Execution

1. Create Baseline for On-Premise Systems - gradually scale up the virtual concurrent users , Store the baseline test results
2. On-Cloud Performance Testing - run scripts for selected critical t-codes and gradually scale-up the virtual concurrent users.
3. Monitor Systems, compare, remediate and repeat performance testing on cloud
4. Run endurance tests over longer periods

Analysis & Reporting

1. Analysis of infrastructure & DB KPI's
2. Analysis of T-codes performance
3. Fix issues/bottlenecks
4. Apply fixes to infrastructure, DB and SAP
5. Verify results against agreed & SAP recommended targets
6. Repeat test or Final Sign-off