Load testing SAP ABAP Web Dynpro applications with IBM Rational Performance Tester

About this paper

This technical solution briefly describes how the SAP CoE / Value Prototyping successfully leveraged IBM Rational Performance Tester 8.0 to test an ABAP Web Dynpro application before it went into production. The paper shows how IBM testing tools can be used to simulate user load on any SAP system and measure the system’s behavior under load. The solution described in this paper was used in an SAP internal project to measure a new SAP application before it was implemented internally.


Why performance testing?
SAP business processes rely on two fundamental elements – functionality and performance. Functionality is what the SAP application enables its users to accomplish, while performance is the system’s ability to complete transactions and furnish information rapidly and accurately.

Measuring the performance of a system manually is time-consuming, labor-intensive, and can be both inaccurate and very expensive. Automation of performance testing helps to significantly reduce the time required to complete tests, increase the quality of the results, and reduce both costs and resource requirements. By using pre-deployment performance testing, it is possible to prevent application failure resulting from performance-related problems.


Lowering the cost of performance testing
IBM Rational Performance Tester generates low processor and memory footprint when emulating multiple users. As a result, high levels of scalability can be achieved even if the team does not have access to large amounts of computing power. In addition, the application features automated data correlation and manipulation, code-free tests and an intuitive user interface that helps reduce the learning curve and minimize the amount of training needed to make the testing team productive.

More efficient test automation means lowering the total costs of a scalable test project.


The SAP CoE / Value Prototyping
SAP’s Value Prototyping builds prototypes for all SAP solutions in a lab in order to allow customers and partners to validate the most complex scenarios – because “seeing is better than believing.”

To meet the growing demand for proving the scalability of its prototypes, SAP Value Prototyping established its High Performance Lab in 2005 to deliver this key service.
The High Performance Lab is fully equipped to conduct load, stress and scalability tests for both simple and very complex SAP landscapes on high-end computers hosted in the Lab’s data centre in Walldorf.

The High Performance Lab’s staff are experts in performance test design, test automation, sizing and performance analysis, system tuning, large data simulation, high-end system administration, and overall management of large performance test projects. Because of its unique capabilities, the Lab can provide a complete end-to-end testing service: ranging from the first customer workshops in which performance KPIs are defined, through to the final presentation sessions, in which the optimum performance capabilities of the solution are readily demonstrated.


The project
This technical paper describes a performance test of a new SAP internal application based on ABAP Web Dynpro. All tests were orchestrated with IBM Rational Performance Tester in the SAP High Performance Lab in Walldorf, Germany.

Performance testing of SAP systems and applications is critical when new applications or modules are developed or deployed. Similarly, if existing applications need to be customized, a performance test ensures proper system behavior in all phases of operation.

While SAP systems typically offer reliable performance, the combination of both SAP and non-SAP modules and other technologies within a portal can make system performance projection more difficult. This can lead to a higher risk of failure for SAP application deployment. Rational Performance Tester can help to minimize the risk before any critical business processes are deployed to a production environment.

SAP has many years’ experience of application performance testing. Nevertheless, the Labs are constantly dealing with new technologies that challenge established methods and tools. Given the current need for cost awareness, SAP decided to assess new options in this performance test, and evaluated IBM Rational Performance Tester 8.0.

IBM offered additional technical resources to support the project in order to minimize the risk of project failure due to lack of experience with the new test tool.


Overview of applied technologies / processes
IBM Rational Performance Tester 8.0 enables organizations to test the scalability and performance of business processes before they go live. Rational Performance Tester combines an easy-to-use test recorder with advanced scheduling, real-time reporting, automated data variation and a highly scalable execution engine to help ensure that SAP applications are prepared to handle large user loads.

IBM Rational Performance Tester helps organizations to maximize the performance and scalability of business-critical enterprise software through rigorous load testing before it is deployed into a production environment. It allows test teams to create, execute and analyze performance tests to validate the scalability and reliability of an SAP application landscape.

With IBM Rational Performance Tester software, organizations can better support the implementation of high-performance business processes and can execute, upgrade or modify existing processes on time and within budget.


Key features of IBM Rational Performance Tester:

• Identifies the presence and cause of system performance bottlenecks
• Creates code-free tests quickly, without the need for programming knowledge
• Provides a rich, tree-based test editor that delivers both high-level and detailed views of tests
• Automates test data variation and enables insertion of custom Java code for flexible test customization
• Automates identification and management of dynamic server responses
• Offers flexible modeling and emulation of diverse user populations
• Enables Windows, Linux and mainframe technology-based test execution
• Creates reports in real time to enable immediate recognition of performance problems and renders an HTML browser-like view of Web pages in the test
• Collects and integrates server resource data with real-time application performance data
• Offers Windows and Linux software-based user interface
• Minimizes the memory and processor footprint, enabling large multi-user tests with limited hardware resources
• Supports load testing against a broad range of applications such as HTTP, SAP, Siebel, SIP, TCP Socket and Citrix.

Test planning and design phase
A performance test usually includes the following five steps:

Define the business process: This step involves determining the business process that needs to be tested, the test data and the optimal hardware platform for the test. Setting up the Rational Performance Tester environment is the final preparation step.

Develop and record the test script: This step walks the software through the actual screens and entries that will be used by the business process, resulting in a test that exposes underlying details, including screen shots of the recorded transaction.

Expand the user load and vary data: This step simulates peak load requirements for the application or process, and identifies dynamic server responses. Test scripts, which include varied input data, can be grouped together in various combinations to reflect the multiple types of users comprising the projected user population.

Review and evaluate test results: This step includes measuring simulated performance against the key performance indicators for the process such as page throughput, server health and response time per transaction. Test execution is accompanied by easy-to-read, real-time reports that are updated throughout the test run and that include metrics on any bottlenecks.

Retest for validation: This step includes re-running the pre-recorded tests using the modified environment to validate the effectiveness of the changes.

In this project, all test cases were documented and agreed by the participating stakeholders: business process experts, developers and the performance test team in the design phase. This ensured a common understanding of the goals of the test project and the expected outcome.


Realization phase
Because of the thorough planning, the implementation of the performance test was quick and easy. IBM Rational Performance Tester helped to record all business scenarios. All dynamic data correlation was handled automatically and only a few adjustments to the recorded test had to be made. This was important because data correlation is critical and complex for SAP applications that use the ABAP Web Dynpro framework.

The framework creates dynamic identifiers for forms and fields. The identifiers are only valid for a single session and can be different for every form access. IBM Rational Performance Tester tracks those dependencies and ensures that the tests correctly capture the dynamic data from the server and update the requests at test runtime.

The visual test editor in IBM Rational Performance displays the content of a test in an easy-to-understand tree view, allowing even the business users to understand what the test is actually doing. During test execution, run-time monitors give real-time feedback about the progress of the test. The raw response time data is processed for quick analysis and easier reporting. Average response times across the whole test run or any range within the test run can be calculated and displayed in customizable reports. Additional data can be included, for example to show the correlation between user load and response time.

Page Performance Reports identify pages that are slow to respond, and mapping to the business process is easy because the reports use the same labeling as the test tree editor for the individual pages or steps of the business process. IBM Rational Performance Tester quickly pinpoints the performance bottlenecks and allows drill-down to the individual HTTP requests and their responses for detailed result and failure analysis.


Evaluation of the test results
Performance testing of SAP ABAP Web Dynpro applications provides some real challenges for load testing tools in general. First of all, the business processes that need to be tested are complex scenarios that traverse through many SAP screens. This results in long tests with hundreds of HTTP requests.

The challenge for a performance testing tool is to group these requests into meaningful “blocks” that represent screens or pages that have been visited by the user. The result is a readable and maintainable test that hides the complex HTTP traffic and relates the request/response communication to end-user activity.

Second, and this is especially true for ABAP Web Dynpro applications, the information that is returned by the server is very dynamic. For typical portal applications it is good enough to simply handle the dynamic session identifiers. By contrast, for ABAP Web Dynpro applications, IBM Rational Performance Tester needs to handle both the dynamic session identifiers and the dynamic user interface identifiers that the server generates for each user session.

The new and improved data correlation engine within IBM Rational Performance Tester is able to automatically correlate all of those identifiers, reducing the test development time to a single day.
It is best practice to reset an SAP system in a performance lab after each test, which is generally not possible in a production environment. Resetting ensures test reliability and reduces test development effort. However, in an SAP system, performance tests have to observe dynamic system behavior, such as increasing table sizes as well.

The Java extensibility interface offered by IBM Rational Performance Tester helps to handle additional complex user activity, such as scrolling through lists or searching for data. This powerful feature was key for solving the problem of the dynamic test environment. Java code can be inserted at any point of a performance test and can help to handle such complex situations.