Published on 09-Dec-2008
Validated on 01 Oct 2012
"Flexible script customization provided by IBM Rational Functional Tester helped us deliver a powerful test automation solution." - Dr. Sivakumar, CTO, Astra Infotech Pvt Ltd
Astra Infotech Pvt Ltd
Change & Release Management, Development & Technology Adoption, Test Data Management
IBM Business Partner:
This technical solution brief describes how Astra Infotech successfully implemented IBM® Rational® Functional Tester for automated testing in a customer’s SAP® application environment. It will show how IBM testing tools can be used in combination with change and defect tracking to ensure quality in any customized SAP application landscape.
Reduce testing workload and cycle time to fit the needs of a fortnightly release schedule. Improve consistency and comprehensiveness of test coverage
Work with Astra Infotech to convert manual testing documentation and processes into automated scripts. Deploy IBM Rational ClearQuest® to provide change management and defect tracking, while simplifying collaboration for the development and testing teams. Leverage IBM Rational Functional Tester to automate and accelerate the testing process
Testing cycle time has been reduced from three days to just two hours per fortnight. The solution runs all tests automatically, with no need for manual intervention – so the development team can focus on other priorities. The cost of automated testing is incrementally reduced with each cycle – assuring return on investment after a certain number of cycles. The introduction of a testing framework with reusable components will make it quicker and cheaper to create new test scripts in future.
Background, starting point and objectives
Astra Infotech Private Limited is a software engineering company and an Advanced IBM Business Partner in India and Europe. The company is a leader and evangelist for leading-edge software engineering practices, and offers the following services:
- Software engineering consulting
- Software testing
- Software engineering training
- Software engineering tools
Astra Infotech has a small but highly skilled team. The company has been active in the industry for more than 15 years, and the team has collective experience of more than 100 years, spanning a variety of applications, organizations and verticals.
Astra Infotech needed to provide a test automation solution for a customer who is a SAP Partner and systems integrator, specializing in implementing solutions using SAP NetWeaver Mobile. The customer was looking to automate integrated testing of customized mobile SAP implementations involving SAP NetWeaver Portal, SAP NetWeaver Mobile, and back-end SAP applications. The major requirement was to reduce the cycle time for regression testing.
Astra Infotech was involved in tool identification, test automation planning, script recording, and support scripting.
The customer’s typical release schedule involved at least a three-day manual test cycle, and releases were generally made on a fortnightly basis. The customer expected that automation would reduce the cycle time to just a few hours of unattended testing, significantly reducing workload and turnaround time for each release and ensuring that testing would be performed consistently during every release. Automated testing would also ensure that the coverage was consistent, which was another major challenge with the existing testing process.
The customer already had an established quality assurance/quality control process, with test cases documented in Microsoft Word. The Word documents specified the actions to be performed, the context of these actions, and the required verifications.
However, since the customer was not yet familiar with automated testing tools, these test cases had to be adapted for automation and coded as test scripts.
The ability to provide a complete and integrated solution for testing and change and defect management was one of the major reasons for choosing the IBM solution. The flexibility in the scripting language also supported the decision significantly. Since the customer is new to test automation, the on-site support from the Astra Infotech also influenced the decision.
Overview of applied technologies
The solution involved IBM Rational ClearQuest for test and defect management, and IBM Rational Functional Tester for test automation.
IBM Rational ClearQuest offers comprehensive software change management. It provides defect tracking, process automation, reporting and lifecycle traceability for better visibility and control of the software development lifecycle. ClearQuest’s automated workflows and e-mail notifications enhance team communication and coordination. Test management unifies development and testing activities, from planning to results, for improved software quality.
Rational Functional Tester
IBM Rational Functional Tester is an advanced automated functional and regression testing tool for testers and GUI developers who need superior control for testing Java, Microsoft® Visual Studio .NET and Web-based applications.
Rational Functional Tester records user interactions with Java, Web, and Visual Studio .NET WinForm-based applications, creating a test script that, when executed, reproduces those actions. During recording, the user can insert verification points that extract specified data or properties from the application that is being tested. During playback, these verification points are used to compare recorded information with live information to ensure consistency.
Following any test recording activity, testers have the option of adding custom code to the test script to perform an unlimited array of tasks, including the data manipulation and environment configuration activities that are often necessary to ensure the test lab is properly constituted for the test run.
After the execution of a test run, Rational Functional Tester generates a report listing the results of the verification point comparisons. With Rational Functional Tester, teams are able to expose problems more reliably and efficiently in even the most complex applications, making it easier to capture and repair defects before the product is released.
Detailed description of solution implementation
Astra Infotech adopted a business-driven, iterative approach to the software test automation project, balancing stakeholder concerns and priorities with short feedback cycles and continuous risk mitigation. This approach is similar to the iterative approach recommended by the IBM Rational Unified Process (RUP), a customizable and scalable process framework.
The Initiation phase involved gaining an initial understanding of the project and planning the overall work.
The major phases of the project were Inception, Elaboration, Construction and Transition. Each of these phases involved one or more iterations, with an executable release at the end of each iteration.
The Inception and Elaboration iterations comprised planning and design activities. In these phases, the scope of testing, the test plan and the identification of test cases and data were determined. The highest-risk test cases were implemented progressively.
The Construction and Transition iterations comprised the realization activities. In these phases, the remaining test cases were implemented.
The Final Assessment phase then assessed how effectively the work had been done.
Inception and Elaboration phases
Astra Infotech and the customer worked together to identify a medium-sized project to assess the effectiveness of IBM Rational Functional Tester. IBM Rational ClearQuest was installed for test management and IBM Rational Functional Tester for scripting.
The project was intended to confirm the effectiveness and resilience of scripting using IBM Rational Functional Tester by performing integrated testing with a typical scenario for SAP NetWeaver Portal, SAP NetWeaver Mobile and back-end SAP applications. Test plans, test cases and test suites were created in IBM Rational ClearQuest, and integrated with scripts in IBM Rational Functional Tester.
This project confirmed the effectiveness of the proposed approach, so the customer and Astra Infotech decided to proceed to the next phases.
Construction and Transition phases
Test cases were analyzed for test automation, and scripts were recorded. The key issues observed were:
- Most of the scripts needed to work with the same screen and the same set of GUI controls. Time was saved by using shared object maps for all of these scripts
- Modular scripts were used to handle variations in navigational sequences
- Navigational sequences were isolated as user action scripts
- The built-in data pool capability was used to parameterize test data
- Application-specific exception handling was established by abstracting similar exception handling capabilities into helper super scripts
- Some of the data to be verified was embedded as tabular data in an HTML table. Special purpose data parsing code was written to extract the required data
- Common portions of scripting code were pulled into modular, reusable libraries
- Usage scenarios were implemented using test suites and shell scripts
- IBM Rational ClearQuest was configured with a special-purpose defect submission form and email notification
At the start of the project, it was observed that GUI controls were identified by number rather than name. These identification numbers were dynamically generated, and changed in every build. This affected the resilience of the test scripts. To avoid these issues in future, the application was modified to give explicit names to the GUI controls.
The solution took three months to implement, during which time the customer issued three releases of the application. This meant that the scripts were tested with all three releases, which helped in assessing their resilience. Except in cases where there were drastic changes (for example, if a whole new set of GUI controls was added), the resilience of the scripts was found to be more than adequate.
Rational Functional Tester enabled the company to reduce its three-day manual testing cycle to just two hours of unattended automated testing for each new customization release of its SAP application-based solution. The reduction in cycle time and workload helps the team spend more time on developing new features and less on testing – so new functionalities can be implemented and released more quickly.
The customer is now planning to use the same approach and solution across the whole organization. Astra Infotech is supporting the customer by helping to define a testing framework with re-usable components that can be used across all projects. The framework will drastically reduce the upfront investment for script generation for various projects.
With manual testing, the workload increases with each test cycle. By contrast, in the automated testing environment provided by Rational ClearQuest and Rational Functional Tester, once the initial investment of time and effort has been made, workload is reduced with each cycle until a baseline is reached. As a result, after a certain number of cycles, return on investment is assured.
Products and services used
IBM products and services that were used in this case study.
© Copyright IBM Corp. 2008 All Rights Reserved. IBM Deutschland GmbH D-70548 Stuttgart ibm.com Produced in Germany IBM, the IBM logo, ibm.com and Rational are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. A current list of other IBM trademarks is available on the Web at “Copyright and trademark information” at: http://www.ibm.com/legal/copytrade.shtml Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Other company, product or service names may be trademarks, or service marks of others. This brochure illustrates how IBM customers may be using IBM and/or IBM Business Partner technologies/services. Many factors have contributed to the results and benefits described. IBM does not guarantee comparable results. All information contained herein was provided by the featured customer/s and/or IBM Business Partner/s. IBM does not attest to its accuracy. All customer examples cited represent how some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions. This publication is for general guidance only. Photographs may show design models.