Regence BlueCross BlueShield of Oregon halves testing time

Regence BlueCross BlueShield of Oregon (Regence) is one of the largest health insurers in the northwest United States, serving Washington, Oregon, Idaho and Utah. A nonprofit organization, Regence provides individual and group medical; dental, vision and life insurance; Medicare and other government programs; and pharmacy benefit management to its 2.5 million members.

Reaching the limits of manual testing
Regence had reached the limits of its manual testing processes. The company’s QA team was having difficulty keeping up with testing requirements for the company’s websites and portals, which are used by its members and providers. Every two weeks, the company releases updates to its websites and portals, but it was taking three full-time employees approximately two weeks to run the 1,250 tests needed to ensure that the portals delivered the necessary functionality. “The testing team was up against a wall,” says Randy Pellegrini, software quality engineer, Regence. “We could bring in more people to do manual testing, we could offshore our QA, or we could actually step across the divide and start automating our testing.”

Pellegrini was assigned to spearhead the transition to automated testing, along with An Doan, James Eisenhauer, and Shelley Kleinsmith, all software quality engineers. The team came up with a plan to transition Regence from a manual testing organization to an automated testing organization. They selected IBM Rational® Functional Tester,IBM Rational Performance Tester and IBM Rational ClearQuest® Test Manager software to form the basis of the new testing automation solution. After initially considering using open source tools, they decided that it was too risky to use tools without support. Plus, the company already had other IBM Rational applications in place, so it made sense to stick with what worked.

Using agile methodologies to implement automation
The team decided to approach the automation project like a software development project. So they relied on agile methodologies, creating an agile scrum team for automation including employees from a number of different departments within Regence. Spearheaded by Kleinsmith, the agile scrum team enables employees to share information about the Rational tools and learn from each other’s experience and mistakes. As part of the agile process, the team holds weekly demos and planning sessions to break the automation project down into manageable chunks and keep it on track. They also create standards for coding, execution and reporting, to assist other groups within the company working on automating their test efforts.

Communicating success to gain buy-in
The automation effort began as a pilot project, but has consistently grown. In March 2010, the company had three members on its agile scrum team that automated 15 tests for one project. In February of 2011, the company had 13 members on the team automating 45 functional tests across eight projects. And the use of the Rational tools has continued to expand as the team has continually communicated their successes to the rest of the company.

Doan built an automation framework for Regence using the Rational tools including a website built by Eisenhauer that queries the Rational ClearQuest Test Manager application and reports test results via the company’s intranet. Eisenhauser also created a service that emails stakeholders a daily snapshot of the day’s testing results. Stakeholders are therefore able to see on a daily basis what the testing automation has accomplished.

Quickly achieving impressive results
In the past, it took three full-time employees two weeks to run 1,250 tests on the company’s website. “Using the Rational tools to automate testing, three employees can run the required 1,250 tests in less than one week, essentially reducing testing time by 50 percent,” says Doan. Plus, the Rational tools have helped improve testing accuracy and expand the scope of testing, enabling the company to test features that would be difficult to test manually. For example, one feature that required testing has 1,000 test data sets. In the past, it took testers five minutes to run and log each set manually, making a total of 80 hours of work. Using the Rational applications to perform automated testing takes just eight hours to complete.

But the most significant aspect of the automation is that it requires no user interaction whatsoever, so the team therefore saves 80 hours of work. In addition, Regence has improved the quality of its website and portals. In the past, the company would have to release emergency patches to fix bugs that had been missed in the testing process. “We haven’t had to do an emergency release in months,” says Doan. “Using the Rational tools, our code quality has improved.”

For more information
To learn more about how IBM Rational software can help you automate testing, contact your IBM sales representative or IBM Business Partner, or visit: ibm.com/rational

Additionally, financing solutions from IBM Global Financing can enable effective cash management, protection from technology obsolescence, improved total cost of ownership and return on investment. Also, our Global Asset Recovery Services help address environmental concerns with new, more energy-efficient solutions. For more information on IBM Global Financing, visit: ibm.com/financing