NRB cuts response times and boosts disaster recovery planning

To read a French version of this case study, click here.

Founded in 1987 by a group of utilities companies and an insurance company, Network Research Belgium (NRB) has grown into a full-service company offering infrastructure, hosting, consulting, systems integration and applications development services. NRB employs 600 people generating annual revenues exceeding €130 million and claims to be one of the top five IT service providers in Belgium. Its main site, in Herstal, includes three fully autonomous data centers with a total of 1,500 square meters of usable space. Based on the resilience of its services and management techniques, NRB has held the ISO 9001 certification since 2004.

NRB hosts several large customers from several industries (insurance, utilities, government), who run their core applications on two IBM zEnterprise® 196 mainframes. Typically, these companies have statutory obligations to their industry auditors to demonstrate effective business continuity and disaster recovery (DR) planning, including the successful execution of a full DR test each year.

As the existing storage for the mainframe environment - two mirrored EMC Symmetrix arrays - approached the end of support, there were growing needs for enhanced performance and availability. DR procedures were overly complex and dependent on manual input. Equally, DR tests were over-running by a number of hours in some cases.

Henri Arnold, head of Mainframe Services for the Managed Operations division at NRB, says: “As we pursue our strategic goal of becoming the number one IT service provider in Belgium, we face strong competition on pricing. To keep our prices competitive while simultaneously maintaining our reputation as a highly responsive and flexible provider, we need to control our internal costs, in part by automating our processes. Our existing storage landscape was not as automated and industrialized as we wanted it to be, particularly when it came to DR procedures, which are a vitally important topic for our customers.”

Superior solution
There has been significant focus in recent years on rising energy costs in the data center, but the fact remains that these costs are typically dwarfed by personnel costs. To enable more competitive pricing to customers while maintaining profitability, NRB is trying to free up its personnel from routine administration. By simplifying systems management and enabling greater automation, the company can ensure that its skilled technical staff are free to focus on optimizing customer environments and providing new added-value solutions.

In the mainframe disk storage environment, NRB needed a general improvement in manageability and reliability, as well as the specific enablement of disk swaps, as Henri Arnold explains: “At that time, EMC was unable to support the ability to swap disks between the production and backup environments. This meant that we couldn’t easily or reliably point the production z196 at the disks on the backup storage system if we had a problem with the production storage system.”

NRB reviewed offerings from EMC and IBM, and IBM Premier Business Partner I.R.I.S. Group provided a demonstration of the IBM System Storage DS8800 array, including a simulated benchmark to demonstrate its superiority over the offering from EMC.

“Our account manager at I.R.I.S. has an excellent knowledge of the mainframe and was able to put us in contact with the right technical experts in IBM,” says Henri Arnold. “The analysis provided by I.R.I.S. showed us that the DS8800 would be the best solution, offering better integration with IBM z/OS and greater ease of use. In fact, we already knew that the IBM storage management tools were superior, based on our experience of deploying two DS8700 systems for a client.”

Transparent migration and swapping
NRB worked with I.R.I.S. and IBM Global Technology Services to implement two IBM System Storage DS8800 arrays and to migrate data from the existing EMC Symmetrix arrays. The initial data migration was facilitated by IBM Data Mobility Services Transparent Data Migration Facility for z/OS software, which allowed data to be moved completely transparently without halting production or causing any other business disruption. Some 23 TB of data were migrated to the primary DS8800, of which 17 TB is synchronously replicated to the secondary DS8800 using IBM Metro Mirror and 6 TB is non-mirrored. The primary DS8800 has 31 TB total disk capacity while the secondary DS8800 has 22 TB total capacity. As part of the migration, the two IBM DS8800 arrays were moved from being 400 m apart on the same site to two different sites 30 km apart.

“With the new solution, we can now switch from using disks in the primary storage system to the secondary storage system in just four minutes, versus four hours in the old environment,” says Henri Arnold. “This means that we can almost immediately restore production on the primary mainframe if the primary storage system suffers some kind of failure. What’s more, the speed and reliability of the switchover make it easy to simulate a DR scenario for testing purposes.”

NRB wishes to use IBM Basic HyperSwap® in the future, an entitled copy services solution for z/OS, to enable even higher availability in the event of a disk storage system failure. This solution will enable the production z196 mainframe instantly to use disks located on the DS8800 in the second site, 30 km distant.

Full protection for data
To further simplify DR testing, NRB physically isolates customer environments from each other by setting up separate LUNs on the DS8800 systems. This makes it easy to run DR tests for one customer without affecting any of the others.

“HyperSwap gives us a much simpler way to swap disks, requiring us to write just a small script for each environment, linked to a command for executing it,” says Henri Arnold. “If the economics are right for our customers, we have the possibility of adopting a fully automated variant, in which the mainframe itself can swap disks if it detects a problem.”

The use of Metro Mirror for synchronous data replication between the two NRB sites provides complete protection in the event of a failure of the entire primary site. In this scenario, NRB can simply perform an Initial Program Load (IPL) on the z196 mainframe in the secondary site and restart production using the replicated data on the secondary DS8800. This entire process is designed to take less than two hours to complete, which is well within the requirements of NRB’s customers.

In the first full disaster recovery test, NRB saw clear improvements with the new IBM systems. The time taken to complete the recovery was five minutes, compared to one hour 30 minutes for the previous solution.

Faster transactions
Replacing its EMC Symmetrix arrays with IBM DS8800 systems enabled NRB to achieve its primary objective of simplifying and strengthening DR procedures. It also delivered significant performance benefits, as Henri Arnold explains: “Thanks to the intelligent caching algorithms used by the DS8800, disk access time has been cut by more than 50 percent, from 1.2 milliseconds to 0.5 milliseconds, with an impressive effect on response times for transactions. In a mainframe environment, where some customer systems are handling four million transactions per hour, that’s a huge advantage. The performance is all the more impressive when you consider that this data is also being synchronously replicated over 30 km.”

He adds, “There is also a general performance increase with IBM DB2®, because the DS8800 is optimized for IBM software. There are DB2 statements that run faster on the DS8800 than they would on a different vendor’s array, and that’s a fundamental benefit for our customers.”

Simplified management
The IBM storage management tools provided the DS8800 systems are helping NRB to automate and accelerate management tasks, freeing up technicians to focus on fine-tuning customer environments. Using IBM System Storage FlashCopy® enables instantaneous point-in-time copies to be made, eliminating the need to interrupt applications to take backups or to clone systems for development work.

“The tools for configuring the storage environment are much better than those we previously used,” says Henri Arnold. “For example, we offer volume-based pricing, and we allow some customers to manage things like adding new disk capacity. In the past, we couldn’t really control or monitor that activity, which was unsatisfactory. The IBM solution gives us the transparency we want while leaving the customers in control of their own environments.”

A further benefit of the IBM solution is that most maintenance operations can be handled by NRB without requiring an IBM engineer to visit. This was not the case for the previous solution from EMC, where tasks such as microcode updates might require a three-hour halt to production and a costly engineer call-out.

“With the IBM storage solution, we are completely autonomous and can make configuration changes faster and without disruption,” says Henri Arnold. “The acquisition cost for the IBM solution was lower than for the EMC option, and we knew also it would be less costly to manage and operate. Taking into account the reduced management requirements, we estimate that the IBM System Storage DS8800 has cut our mainframe storage costs by 25 percent, while delivering better transactional performance and enabling superior flexibility, speed and reliability for disaster recovery procedures. For our customers, that means we can offer an even more responsive and reliable service while maintaining our competitive pricing.”