Western Power Distribution accelerates data recovery with IBM and Tectrade

Western Power Distribution (WPD) delivers electricity to 1.4 million customers in South West England, and to 1.1 million customers in South and West Wales. The company employs approximately 2,300 people and maintains a transmission and distribution cable network stretching more than 82,000 km.

WPD’s tape-based backup solution was working well, and the organisation was confident in its ability to recover individual files during normal operations, or even the entire server infrastructure in the event of a disaster. However, speed of recovery was a major issue. Al Kellaway, Computer Manager at Western Power Distribution, explains: “To minimise the manual effort involved in handling tapes, we tended to store data from multiple servers on each tape. This created problems with media wait-times when we attempted to recover servers in a disaster-recovery test scenario. Essentially, a server would secure a tape for recovery, and the other servers needing that tape would have to wait their turn.”

The issue of tape contention meant that WPD could only recover three or four servers in parallel – pushing the full recovery time in a disaster out to as much as ten days. In addition to representing a business risk, the existing backup solution tied up valuable IT staff resources in tape management.

Cost-effective solution

Aiming to introduce a new backup solution that would be faster and easier to manage, WPD created a framework of requirements and invited a number of companies to bid for the work. “We had a natural preference for a solution incorporating IBM Tivoli Storage Manager, since we had used it successfully for around ten years – but it was by no means a foregone conclusion,” recalls Al Kellaway. “We wanted a cost-effective solution that would not require us to mirror the entire infrastructure to a second full data centre.”

The proposal from Tectrade, an IBM Premier Business Partner, met all of WPD’s criteria. Using its own WAN infrastructure, WPD dedicated 1GB/s of fibre-optic bandwidth to connect its main data centre in Plymouth to a new Tivoli Storage Manager environment at a remote site in Exeter. The new Tivoli environment, designed and implemented with the assistance of Tectrade, runs on an IBM System p5 570 server connected to an IBM System Storage DS4800 with 1TB of Fibre Channel and 20TB of SATA disks. Data is initially backed up to the high-performance Fibre Channel disks on the DS4800, then steadily migrated to the lower-cost SATA disks throughout the course of the day.

“We now back up all data from the primary computer suite in our Plymouth site – across the network to the remote site,” says Al Kellaway. “As a layer of insurance, we then take a reverse copy of the backup data to an IBM System Storage TS3310 tape library in our main site.”

He adds: “Tectrade managed the entire solution implementation, from technical consultancy and design through to the delivery and installation of the IBM hardware. Your view of a company is always based on the people you work with, and the Tectrade consultant was excellent. Indeed, we will continue to work with him as we plan our future capacity requirements.”

Tectrade was supported in the project by Triangle, another IBM Premier Business Partner. Triangle’s expertise with IBM System p complemented Tectrade’s IBM Tivoli and System Storage expertise. Triangle prides itself on an ability to see the bigger picture, employing proven innovators to help customers achieve measurable improvements through faster, simpler and more cost-efficient business processes.

Tiered backup solution

WPD previously had a single server/single application approach to its System p environment. Following a major consolidation and virtualization effort, the company now has around 20 logical partitions (LPARs) on a dozen System p servers, linked in a SAN to an IBM System Storage DS4800.

The System p environment runs the company’s line-of-business applications, including asset management for poles, pylons and transformers. It also runs the distribution billing system – not for retail customers, but rather for suppliers who use the WPD infrastructure to transmit power. Other line-of-business applications include the payroll system and streetworks management, and WPD also runs infrastructure tools such as IBM Tivoli Enterprise Console on the System p platform.

“We have placed our services into tiers, which determines their priority for recovery,” says Al Kellaway. “As part of our disaster planning, some services change priority depending on the time of the month. Payroll is a good example – if salaries are not due to be paid for a couple of weeks, then other services can be restored as priority cases. By moving to disk-based backups, we have accelerated the recovery of servers significantly, and enabled more systems to be recovered in parallel.”

For mission-critical systems such as power distribution, WPD has a separate high-availability environment replicated across six sites, each capable of completely independent operation.

Efficiency through virtualization

WPD’s use of virtualization now extends to the Intel-based environment: the company is using VMware to run around 80 virtual servers on 12 physical machines. Following the move to disk-based backup, individual virtual servers can be recovered within just seven minutes in a DR scenario, and WPD can recover individual files almost instantly.

Says Al Kellaway: “For the full recovery of critical systems, we’ve gone from as much as ten days to as little as two days, thanks to the move to disk-based backup with Tivoli Storage Manager on the DS4800.”

He concludes: “The decision for IBM hardware was certainly influenced in part by our use of IBM System p as a strategic platform. It makes sense to do as much as possible with a single vendor, to keep training and support costs low. Our 220 servers are managed by a team of just 12 people; by standardising on a small set of technologies, we can keep training and support costs very low.”