Palomar Pomerado Health prepares for new strides in digital healthcare with storage virtualization

Palomar Pomerado Health (PPH), the largest public health district in California, is expanding services significantly. Funded by an US$496 million bond measure, PPH is adding another hospital and planning to adopt more digital technologies in the near future.

In fact, the widespread use of technology is one of the ways that hospital management helps the staff provide the best of care for patients. For example, many manual processes have been streamlined by the Cerner Millennium® suite of clinical software, which includes financials, laboratory systems, surgery scheduling, clinical and pharmacy systems and other processes.

Every day, more patient information is being stored in digital form, making IT system reliability and availability more critical than ever. According to Paul Engberg, director of technical services at PPH, the hospital is moving toward fully automated medical records, which will increase the need for IT to deliver secure, reliable access to information. Further into the future are plans to move toward computerized physician order entry (CPOE)—a digital physician bedside care system that will again raise the bar on system availability.

Infrastructure upgrades position PPH for continued growth

In response to a growing number of users and the need for greater system availability, the PPH IT staff implemented a wide-ranging upgrade of the computing infrastructure. Previously, three IBM System p™ 650 servers and a large IBM Enterprise Storage System (ESS) supported Cerner production and test functions, including picture archiving and communication system (PACS) image storage.

In an effort to establish the best direction for the upgrade, the IT staff discussed the hospital’s needs with the Cerner team. According to Engberg, “Cerner recommended that we move to two IBM System p5™ 590 servers configured in a high-availability cluster, which would support more users than the three existing p650 servers and dramatically improve server availability. Cerner also recommended that we adopt a virtualized storage environment based on IBM System Storage SAN Volume Controller (SVC).”

Virtualized storage forms a flexible infrastructure

A virtualized storage environment can dramatically improve the ability of PPH IT staff to quickly provision the best storage solution for each application. With many new storage uses on the horizon, IT needs the flexibility to assign the most cost-effective storage systems based on user requirements. A virtualized environment can help simplify this task because servers and applications see virtual devices, not physical devices. Therefore, IT can assign the most appropriate subsystems within the physical infrastructure—even change the assignments on the fly without affecting applications.

To increase the capacity and performance of the storage environment, PPH IT staff augmented the existing ESS with new IBM DS6800 and DS4300 storage arrays configured in a storage area network. The existing Tivoli® Storage Manager software continues to handle tape backups.

Tiered storage helps improve cost effectiveness

As part of the infrastructure upgrade, PPH moved to a tiered storage architecture. Each of the storage systems was assigned to a tier, based on its performance and cost:

Tier 1: IBM DS6800 (4 TB) provides the highest performance and is used for Cerner and Lawson production data.
Tier 2: IBM DS4300 (14 TB) stores all imaging files.
Tier 3: IBM ESS (15 TB) supports non-production data.

By helping to eliminate the need to over-provision storage, Engberg expects that the tiered approach can significantly lower the overall cost of storage for PPH.

With tiers now defined, IT staff can match storage subsystems to applications. Says Engberg, “When making storage assignments, it is easy to find the best match between our storage subsystems and the users’ needs and budget. As new storage devices are added over time, we can change those assignments without disrupting applications.”

Future storage requirements are driven by the kinds of applications adopted by the hospitals departments over time. Says Engberg, “We don’t really know what all of those requirements might be; many of those decisions are driven by our clinical staff. So having a flexible storage environment is a big asset for IT—we can now respond quickly to new requirements without having to rent a forklift or throw out existing equipment.”

But one storage requirement is a certainty: storage needs will continue to grow exponentially, especially due to digital imaging. “Not only are we storing more images every day, but new higher resolution imaging technology will drive up file sizes,” says Engberg. “SVC gives us much greater flexibility in the way we manage that growth. For example, we have requirements to save images for 21 years. That is driving the need for image archiving capability in the next year. SVC will play a role in helping us accomplish that in the most effective way.”

IBM SVC reduces disruption to users during the upgrade

Because of the all-encompassing nature of the infrastructure upgrade, the rollout was accomplished in phases. First, the new servers were installed and the Cerner Millennium® system software updated. An upgrade to the SAN fabric followed, including the addition of IBM SVC and new IBM DS4300 and DS6800 storage subsystems. Finally, storage tiers were defined and data was migrated from the legacy ESS to the appropriate storage tier.

Up-front planning is always the key to a smooth upgrade process, states Engberg. “Having SVC in place prior to the data migration helped minimize the disruption to users,” he says. “Once all the legacy storage was behind SVC, the data could be migrated to the appropriate tier of storage without any disruption to applications.”

Dennis Dechant, PPH system administrator, notes the streamlined migration process as well. “I did the entire migration of our Lawson financial applications to the DS6800 while users were online. No one noticed any degradation during the process,” he adds.

Dechant also appreciates the help provided by the vendors. “Even though Cerner provided the storage systems, IBM technical people came onsite to help us set up SVC. In the process, they also taught us how to manage SVC on an ongoing basis.”

IBM SVC helps boost storage availability

SVC helps improve storage availability by enabling storage system maintenance to take place without disrupting users. According to Dechant, some storage maintenance activities—such as microcode upgrades—previously required taking the storage device offline. Now, with all storage behind SVC, “We can non-disruptively move the files off the storage device before taking it offline,” he says. “In that way, applications continue to operate and users are unaffected. When we are done with the maintenance, we simply move the data back to the upgraded storage device.”

The higher performance and higher availability of the new infrastructure has direct benefits to staff, says Engberg. “Because the hospital is moving toward electronic medical records, we need to have an IT infrastructure that is highly available and resilient to failures. And IT needs the flexibility to respond quickly to new requirements from our users,” he explains. “In the storage arena, SVC combined with IBM storage systems provides that infrastructure.”

Storage administrators become more productive

With storage management centralized on IBM SVC, storage administration is much faster according to Mitesh Manek, systems administrator. “We no longer need to get inside the storage subsystems or remember how to use the various storage subsystem configuration utilities. Almost all storage management functions are accomplished using the SVC console. Having one uniform console gives me a system-wide view across all devices so I can see the big picture. It also has streamlined our administrative work.”

SVC also gives the IT staff more time to work on higher-level aspects of the infrastructure. For example, “I no longer have to worry about spreading disk activity across physical devices to avoid hotspots that slow performance,” says Manek. “At any time, we can migrate data among the storage subsystems to balance the workload and maximize performance. With SVC, I’m sure we can improve performance compared to our non-virtualized environment.”

Logical unit number (LUN) sizing has also become very efficient, in part because SVC enables capacity to be allocated in small increments. Says Manek, “I don’t have to allocate a 64 GB LUN if 32 GB will do. This fine granularity has enabled us to improve the utilization of our storage investment. And all of these types of changes can be accomplished without disrupting users.”

Storage virtualization brings disaster recovery closer to reality

To help maintain operations if something should happen to the primary data center, the PPH IT staff plans to add a second SAN at another location. Says Engberg, “An SVC node at the secondary site will maintain mirrored data on a second DS6800 storage system. By using SVC to manage storage at the two sites, we eliminate any impact on host software so all the power of the p5-590 servers is dedicated to production work. Having mirrored data will save us many days rebuilding systems if we are hit by a disaster. We expect to be able to restore operations in a few hours, which will enable our staff to continue providing quality patient services with very little disruption.”

All-encompassing approach to virtualization pays off

The decision to virtualize all storage with SVC was a no-brainer, says Dechant. “There are really no negatives to moving to a virtualized storage environment,” he explains.

Engberg agrees, adding that SVC does not lock PPH into a single vendor for storage systems. “When you are investing for the long-term, it is all about flexibility. Today, all our storage is from IBM. But we want the ability to add some new device from another vendor so we can be opportunistic. That’s one of the big reasons why we saw no negatives in the move to SVC.”

The PPH IT staff put all Cerner data behind SVC—images, the production clinical system and test systems. Says Engberg, “This proved to be the right decision because we are now receiving the benefits of virtualization in every area of the business. Flexibility and ease of administration are benefits to IT. And users quickly get exactly the right storage for their needs—whether it is a new application or just a request for more space. And as we implement our business continuity initiatives, having virtualized all storage enables us to provide greater resiliency to all our users. Therefore, why would we not put all our storage behind SVC?”

“Storage virtualization, managed by IBM SVC, is key to our plan to maintain a highly reliable yet flexible IT infrastructure,” adds Engberg. “Our infrastructure is now well positioned to support the move to paperless medical records and automated systems that help our staff when they are at the patient’s bedside or even outside our facilities at their offices or homes.”

For more information

For more information, contact your IBM sales representative, IBM Business Partner or visit us on the Web at: ibm.com/servers/storage

For more information on Palomar Pomerado Health visit www.pph.org

For more information on Cerner Corporation visit www.Cerner.com