Metro shows how IBM external storage can provide an enterprise-wide high availability solution for SAP software environments

About this paper

This technical brief describes the project to migrate Metro’s SAP R/3 Enterprise system onto IBM external storage. This brief provides a technical overview of system sizing and layout, project planning and realization of an enterprise wide storage solution, providing Metro with a flexible, scalable, and cost-effective solution.

Customer Objectives

• Enable the entire enterprise’s heterogeneous environment (or any subset of it) to be switched over to systems at the disaster recovery site in less than 60 minutes in the event of a disaster or a planned outage
• Decrease the technical support requirements caused by having different replication tools in each of the company’s IT environments (IBM i5/OS, IBM AIX, IBM z/OS and Microsoft Windows)
• Eliminate delays in data synchronization
• Shorten the time and decrease the risk of implementing new projects by leveraging the ability to deploy large amounts of storage whenever and wherever required
• Maximize the utilization of storage across the enterprise
• Eliminate the need to make numerous separate business cases for storage acquisition, and diminish the risk of inaccuracy in predicting requirements for each of the different environments.

IBM Solution

• Introduce a dual-site solution (10 km apart) based on two IBM System Storage DS8300 and two IBM ESS800 disk storage systems
• Use Independent Auxiliary Storage Pools (IASP) together with i5/OS cluster services to enable the IASP switchability required for this solution
• Mirror the storage systems using IBM Metro Mirror (formerly Peer-to-Peer Remote Copy), and use IBM FlashCopy for high speed back-up in the z/OS environment
• Leverage the Fiber Channel load-source and multipath I/O capabilities of the IBM System i5 servers
• Deploy IBM Director for server management and the IBM Copy services tool kit for additional control of the storage environment.

Customer Benefits

• Time required to transfer operations to the disaster site has been reduced by two thirds
• Highly flexible environment enables data to be moved rapidly to wherever it is needed
• New business projects can be implemented quickly, and tested on full copies of production data when required
• Synchronization problems have been eliminated
• When questions or issues arise, there is only one IT vendor to deal with
• Storage costs have been reduced and utilization rates improved
• Highly scalable solution can easily support the organization as it grows and takes on new business ventures
• Only one business case needs to be made each year for storage across the entire enterprise.

Background, starting point and objectives
Boasting more than 55 years of experience in the food industry, Metro Inc. operates a network of more than 550 food stores and 250 pharmacies under a variety of banners. The majority of stores are found in the Canadian provinces of Quebec and Ontario. With so many stores to manage, the corporation maintains a staff of 65,000 employees, providing quality service to its customers.

In August 2005, Metro Inc. acquired A&P Stores of Canada. Metro Inc. needed an infrastructure that would enable it to easily integrate A&P Store into its existing IT environment.

The company needed to be able to set up new IT environments rapidly. To reduce the risk involved, Metro needed a way to test these new systems using full copies of production data. This would require a storage environment flexible enough to move large quantities of data to and from different storage systems rapidly, without the need for manual intervention by technical staff.

Metro’s IT environment is based around ERP software from SAP, running in a two-tier environment on IBM System i servers. In addition to the SAP software landscape, the company also runs numerous business-critical applications on a heterogeneous server landscape, consisting of several Intel-based Microsoft Windows servers, as well as IBM System z and AIX partitions. The loss of any one of these business critical applications would lead to severe disruption to the business. The ability to switch over any combination of servers to the backup site as quickly as possible is critical to ensure business continuity at Metro.

The SAP software landscape was originally hosted on servers with dedicated, internal disks; data for the other applications was also stored in a variety of different systems. The heterogeneous server landscape meant that every different type of server required a different technology in order to be switched to a back-up site, which significantly increased the risk of performing a switch-over, and required significantly more technical support on an ongoing basis. In addition, the company’s data replication software was experiencing synchronization issues due to high transaction volumes.

Without storage consolidation, accurate prediction of the storage growth for each of the servers was difficult, and a business case was required to justify the storage demand for each individual server.
These limitations and heavy technical support requirements led to a new set of requirements:

• High Availability must be ensured
• The switch-over scenario had to be based on only one technology
• Prediction of storage growth needed to be simplified .

Metro needs its business-critical applications to be able to support near-24x7 operations. The company’s goal was to reduce the switch-over time to less than 60 minutes in case of a disaster, and to resolve the synchronization issues. Before introducing IBM System Storage technologies, Metro was using internal disks on its System i server, with third-party software application for data replication.

As most of Metro’s key applications were running on System i, it was important to develop a solution that would take advantage of the company’s strong System i technical skills. To identify and deploy the best possible solution, Metro worked very closely with IBM, leveraging their visits to the IBM Rochester lab during the Large User Group (LUG) meetings.

Metro evaluated a number of external storage products from both IBM and its competitors. The decision to choose IBM external storage was based on price, superior data replication capabilities, and the availability of System i technical support. In addition, the ability to work with a single vendor when workloads need to be switched would minimize the risk of compatibility and support problems and reduce the time taken to determine the causes of any problem.


Sizing the infrastructure solution
At the time of implementation, Metro Inc. had 1,500 named SAP software users, with a peak of 500 concurrent users for the sales and distribution, financial accounting, cost controlling and plant maintenance functionalities within the SAP ERP application and the SAP NetWeaver Business Intelligence component of SAP NetWeaver 7.0.


Move from internal to external drives needed careful planning
The IBM System i and System Storage teams worked closely together with Metro. The IBM internal “Disk Magic” sizing tool was used in order to determine the best hardware configuration. Several models were built, taking into account the different I/O characteristics of the day-time and night-time batch run workloads.

Following the disk magic sessions, Metro set out to decrease the number of I/Os in their System i environment. In order to minimize the investment in external storage. Metro attended the IBM “DB2 for i5/OS performance workshop”, and was subsequently able to decrease the number of I/Os by 50 percent within approximately three months, through the creation of indexes tailored to business needs.

Metro went to the IBM Rochester Benchmark Center in order to carefully size the system auxiliary storage pool (system ASP) in preparation for the move to IASP.

The project originally began with the IBM ESS800 and the I/O workload was gradually moved, making it easier to compare the performance of the external and internal drives. The production work was not fully moved to the IBM ESS800 because of performance concerns. The newly announced DS8300 was then installed, and finally all the production workload was moved to IBM external drives (with the exception of the system ASP, as there was no need to move it at this time).

As an early adopter of IASP for the SAP software environment, it was important for Metro to have SAP BASIS skills on site. IBM Global Business Service provided a basis consultant, and the SAP on System i teams from both IBM and SAP were also involved. The redbook “IBM eServer iSeries Independent ASPs: A guide to moving Applications to IASPs” (IBM document SG24-6802-00), includes a chapter on moving SAP to IASP and was used as reference material.


IBM DS8000 Storage Server as primary data host
Metro’s SAP software environment consists of two System i 595 servers (0954-5891), running i5/OS V5R4 with dynamic logical partitioning (DLPAR) and uncapped processors. Each i595 system is connected to a DS8300 and/or the ESS800, with 27TB of data stored at the production site and 39 TB at the backup/development site.

Metro and IBM developed a mirroring solution, utilizing IBM Metro Mirror functionality to provide a synchronous mirror. The IBM System Storage DS8000 and ESS storage servers are designed to provide high availability, with no single point-of-failure. Metro Mirror is able to utilize more than one physical link for data replication and mirroring, ensuring minimal latency and rapid data synchronization after a fail-back of the IT center.


System design
Each i595 is a 32-way system, sharing the workload between them. The following list describes the installed applications:

Applications running under i5/OS:

• SAP R/3 Enterprise
• SAP NetWeaver BI
• IBM WebSphere Business Integration
• IBM WebSphere MQSeries
• IBM Content Manager OnDemand
• Trusted Link (EDI)
• Showcase
• Legacy Applications
  

Applications running under AIX 5L 5.3:

• Control/M - Job scheduler
• Tivoli Storage Manager 5.4
• Several internal A&P applications

Independent Auxiliary Storage (IASP) pools
Key to this solution is the use of Independent Auxiliary Storage (IASP) pools. IASPs are a type of user ASP, numbered 33 through 255. IASPs are different from basic ASPs in several ways. Independent ASPs can be used on a single system or switched between multiple systems or LPARs when the IASP is associated with a switchable hardware group.

When used on a single system, the IASP can be dynamically varied on or off without restarting the system. In iSeries Navigator, the IASP and its contents can be dynamically made available or unavailable to the system.

When used across multiple systems, clustering support is required between the systems, and the cluster management GUI in iSeries Navigator is used to switch the IASP across systems in the cluster. This is referred to as a switchable IASP. At any given time, the IASP can be used by only one of those systems.
Multiple systems cannot use the IASP simultaneously. IASPs also differ from basic ASPs because they are identified by a device name on the System i5 platform. This device can be varied on or off to make it available or unavailable. This can be done without a system IPL, which saves a lot of time and increases the flexibility offered by ASPs. (Reference Redbook “IBM System Storage Copy Services and System i5”, IBM document SG24-7103-01)
Here is an example of how the switch between the IASP environments is performed:

• Stop the SAP applications that are running at the primary site
• Vary-off the IASP
• Failover the PPRC (Metro Mirror)
• Transfer IP (virtual private network)
• Vary-on the IASP at the remote site
• Reverse the PPRC (Metro Mirror) relationship, so that secondary becomes primary and vice versa
• Start the SAP applications at the secondary site.

Project results
The combination of System i, external storage and SAP software has enabled Metro to develop a highly scalable IT architecture, facilitating the support of business growth and offering an enterprise-wide solution to the company’s availability requirements:

• With the new infrastructure Metro is able to perform a switch-over in less than 60 minutes. IBM Copy services tool kit was key in obtaining this goal.
• By using IBM Metro Mirror for data replication, Metro has been able to eliminate the need for third-party data replication software, reducing licensing costs and reducing complexity.
• With internal disk storage it was difficult to perform tests on full copies of production data. With a centralized storage environment, storage can be assigned to any server in a very flexible way.
• With a consolidated storage environment, Metro is able to predict storage growth more accurately, ensuring that the right amount of hardware is always available to meet the demands of the business.
• The move to IBM external storage has delivered equal or better performance for end-users in the heterogeneous server environment.
• Technical staff supporting this environment only need to be trained on one type of storage hardware and one type of data replication. This saves time for education and minimizes the risk during fail-over.

Metro’s future plans
Metro intends to further improve on their current availability and to minimize the current maintenance windows.

In June 2007, Metro asked the IBM Optimum Care program to perform an “Availability Assessment.” This IBM service led to several IBM recommendations that are currently being implemented, and that will position Metro to accomplish its future plans.