Bauerfeind consolidates business processes with SAP Business Suite on a dynamic IBM infrastructure

About this paper

This paper describes how a company replaced its legacy systems (MAS90 and others) with a company-wide deployment of SAP Business Suite, and how its new IBM Power Systems and IBM System Storage infrastructure enables the SAP environment to be managed in a flexible, dynamic manner. Administration and SAP operation are simplified considerably by using a global file system (IBM GPFS). This also enables the utilization of a highly available, horizontally scalable database. Furthermore, the paper discusses how combining IBM Tivoli Storage Manager (TSM) and IBM CommonStore can provide an efficient solution for managing the storage and backup of sensitive business data.

Customer Objectives

• Standardized processes based on SAP applications
• Worldwide provisioning of standardized business processes through a central IT organization
• International growth through standardized and consolidated processes
• Flexible extension of the application landscape through scalable IT
• Simple administration supported by virtualization
• Fail-safe operation for internal and external users
• Long-term storage of documents, integrated with SAP.

IBM Solution

• Design and sizing of an IT architecture to replace legacy systems
• Introduction of a fully virtualized server and storage landscape:
• IBM Power Systems and PowerVM for server virtualization
• Two IBM Power 570 servers, with POWER6 processors and 96GB of memory, running Oracle Real Application Cluster (RAC) database cluster and SAP central instance
• IBM BladeCenter H with POWER6 processor-based blades, running SAP application servers
• Two IBM Power 550 servers running IBM Tivoli Storage Manager and IBM CommonStore
• IBM System Storage SAN Volume Controller (SVC) for storage virtualization:
• IBM System Storage DS5300 hardware
• Other heterogeneous legacy storage hardware
• IBM General Parallel File System (GPFS) is used as a global file system for fast re-allocation and provisioning of SAP instances
• IBM Tivoli Storage Manager is used for backup of critical SAP data and documents
• IBM CommonStore is used for document management and data archiving via standard SAP Active-Link interface.

Customer Benefits

• Additional SAP users can be integrated seamlessly as the company grows
• Low administrative effort and operational costs:
• High degree of standardization in both the SAP environment and IT infrastructure
• Fewer peripherals and reduced network complexity
• The IT team can serve many more systems and users without increasing their headcount
• Near-zero downtime:
• Oracle Real Application Cluster provides high availability for database servers
• If an SAP application server detects a fault, GPFS enables rapid re-provisioning
• PowerVM virtualization provides high average system utilization while maintaining constant response times
• SVC enables dynamic adaptation and tiering of physical storage to meet application requirements
• The landscape delivers excellent price-performance, as demonstrated by a VMS benchmark.¹

Background, starting point and objectives

About Bauerfeind
Bauerfeind AG is a specialist manufacturer of medical aids, such as supports, orthoses, compression stockings and foot orthoses. Headquartered in Zeulenroda, Germany, the company employs around 1,600 staff. In addition to its manufacturing facilities, the company also runs the Bauerfeind Innovation Center, which works with scientists, medical practitioners and retailers to research and develop new products, and the Bauerfeind Academy, which provides training about the company’s products.

Before 2001, the Bauerfeind group operated with a decentralized organizational structure. In addition to its headquarters, which were at that time located in Kempen, the company established other locations in Germany, Europe, the US and Asia.

In 2001, Bauerfeind moved its head office to a new facility in Zeulenroda. The company’s owner, Professor Bauerfeind, realized that the business needed to renew its corporate organization structure in order to adapt successfully to future business requirements.
At the heart of the Professor’s renewal strategy was the concept of centralization, combined with the incorporation of a joint stock company. Business processes were reorganized to provide a consistent and standardized corporate approach for the whole company.

In 2002, the company began designing a modern, centralized, corporate-wide business management strategy based on SAP Business Suite, and initiated a five year project program to implement the solution. In the course of this project, it was necessary to replace legacy systems like Brain AS and MAS 90, and build a completely new network infrastructure as well as a new centralized storage solution. This solution had to be scalable to meet the needs of the company as it continued to grow.

About IBM
Bauerfeind entered a strategic partnership with IBM Germany to help design and build the new infrastructure. A solution based on IBM servers, storage and software was tested and deployed, and then extended and renewed step-by-step.

IBM’s ability to provide a comprehensive solution, including middleware and management tools as well as hardware, was a key factor in the success of the project – giving Bauerfeind a set of technologies that were proven to interoperate seamlessly, and a single resource for technical support.

About SVA
System Vertrieb Alexander GmbH (SVA) is one of the leading system integration specialists in the field of data center infrastructure in Germany, and employs more than 170 people at 10 locations. SVA’s objective is to combine high-quality IT products from different manufacturers with its own project know-how, services and flexibility to find the optimal solution for the customer. The core competencies of SVA are:

• Highly available IT architectures
• Data backup and disaster recovery
• Storage area networks
• Virtualization solutions in server and storage environments
• IT security solutions
• High performance computing solutions
• Consulting, professional services and outsourcing.

Working within this competence framework, a team from SVA’s Dresden office provided consultancy and implementation services for the implementation of the IBM infrastructure solution at Bauerfeind. For more than nine years, SVA specialist consultants and system technicians have been analyzing and implementing innovative technologies to meet Bauerfeind’s complex requirements. The strategic cooperation between Bauerfeind, SAP, IBM and SVA from 2001 to 2010 was absolutely critical to the success of the project.


Standardized workflows and processes based on SAP Business Suite 7

Selecting SAP Business Suite
When Bauerfeind began its program of organizational renewal, the introduction of SAP Business Suite was one of its most important objectives. In order to create and promote standardized processes across its international organization, the company needed a highly versatile centralized ERP solution that could be customized to fit the precise requirements of its business.

The solution also needed to be comprehensive enough to support many different aspects of the business, including eCommerce and international trade, as well as EDI integration with the ERP solutions of its customers and suppliers.
SAP Business Suite was one of the few solutions on the market that could meet these complex requirements, so Bauerfeind decided to go ahead with the implementation.

The current SAP landscape
Bauerfeind is now using SAP Business Suite 7 and SAP NetWeaver 7.0 on a centralized IBM Power Systems infrastructure, running the AIX operating system. The following components have been installed:

• SAP Business Suite 7 (SAP ERP 6.0, enhancement package 4)
• SAP CRM 7.0
• SAP NetWeaver Business Warehouse (BW) 7.0
• SAP NetWeaver Process Integration (PI) 7.0
• SAP BusinessObjects Global Trade Services 7.1 (for import, export and customs online)
• Web platform for eCommerce:
• SAP Internet Sales 5.0
• SAP NetWeaver Portal 7.0
• SAP TREX 7.0
• SAP Solution Manager

Several external applications are connected to these SAP systems in real time: for example, a warehouse management system, an EDI interface, and further application interfaces via SAP NetWeaver PI.

Because the SAP applications mostly manage business-critical processes, reliability and high availability were the main criteria when selecting the underlying technology. Very early in the project, Bauerfeind decided to run the databases for the SAP environment in an active-active configuration using multiple nodes in an Oracle Real Application Cluster (RAC) on IBM Power Systems servers with a Unix OS. The resulting infrastructure is:

• Oracle 10.2.04 RAC
• IBM GPFS 3.2 as global shared file system
• IBM AIX 6.1 as operating system

In total – from the initial planning phase through to the go-live of the first SAP system – the implementation of this comprehensive environment took five years, from 2002 to 2007.

The SAP environment not only serves users at head office in Zeulenroda; it also serves many of the foreign subsidiaries remotely via VPN connections. These VPNs use fully redundant WAN network connections from several different providers, and can also use ISDN dial-up connections as a standby in case the WAN fails.

In total, 500-600 people (approximately 50 percent of the 1,100 licensed SAP users) are concurrent users of the SAP ERP system. Additionally, there are many specialized users of other SAP applications such as SAP CRM and SAP NetWeaver BW. Another important contribution to the total load on the infrastructure is made by the approximately 2,000 simultaneous B2B users. By integrating professional users (suppliers of medical equipment, pharmacies, and medical practices) online into the SAP eCommerce system, Bauerfeind can offer a rapid and efficient ordering process, with delivery within 24 hours.

As a result of all this user activity, Bauerfeind needs to manage data volumes of currently 2.8 TB for production systems such as SAP ERP and SAP CRM, and 2.6 TB for the SAP NetWeaver BW system. A further 0.5 TB is required for other systems that are integrated with the SAP environment.

Availability and virtualization as specifications
As we have seen, the continuous availability of business applications around the clock is crucial for Bauerfeind. The window for planned downtime is extremely narrow, because the company has international subsidiaries in nine time zones, and users need access to the SAP systems 24x7. This factor was a major influence on platform selection and architecture from the beginning of the project.

Additionally, to ensure a flexible and dynamic infrastructure that would facilitate functional extensions and scale to support corporate growth, Bauerfeind nominated server virtualization as another prerequisite for the global SAP roll-out.


SAP infrastructure with server and storage virtualization

SAP Central Services and Oracle database servers
To meet the availability requirements, Bauerfeind opted for a cluster solution on the database level. This was put into practice with an Oracle Real Application Cluster (RAC) and connected Oracle Clusterware. The underlying topology was designed with full redundancy for every single point of failure, and the cluster nodes were distributed between two separate locations on the Bauerfeind campus for disaster recovery reasons. Additionally, there is a third “quorum data center” where dedicated servers and disks are installed as tie-breaker nodes. This is important for very fast-reacting clusters to rule out so-called ‘split-brain situations’.

A split-brain situation occurs when the network connection or the heartbeat between the two active data centers is interrupted. In this situation, each of the independent GPFS and RAC nodes attempts to take over the entire SAP system, because the cluster software assumes that there has been a complete breakdown of the other nodes. Thus, a group of users would remain active on node A, while a second group remains active on node B, both of which would be running the same SAP instance.

The transactions that the two groups of users would perform would then lead to inconsistencies in the SAP database, leading to data corruption and making it impossible to manage business processes correctly. To avoid this situation, the tie-breaker node is used, the cluster site accessing and reserving this disks first becomes primary, and leads during data resynchronization.

The databases for the large SAP production systems run on IBM Power 570 servers in an active-active cluster configuration. One reason for using Oracle RAC is to enable horizontal scaling by adding additional nodes to handle growing workload. However, Bauerfeind’s main reason for using Oracle RAC is its ability to deliver high availability. If one of the two database nodes suffers a failure, the other can take over the whole workload, preserving 50 percent of the database server throughput and keeping systems online until the broken node can be repaired. SAP Central Services for ABAP systems (ASCS) are also moved to the remaining node using the Oracle SAPCTL utility, eliminating this single point of failure.

In normal operation, Bauerfeind uses IBM PowerVM and a shared processor pool database to manage load peaks on the Power 570 systems autonomously – so there is no need to switch additional nodes on or off. This configuration can be managed in a more dynamic and granular way than temporarily allocating incremental Oracle RAC-DB nodes.

SAP application servers
The SAP application servers are distributed between 12 IBM Power JS43 and JS22 blades. This solution was chosen because it was a cost-effective alternative for SAP dialogue instances, and because if a single blade breaks down, only a small proportion of dialogues will be lost.

The JS blades are dynamically virtualized with up to twelve LPARs, using PowerVM and Integrated Virtualization Manager (IVM). The configuration uses a clever distribution of production and non-production SAP instances to enable the hardware to handle load peaks as efficiently as possible. The non-production systems (development, test, QA and so on) have lower priority than the production systems, which means they can be used as resource buffers for short-term load peaks in the production systems. Thus, even on the blades, a very good utilization rate for both CPU (typically 0.8 CPU/LPAR in uncapped mode) and memory (typically 8 to 16GB/LPAR can be achieved.

Bauerfeind uses SAP Dispatcher to manage initial load-balancing at log-on in between the dialogue instances on the blades. To guarantee efficient utilization of capacity on the Oracle RAC database nodes, SAP users are grouped into logon groups according to the main applications that they use. These groups then primarily access tables on one or other of the two RAC nodes.

IBM General Parallel File System
The IBM General Parallel File System (GPFS) serves two purposes for Bauerfeind:

1. GPFS is the required clustered file system for SAP environments running on the AIX operating system to enable Oracle RAC nodes to access the same database tables simultaneously.
2. The GPFS based global file system also enables Bauerfeind to provision SAP systems on different Power servers or LPARs rapidly and flexibly.

About GPFS
IBM General Parallel File System (GPFS) provides concurrent high-speed file access for applications that are running on multiple nodes of a cluster. For commercial environments like SAP database servers running Oracle RAC, GPFS enables shared access of multiple active database server nodes and is a pre-requisite to run such a setup within an AIX OS.

GPFS is fault-tolerant and can be configured to allow continued access to data even if cluster nodes or storage systems fail. This is accomplished though robust clustering features and support for data replication. As well as AIX clusters, GPFS can be used for Linux or Microsoft Windows Server clusters, or even heterogeneous clusters that include a mixture of AIX, Linux and Windows nodes.

IBM SAN Volume Controller
Below the GPFS layer, copies of the data are stored on several different IBM DS disk systems to provide full redundancy. All the storage systems are connected to a four-node IBM SAN Volume Controller (SVC), configured as a stretched cluster, via a fully redundant SAN.

SVC offers a unified administration console for the underlying heterogeneous IBM storage landscape. The console provides access to standardized functionalities such as Metro Mirror, Flash Copy and VDisk Mirroring, and a standardized Subsystem Device Driver (SDD), independent of the underlying hardware. For Bauerfeind, this offers clear administrative and operational advantages for storage management.

Furthermore, the combination of SVC and IBM Tivoli Storage Manager enables improved information lifecycle management, storing different types of data on the most suitable media according to their current relevance and the frequency with which they are accessed. By defining several document classes, Bauerfeind is able to use SVC to apply a cost-optimized “tiered storage” concept.

About SVC
IBM SAN Volume Controller (SVC) is an appliance that provides intelligent storage-level virtualization. It can scale up from two to eight redundant controller nodes. It offers a unified administration console and a single standardized driver for heterogeneous storage landscapes – even if the storage systems come from different vendors.

SVC is used to manage data volumes for non-disruptive data migration, and it extends the functionality of low-end storage devices with snapshot and remote mirroring capabilities. For information lifecycle management, SVC can be used as a way to migrate data between different types of storage (for example, high-speed Fibre Channel disks and more cost-effective high-capacity SATA disks), creating an intelligent tiered storage environment.

For high availability, the SVC cluster can be split across two locations, with data mirrored between the two. Logically, and from the management of view, this stretched cluster is still just one cluster, and can be controlled using a single management console.

VDisk Mirror and Metro Mirror
The availability of data is guaranteed by mirroring at different storage levels. Firstly, there is disk-to-disk mirroring using SVC’s VDisk Mirror functionality. For this, SVC saves two copies of the data to two different DS storage systems in the same data center. Compared to mirroring at the operating system level (for example, using AIX LVM), this solution offers the advantage that all environments (AIX, Linux and Windows) can be mirrored using the same interface.

The VDisk Mirror protects against the failure of single disk systems, but is not a full disaster recovery solution. To guard against a genuine disaster scenario, Bauerfeind uses SVC’s Metro Mirror functionality. Data is copied from DS systems in one data center to similar systems in the other data center, using the SVC Flash Copy function. Predefined consistency groups guarantee that business data that belongs together is always copied synchronously, providing a point-in-time-copy. SAP operations are not affected by these online copies.

IBM System Storage DS5300
Bauerfeind is currently in the process of moving from its existing storage infrastructure, which involves a number of different disk storage systems from different vendors, and standardizing on IBM System Storage DS5300 technology. The company will deploy two IBM System Storage DS5300 systems with 13 EX5000 expansion units. Both are connected to the stretched SVC environment and will provide 24TB of mirrored net capacity.

Energy monitoring
For Bauerfeind AG, sustainability is of utmost importance. From an IT perspective, a holistic approach is required, and energy efficiency must be considered in any infrastructure project. For this reason, Bauerfeind not only monitors the availability of SAP and OS instances, but also the systems’ energy usage. All IBM components in the data centers are included in the monitoring to gain a true and accurate view of total energy consumption.

The infrastructure for the SAP environment was designed with sustainability in mind. The main improvements were: consolidating existing rack-mounted servers into a more compact IBM BladeCenter environment, upgrading from IBM POWER5 to POWER6 processors (which use up to 50 percent less electricity to deliver the same level of performance), and increasing the use of IBM PowerVM virtualization technologies.

As a result, Bauerfeind has gained an infrastructure that not only uses fewer, more energy-efficient processors in total, but also leverages IBM energy management tools to monitor and manage its systems in a more intelligent fashion, minimizing electricity consumption and heat output.


System and data management

System administration using native tools
To monitor and manage its server platforms, the Bauerfeind IT department uses both the tools that are built into the IBM hardware, and SAP NetWeaver’s native tools.

The creation or re-allocation of an SAP instance to a database node or application server is handled by an automated process based on the AIX mksysb utility and a Unix NIM (Network Install Manager) installation. Complete backups of current SAP systems are created using mksysb, and are managed as “Master Images” using NIM. The newly created instances can – after a quick adaptation – access all necessary SAP data through the global file system under GPFS. Thus, provisioning of additional nodes can be achieved quickly and easily. It takes the SAP application server no more than 20 minutes to make the SAP dialogue instance available.

To monitor the production systems in terms of performance and utilization, Bauerfeind uses a combination of the AIX nmon tool and SAP system transactions such as RZ 20. As well as monitoring the state of the physical servers, these tools can also record and display the PowerVM metrics of the virtualized LPARs, integrated into the SAP management console. The nmon reports are collected 24/7 and, if needed, can be consolidated over the entire virtualized landscape. Thus, load analysis and reliable growth predictions for the distributed server landscape are possible in near real time.

For SAP NetWeaver and application management, Bauerfeind uses the standard functionalities of SAP Solution Manager to handle application monitoring and alerts and to provide a support gateway. Bauerfeind plans to change its current strategy, which is mainly based on component monitoring, to a business process monitoring approach. Thus, on the SAP level, the company will not only monitor and control the KPIs of the IT systems themselves, but also the KPIs of the business processes they support. Based on the results, the IT resources are adapted to improve performance and efficiency. This can be done via the non-disruptive dynamic optimization of PowerVM parameters.

The efficiency of these processes and tools means that the SAP basis environment can be managed by a small team of just three or four people, who work as part of the network management and SAP application development team.

Data management with Tivoli Storage Manager and CommonStore
Data management and backup are based on products from the IBM Tivoli Storage Manager (TSM) suite. With Tivoli Storage Manager for ERP 5.5 – a version of TSM that includes special extensions for SAP data backup – the database nodes can be backed up without transferring data across the LAN. This is possible because instead of copying the data to a TSM server via LAN, the data is moved directly from disc to disc or disc to tape within the SAN. However, the logical and chronological control of backups remains on the central TSM server.

Because of the distributed database implementation, with Oracle RAC on a global GPFS file system, database tables have to be backed up via Oracle (RMAN) and SAP service programs (BRTOOLS). Consistency between the according DB logs and control files is guaranteed via an interface to TSM, which acts as the main backup instance. There is also a parallel backup via the TSM BACKINT functionality. Thus, each week, there is one full backup and six incremental backups of SAP data.

Because of its central functionality in backing up and storing all corporate data, the TSM server is implemented as a high-availability cluster, running on two IBM Power 550 servers. In this cluster, there are four TSM server instances with special functionalities:

1. Backup of AIX platforms (mainly for SAP databases)
2. Backup of Windows platforms (mainly for Microsoft Office and connectivity systems)
3. Archiving data from IBM CommonStore (for storage of SAP-related documents; see the following section for details)
4. TSM Library Manager (to manage access to the IBM 3584 and 3310 tape libraries)

Because Bauerfeind has the ability to quickly reinstall its SAP application servers whenever necessary (as described in the section on SVC), there is no need to back these servers up as long as they do not store any persistent data.

Data archiving with IBM CommonStore for SAP
Although electronic ordering via the B2B interface is preferred, about one third of all orders still reach Bauerfeind via fax. These fax orders are registered in SAP ERP manually, and electronic copies of the original documents are then logically linked to the online order data.

SAP offers two interfaces for this: the classic ArchiveLink and the more recent WebDAV interface. Bauerfeind is using ArchiveLink, which connects to a document repository based on IBM CommonStore for SAP 8.4. A TSM server is used to back up and archive the data held in CommonStore.

In total, the repository currently holds 90 million documents. Archiving is used to keep the SAP databases to a reasonable size. This in turn improves database performance and reduces backup and restore times.


Results achieved and future plans
Investment in modern technology, and the learning-curve required to complete the implementation, have resulted in the Bauerfeind IT department gaining a high level of competence in both the SAP and IBM technologies. More importantly, this enables them to deliver excellent service to the business, which results in satisfied SAP end-users.

The standardization of IT processes on a reliable platform enables efficient management of the SAP landscape. This in turn frees up resources to invest in other innovative projects. The combination of the dynamic IBM infrastructure with the versatile and customizable SAP environment makes it easier for the company to pursue its key strategies, especially in terms of international expansion.

Looking to the future, Bauerfeind intends to further expand the SAP environment, and to integrate a number of other technical applications:

• Portal applications and web services
• SAP Employee and Manager Self Service applications
• Product Lifecycle Management
• CAD/CAM integration
• Measurement system applications
• Guaranteed security on all levels

By minimizing the time that needs to be spent on low-level data center operations, the IBM solution enables the Bauerfeind IT team to focus on the implementation of these projects, working closely with other departments across the organization.