DB2 10.5 for Linux, UNIX, and Windows

IBM PowerHA SystemMirror for AIX (formerly known as High Availability Cluster Multi-Processing for AIX or HACMP™)

IBM® PowerHA® SystemMirror for AIX® is cluster managing software. The nodes in PowerHA SystemMirror clusters exchange messages called heartbeats, or keepalive packets. If a node stops sending these messages, PowerHA SystemMirror invokes failover across the other nodes in the cluster; and when the node that failed is repaired, PowerHA SystemMirror reintegrates it back into the cluster.

There are two types of events: standard events that are anticipated within the operations of PowerHA SystemMirror, and user-defined events that are associated with the monitoring of parameters in hardware and software components.

One of the standard events is the node_down event. This is when a node in the cluster has failed, and PowerHA SystemMirror has initiated failover across the other nodes in the cluster. When planning what should be done as part of the recovery process, PowerHA SystemMirror allows two failover options: hot (or idle) standby, and mutual takeover.

Note: When using PowerHA SystemMirror, ensure that DB2® instances are not started at boot time by using the db2iauto utility as follows:

   db2iauto -off InstName

where InstName is the login name of the instance.

Cluster configuration

In a hot standby configuration, the AIX processor node that is the takeover node is not running any other workload. In a mutual takeover configuration, the AIX processor node that is the takeover node is running other workloads.

Generally, in a partitioned database environment, DB2 database runs in mutual takeover mode with database partitions on each node. One exception is a scenario in which the catalog partition is part of a hot standby configuration.

One planning consideration is how to manage big clusters. It is easier to manage a small cluster than a big one; however, it is also easier to manage one big cluster than many smaller ones. When planning, consider how your applications will be used in your cluster environment. If there is a single, large, homogeneous application running, for example, on 16 nodes, it is probably easier to manage the configuration as a single cluster rather than as eight two-node clusters. If the same 16 nodes contain many different applications with different networks, disks, and node relationships, it is probably better to group the nodes into smaller clusters. Keep in mind that nodes integrate into an PowerHA SystemMirror cluster one at a time; it will be faster to start a configuration of multiple clusters rather than one large cluster. PowerHA SystemMirror supports both single and multiple clusters, as long as a node and its backup are in the same cluster.

PowerHA SystemMirror failover recovery allows predefined (also known as cascading) assignment of a resource group to a physical node. The failover recovery procedure also allows floating (or rotating) assignment of a resource group to a physical node. IP addresses, and external disk volume groups, or file systems, or NFS file systems, and application servers within each resource group specify either an application or an application component, which can be manipulated by PowerHA SystemMirror between physical nodes by failover and reintegration. Failover and reintegration behavior is specified by the type of resource group created, and by the number of nodes placed in the resource group.

For example, consider a DB2 database partition (logical node). If its log and table space containers were placed on external disks, and other nodes were linked to those disks, it would be possible for those other nodes to access these disks and to restart the database partition (on a takeover node). It is this type of operation that is automated by PowerHA SystemMirror. PowerHA SystemMirror can also be used to recover NFS file systems used by DB2 instance main user directories.

Read the PowerHA SystemMirror documentation thoroughly as part of your planning for recovery with DB2 database in a partitioned database environment. You should read the Concepts, Planning, Installation, and Administration guides, then build the recovery architecture for your environment. For each subsystem that you have identified for recovery, based on known points of failure, identify the PowerHA SystemMirror clusters that you need, as well as the recovery nodes (either hot standby or mutual takeover).

If you plan to use PowerHA SystemMirror on two or more computers that share the same home directory, you must install the database manager in the same installation path. Using symbolic links to a similar installation path might cause issues in this scenario. The installation paths must be the same physical path.

It is strongly recommended that both disks and adapters be mirrored in your external disk configuration. For DB2 physical nodes that are configured for PowerHA SystemMirror, care is required to ensure that nodes on the volume group can vary from the shared external disks. In a mutual takeover configuration, this arrangement requires some additional planning, so that the paired nodes can access each other's volume groups without conflicts. In a partitioned database environment, this means that all container names must be unique across all databases for all SMS or DMS table spaces. Automatic storage table spaces manage this requirement for you.

One way to achieve uniqueness is to include the database partition number as part of the name. You can specify a node expression for container string syntax when creating either SMS or DMS containers. When you specify the expression, the node number can be part of the container name or, if you specify additional arguments, the results of those arguments can be part of the container name. Use the argument " $N" ( blank]$N) to indicate the node expression. The argument must occur at the end of the container string, and can only be used in one of the following forms:

Table 1. Arguments for Creating Containers. The node number is assumed to be five.
Syntax	Example	Value
blank]$N	" $N"	5
blank]$N+ number]	" $N+1011"	1016
blank]$N% number]	" $N%3"	2
blank]$N+ number]% number]	" $N+12%13"	4
blank]$N% number]+ number]	" $N%3+20"	22

Note:

% is modulus.
In all cases, the operators are evaluated from left to right.

Following are some examples of how to create containers using this special argument:

Creating containers for use on a two-node system.

   CREATE TABLESPACE TS1 MANAGED BY DATABASE USING
      (device '/dev/rcont $N' 20000)

The following containers would be used:

   /dev/rcont0   - on Node 0
   /dev/rcont1   - on Node 1

Creating containers for use on a four-node system.

   CREATE TABLESPACE TS2 MANAGED BY DATABASE USING
      (file '/DB2/containers/TS2/container $N+100' 10000)

The following containers would be used:

   /DB2/containers/TS2/container100   - on Node 0
   /DB2/containers/TS2/container101   - on Node 1
   /DB2/containers/TS2/container102   - on Node 2
   /DB2/containers/TS2/container103   - on Node 3

Creating containers for use on a two-node system.

   CREATE TABLESPACE TS3 MANAGED BY SYSTEM USING
      ('/TS3/cont $N%2, '/TS3/cont $N%2+2')

The following containers would be used:

   /TS3/cont0   - on Node 0
   /TS3/cont2   - on Node 0
   /TS3/cont1   - on Node 1
   /TS3/cont3   - on Node 1

Configuring DB2 database partitions for PowerHA SystemMirror

Once configured, each database partition in an instance is started by PowerHA SystemMirror, one physical node at a time. Multiple clusters are recommended for starting parallel DB2 configurations that are larger than four nodes. Note that in a 64-node parallel DB2 configuration, it is faster to start 32 two-node PowerHA SystemMirror clusters in parallel, than four 16-node clusters.

A script file is packaged with DB2 Enterprise Server Edition to assist in configuring for PowerHA SystemMirror failover or recovery in either hot standby or mutual takeover nodes. The script file is called rc.db2pe.ee for a single node and rc.db2pe.eee for multiple nodes. They are located in the sqllib/samples/hacmp/es directory. Copy the appropriate file to /usr/bin on each system in the PowerHA SystemMirror cluster and rename it to rc.db2pe.

In addition, DB2 buffer pool sizes can be customized during failover in mutual takeover configurations from within rc.db2pe. (Buffer pool sizes can be configured to ensure proper resource allocation when two database partitions run on one physical node.)

PowerHA SystemMirror event monitoring and user-defined events

Initiating a failover operation if a process dies on a given node, is an example of a user-defined event. Events must be configured manually as a user defined event as part of the cluster setup.

For detailed information on the implementation and design of highly available IBM DB2 database environments see the IBM Software Library web site (http://www.ibm.com/software/sw-library/).