This document explains the severe limitations of a direct network connection between System z and IBM DB2 Analytics Accelerator for z/OS. These limitations are the reason why IBM does not support such a setup when it is used in production, development, or test environments. The document also recommends a basic and a high-availability network setup that are free of these limitations.
The term wall IP is used for the IP address that is brought up on the active Netezza host. It is the IP address which is used by DB2 for z/OS to connect to IBM DB2 Analytics Accelerator for z/OS.
The term host-N maintenance IP address refers to the unique IP address that is assigned to a Netezza host for maintenance purposes. This IP will always be assigned to a particular host and is not reassigned.
The term OSA-Express card refers to an OSA-Express 3 card or an OSA-Express 4S card.
Connecting a single Central Processing Complex (CPC) to a single accelerator
Normally, two systems can be connected with a single network cable. IBM DB2 Analytics Accelerator for z/OS, however, is a complex system, which consists of more than one server (host). Using a single cable, your System z can be connected to the active or the passive Netezza host. The wall IP address is always brought up on the active host. Under regular conditions, this type of connection works.
However, in case of a failover (either triggered by a software or a hardware error), the roles of the hosts change and the wall IP is brought up on the other host. Thus, the connection to the wall IP address (active host) is broken.
An additional connection to the other host is needed.
To bring a second OSA-Express card into play, you must add an entry to the z/OS routing table. Both routing table entries must be valid for the same network. If MULTIPATH is set, z/OS will use both routes or interfaces in a round-robin fashion. This is counterproductive because every other packet will be lost. Therefore, do not use the MULTIPATH setting at all.
If NOMULTIPATH is set instead, z/OS chooses the first available route that it "finds" in the routing table. There is no alternation between the configured routes. In the example, the first available route refers to the upper connection, which leads to the active host.
This connection works fine until a failover occurs on the accelerator side, in which case the roles of the hosts change, and the wall IP is brought up on the other, formerly passive Netezza host. However, as long as the upper connection remains physically intact, z/OS will continue to use it, due to its routing configuration. Thus the network packets are sent to the wrong host.
The solution is to manually reverse the entries in the routing table so that the lower connection is used or to stop the first OSA card.
Connecting two CPCs to a single accelerator
The same rules apply if you connect two CPCs to the same accelerator. Likewise, this approach cannot work because of the way IBM DB2 Analytics Accelerator for z/OS utilizes the dual port adapters. IBM DB2 Analytics Accelerator for z/OS uses a virtual network device driver called 'bond' which creates a virtual network interface to the operating system and its applications, such as the IBM DB2 Analytics Accelerator for z/OS application itself. While the applications use the bond interface, the bond driver determines the physical connection that is used. Only one of the connections is active. The other connection is passive. Should the active connection fail, the passive connection is activated automatically. The applications can continue to use the bond interface. A configuration change is not necessary.
The setup of two CPCs connected to one IDAA would look like the following.
Taking the bond interface into account, the connections look as shown in this diagram from the NZ hosts' point of view:
You notice that the connection to CPC One is active, and that the connection to CPC Two is passive.
On the second host, the situation is the same. The wall IP is not available and the IDAA server application is not started on this host.
Therefore, the overview diagram looks like this:
In this picture the green cable is active and can reach the IDAA server application by using the wall IP address. The blue cable is active, but cannot reach the IDAA server application because it is not running on the passive host. The red cables are passive and can therefore not reach any IP address.
Since the cable that is used for the bond driver is automatically selected, it is hard to predict the active and the passive cable. However, it is inevitable to end up in a situation where one cable is working, one cable is directed to the "wrong" host, and two cables are passive. This means that one of the CPCs might be able to contact the IDAA server application, whereas the other will definitely fail to do so.
In case of a failover on the accelerator side, you will have to reverse the entries in the z/OS routing table manually, so that the other connection is used.
Because of these limitations with regard to failover support, IBM does not support direct network connections between System z and IBM DB2 Analytics Accelerator for z/OS in production, development, or in continuous test environments.
IBM recommends a basic network setup that uses one OSA-Express card, one switch, and three cables. The cables connect the switches with the OSA-Express card and the Netezza hosts. In such an environment, a failover on the accelerator side does not matter because the switch always provides a path from System z to the active host (see figure).
The recommended high-availability network uses two OSA-Express cards, two switches, and the required number of additional cables. In such an environment, failover on the accelerator side does matter either because the links between the switches guarantee that a path from System z to the then active host can always be found. If one switch fails, incoming signals are routed through the other switch, which can also connect to both Netezza hosts. To avoid further single points of failure, add sufficient redundancy to the remaining network architecture.
Additional CPCs can be easily integrated into both recommended setups.