Preinstallation checklist for Db2 pureScale Feature (Linux)

Perform the following preinstallation steps and verify them on each host before installing the IBM® Db2 pureScale Feature.

Before you install

These steps must be performed on all hosts:

All hosts must use the same Linux® distribution.
Ensure that the required Linux Version and Service Pack is installed and is homogeneous across all hosts.
Db2 pureScale instances can be configured to use InfiniBand network, RoCE network or TCP/IP network. On an InfiniBand or RoCE network, ensure that OpenFabrics Enterprise Distribution (OFED) software is installed, and configured. See Configuring the networking settings of hosts on a RoCE network (Linux) and Configuring the networking settings of hosts on an InfiniBand network (Linux) for more information.
Increase the Mellanox HCA driver mlx4_core parameter log_mtts_per_seg value from 3 (the default) to 7 on the host where the cluster caching facility (CF) resides.
Note: The parameter value for log_mtts_per_seg is not required if using ConnectX-4 or higher.
To increase the size, issue this command as root:
- Starting from SLES 12 and RHEL 6.10, these module options must be built into the initial image used by the kernel during boot:
```
echo "options mlx4_core log_mtts_per_seg=7" >> /etc/modprobe.d/modprobe.conf
dracut -f
```
- On SLES 11:
```
echo "options mlx4_core log_mtts_per_seg=7" >> /etc/modprobe.conf.local
```
- On RHEL 6:
```
echo "options mlx4_core log_mtts_per_seg=7" >> /etc/modprobe.d/modprobe.conf
```
For this change to take effect, you must reboot the server. To check whether your change is effective on the module, issue this command:
```
<host-name>/sys/module/mlx4_core/parameters # cat /sys/module/mlx4_core/parameters/log_mtts_per_seg 
7
```
Db2 pureScale Feature requires libstdc++.so.6. Verify that the files exist with these commands:
```
ls /usr/lib/libstdc++.so.6* 
ls /usr/lib64/libstdc++.so.6* 
```
Ensure that OpenSSH is installed from the SLES media or RHEL media, as appropriate.
As root, validate ssh access between all hosts. From the current host, run the hostname command on the current host and on all other hosts in the cluster by using the ssh command. The result of the hostname command matching the host name identified in the ssh command verifies ssh access between the two hosts.
```
$ ssh host1 hostname
host1
```

Determine the number of paths to your device on the system with DM-MP, or EMC PowerPath driver:

On systems with the DM-MP driver, these are the commands to run, and sample output:

Check the Linux SCSI devices:

[root@host1 ~]# lsscsi
[3:0:1:0]    disk    IBM      2107900          .450  /dev/sdk
[3:0:1:2]    disk    IBM      2107900          .450  /dev/sdl
[3:0:1:3]    disk    IBM      2107900          .450  /dev/sdm
[3:0:1:4]    disk    IBM      2107900          .450  /dev/sdn
[3:0:2:0]    disk    IBM      2107900          .450  /dev/sdo
[3:0:2:2]    disk    IBM      2107900          .450  /dev/sdp
[3:0:2:3]    disk    IBM      2107900          .450  /dev/sdq
[3:0:2:4]    disk    IBM      2107900          .450  /dev/sdr
[4:0:0:0]    disk    IBM      2107900          .450  /dev/sdc
[4:0:0:2]    disk    IBM      2107900          .450  /dev/sdd
[4:0:0:3]    disk    IBM      2107900          .450  /dev/sde
[4:0:0:4]    disk    IBM      2107900          .450  /dev/sdf
[4:0:1:0]    disk    IBM      2107900          .450  /dev/sdg
[4:0:1:2]    disk    IBM      2107900          .450  /dev/sdh
[4:0:1:3]    disk    IBM      2107900          .450  /dev/sdi
[4:0:1:4]    disk    IBM      2107900          .450  /dev/sdj

List the LUN device mappings:

[root@host1 ~]# sg_map -x
/dev/sg9  4 0 0 0  0  /dev/sdc
/dev/sg10  4 0 0 2  0  /dev/sdd
/dev/sg11  4 0 0 3  0  /dev/sde
/dev/sg12  4 0 0 4  0  /dev/sdf
/dev/sg13  4 0 1 0  0  /dev/sdg
/dev/sg14  4 0 1 2  0  /dev/sdh
/dev/sg15  4 0 1 3  0  /dev/sdi
/dev/sg16  4 0 1 4  0  /dev/sdj
/dev/sg17  3 0 1 0  0  /dev/sdk
/dev/sg18  3 0 1 2  0  /dev/sdl
/dev/sg19  3 0 1 3  0  /dev/sdm
/dev/sg20  3 0 1 4  0  /dev/sdn
/dev/sg21  3 0 2 0  0  /dev/sdo
/dev/sg22  3 0 2 2  0  /dev/sdp
/dev/sg23  3 0 2 3  0  /dev/sdq
/dev/sg24  3 0 2 4  0  /dev/sdr

List the multipath devices:

[root@host1 ~]# multipath -l
mpath2 (36005076304ffc21f000000000000111f) dm-0 IBM,2107900
'               '                            '          '-- Vendor,Product 
'               '                            '------------- device-mapper or 
'               '                                           disk name
'               '
'               '------------------------------------------ WWID 
'
'---------------------------------------------------------- user friendly 
                                                            name
[size=100G][features=1 queue_if_no_path][hwhandler=0][rw]
'                     '                    '
'                     '                    '--------------- hardware handler, 
'                     '                                     if any (seen in 
'                     '                                     cases of FastT,EMC)  
'                     '
'                     '------------------------------------ features supported 
'                                                           or configured
'
'---------------------------------------------------------- Size of the disk

\_ round-robin 0 [prio=0][active]
 '      '          '       '------------------------------- Path Group State
 '      '          '--------------------------------------- Path Group Priority
 '      '
 '      '-------------------------------------------------- Path Selector and 
 '                                                          repeat count
 '
 '--------------------------------------------------------- Path Group Level

 \_ 4:0:0:0 sdc 8:32  [active][ready]
    ------- --- ----  ------- ------
      '      '   '       '     '--------------------------- Physical Path State
      '      '   '       '--------------------------------- Device Mapper State
      '      '   '----------------------------------------- Major, Minor number 
      '      '                                              of disk
      '      '--------------------------------------------- Linux SCSI device name
      '
      '---------------------------------------------------- SCSI Information: 
                                                            Host_ID, Channel_ID, 
                                                            SCSI_ID, LUN_ID

 \_ 4:0:1:0 sdg 8:96  [active][ready]
 \_ 3:0:1:0 sdk 8:160 [active][ready]
 \_ 3:0:2:0 sdo 8:224 [active][ready]

mpath6 (36005076304ffc21f0000000000001123) dm-3 IBM,2107900
[size=100G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=0][active]
 \_ 4:0:0:4 sdf 8:80  [active][ready]
 \_ 4:0:1:4 sdj 8:144 [active][ready]
 \_ 3:0:1:4 sdn 8:208 [active][ready]
 \_ 3:0:2:4 sdr 65:16 [active][ready]
mpath5 (36005076304ffc21f0000000000001122) dm-2 IBM,2107900
[size=1.0G][features=0][hwhandler=0][rw]
\_ round-robin 0 [prio=0][enabled]
 \_ 4:0:0:3 sde 8:64  [active][ready]
 \_ 4:0:1:3 sdi 8:128 [active][ready]
 \_ 3:0:1:3 sdm 8:192 [active][ready]
 \_ 3:0:2:3 sdq 65:0  [active][ready]
mpath4 (36005076304ffc21f0000000000001121) dm-1 IBM,2107900
[size=100G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=0][active]
 \_ 4:0:0:2 sdd 8:48  [active][ready]
 \_ 4:0:1:2 sdh 8:112 [active][ready]
 \_ 3:0:1:2 sdl 8:176 [active][ready]
 \_ 3:0:2:2 sdp 8:240 [active][ready]

The block device name is listed as the Linux SCSI device name. If there are multiple paths, multiple block devices are displayed under each pseudo name.

Note: The Db2 installer takes the DM-MP device provided to the IIH (Installation-Initiating Host) and depends on its WWID to find the storage volumes on the remote hosts. There is no requirement for the DM-MP devices to have the same disk names (dm-#) across all hosts.

On systems with EMC PowerPath driver, these are the commands to run, and sample output:

Run the powermt command to display all path and device mappings. This command lists the block devices and paths which are mapped to the device path (for example, /dev/emcpowerd of which the EMC pseudo name is emcpowerd):

host1:~ # powermt display dev=all
Pseudo name=emcpowerd
Symmetrix ID=000194900547
Logical device ID=0040
state=alive; policy=BasicFailover; priority=0; queued-IOs=0
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
   3 qla2xxx                   sdg       FA  7eB   active  alive      0      0

Pseudo name=emcpowerc
Symmetrix ID=000194900547
Logical device ID=0041
state=alive; policy=BasicFailover; priority=0; queued-IOs=0
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
   3 qla2xxx                   sdh       FA  7eB   active  alive      0      0

Pseudo name=emcpowerb
Symmetrix ID=000194900547
Logical device ID=0126
state=alive; policy=BasicFailover; priority=0; queued-IOs=0
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
   3 qla2xxx                   sdi       FA  7eB   active  alive      0      0

Pseudo name=emcpowera
Symmetrix ID=000194900547
Logical device ID=013C
state=alive; policy=BasicFailover; priority=0; queued-IOs=0
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
   3 qla2xxx                   sdj       FA  7eB   active  alive      0      0

The block device name is listed under I/O Paths column. If there are multiple paths, multiple block devices are displayed under each pseudo name.

For RDMA (InfiniBand) or RoCE) network on SLES, ensure that the openibd service is enabled.

# chkconfig --list | grep -i openibd
openibd 0:off 1:off 2:off 3:off 4:off 5:off 6:off

The openibd service must be enabled. To enable the service:

# chkconfig openibd on
# chkconfig --list | grep -i openibd
openibd 0:off 1:off 2:on 3:on 4:on 5:on 6:off

For a RDMA (InfiniBand or RoCE) network on RHEL, ensure that the rdma service is enabled:

chkconfig --list | grep -i rdma
rdma 0:off 1:off 2:off 3:off 4:off 5:off 6:off

The rdma service must be enabled. To enable the service:

# chkconfig rdma on
# chkconfig --list | grep -i rdma
rdma 0:on 1:on 2:on 3:on 4:off 5:off 6:off

Confirm that the required network adapters are installed on each server. Ensure that an Ethernet network (eth0) and an InfiniBand network (ib0) or RoCE network (eth1) display. This sample uses the netstat -i command to list all available network adapters with an InfiniBand communication adapter port.
```
root@host1:/> netstat -i 
Iface   MTU Met   RX-OK RX-ERR RX-DRP RX-OVR   TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0   1500   0 6876034      0      0      0 5763121      0      0      0 BMRU
ib0   65520   0  106972      0      0      0       9      0      0      0 BMRU
lo    16436   0  180554      0      0      0  180554      0      0      0 LRU
```
Note: The Db2 pureScale Feature does not support a mixed environment of InfiniBand and RoCE networks, all servers must use the same type of communication adapter port.
As root, ensure that the /tmp and other local host directories has the required amount of free space (as listed in Installation prerequisites for Db2 pureScale Feature (Intel Linux). The following command shows the free space in the /tmp directory.
```
$ cd /tmp
$ df -k . 
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda7 2035606 164768 1870838 9% /tmp
```
Optional. If you are doing a root installation of the Db2 pureScale Feature, you must set the ulimit value of filesize to unlimited. You must also set the value of umask to 022. If you do not set the values of ulimit and umask correctly, your Db2 pureScale Feature installation might fail.
You can view the current values of ulimit and umask by issuing this command:
```
id root; ulimit -f; umask
```
You must have root authority to use these commands.
An Db2 pureScale instance requires specific users and groups, including fenced users. If you are installing using the Db2 Setup wizard, you can create the users before starting the wizard or have the wizard create them for you as you progress through the panels. If you are not creating or modifying instances you can create the required users after completing the installation.
Optional. To use a specific set of ports, ensure that the ports are free on all hosts. Otherwise, the installer selects the unique set of ports across all hosts. The Fast Communications Manager (FCM) requires a port range of the three mandatory ports plus the value provided for the logical members field. This port range can designate up to 130 hosts (128 members + 2 cluster caching facilities.) The default FCM start port is 60000 and must be in the range 1024 - 65535. In addition, two ports are required for cluster caching facilities. These two ports are chosen automatically.
Use the grep command on the /etc/services file to ensure that a contiguous range of ports is available.
Optional. For Db2 managed IBM Spectrum Scale installations, verify the remote shell and remote file copy settings default to db2locssh and db2scp. For example:
```
/usr/lpp/mmfs/bin/mmlscluster
  Remote shell command:      /var/db2/db2ssh/db2locssh
  Remote file copy command:  /var/db2/db2ssh/db2scp
```
Refer to the Db2 cluster services port usage information section in Configuring communication adapter ports and switches (Linux) section, ensure that all RSCT and Spectrum Scale mandatory ports are available on the hosts. If firewall is set up in the cluster, the relevant ports must be opened.
Refer to Screening router firewalls section, ensure relevant TCP/IP ports are opened in the firewall of each host to allow communications among all pureScale hosts.

Using the Db2 Setup wizard

To install the Db2 pureScale Feature, you must know these items. You can enter your values for each of these steps in the preinstallation cheat sheet section that follows.

The name of the directory where you want to install the Db2 pureScale Feature.
The device path for the shared disk partition where the Db2 database and the shared configuration files of the Db2 pureScale instance are stored by default. You should use the Db2 cluster file system.
If using a Db2 cluster services tiebreaker, the device path for the tiebreaker partition. This partition must be different from the shared disk partition specified earlier.
The Db2 cluster services tiebreaker is used in a recovery scenario when your Db2 pureScale instance is split in half due to a communication failure. Db2 cluster services use the tiebreaker to determine which half of the Db2 pureScale instance will remain online. This partition must be at least 25 MB in size and accessible from all hosts.
The host name, short host name, or IP address for each of the hosts you want to participate in the Db2 pureScale environment.
Optionally, you can specify the names of the hosts you want to set up as cluster caching facilities rather than accept the default choices provided by the Db2 Setup wizard. You can manually select which hosts act as cluster caching facilities, or have the Db2 installer assign them.

Preinstallation cheat sheet

Enter the appropriate required item value in the "Your Value" field.

Table 1. Preinstallation cheat sheet
Required Item	Your Value	Example
Instance owner/group name		`db2sdin1`/`db2iadm1`
Fenced user/group name		`db2sdfe1`/`db2fadm1`
Installation directory name		/opt/IBM/db2/11.1 Note: Ensure that the installation directory has the following permissions: drwxr-xr-x. Example: /opt, ibm, db2, and V11.5 need to have the following permissions: drwxr-xr-x.
Shared file system disk		/dev/dm-0
Hosts to include		`db2_host01` - `db2_host04`.
Netname interconnect for each member and CF		InfiniBand network example: `db2_<hostname>-ib0` RoCE network example: `db2_<hostname>_en1` TCP/IP network example: `db2_<hostname>_en1` Note: `db2_<hostname>_en1` does not map to a regular Ethernet adapter. It must map to the pseudo IP address for the RoCE communication adapter port. For multiple RoCE adapters configuration, ensure that the third octet of the pseudo IP address to all RoCE adapters on the same host are different. For example, `9.43.1.40 test-en0 9.43.2.40 test-en1 9.43.3.40 test-en2 9.43.4.40 test-en3` For multiple TCP/IP adapters configuration, ensure that all adapter ports on the same member and CF are bonded to form a single network interface.

Table 2. Preinstallation cheat sheet - optional items
Optional Item	Your Value	Example
Tiebreaker disk		On AIX®: /dev/hdisk13 On Linux: /dev/dm-0 or /dev/sdc
FCM port range		`60000` - `60004`
cluster caching facilities port range		`56000` - `56001`
Db2 communication port		`50001`
Hosts to set up as cluster caching facilities		`db2_host03` and `db2_host04`
On InfiniBand, the cluster interconnect netnames of the cluster caching facilities		Primary: `db2_<hostname1>-ib0`,`db2_<hostname1>_ib1`,`db2_<hostname1>_ib2,db2_<hostname1>_ib3` Secondary: `db2_<hostname2>-ib0`,`db2_<hostname2>_ib1`,`db2_<hostname2>_ib2,db2_<hostname2>_ib3`
On a RoCE network, the cluster interconnect netnames of the cluster caching facilities		Primary: `db2_<hostname1>_en1`,`db2_<hostname1>_en2`,`db2_<hostname1>_en3,db2_<hostname1>_en4` Secondary: `db2_<hostname2>_en1`,`db2_<hostname2>_en2`,`db2_<hostname2>_en3,db2_<hostname2>_en4`
On a TCP/IP network, the cluster interconnect netnames of the cluster caching facilities		Primary: `db2_<hostname1>_en1` Secondary: `db2_<hostname2>_en1`
Hosts to set up as members		`db2_host01` and `db2_host02`

What to do next

When you completed all the steps in the preinstallation checklist and filled out the cheat sheet, you can check and verify you meet the installation prerequisites by running the db2prereqcheck command.