Upgrading GPFS components with the spectrumscale installation toolkit

You can use the spectrumscale installation toolkit to upgrade all components (GPFS™, NFS, Object, SMB, and the Performance Monitoring tool).

Important: If you have all protocols enabled, plan on a system down time of approximately 20 minutes per protocol node.

Before using the spectrumscale installation toolkit to perform upgrades, do the following:

Go to the IBM Spectrum Scale™ page on Fix Central, select the new spectrumscale package and then click Continue.
Choose the download option Download using Download Director to download the new spectrumscale package and place it in the wanted location on the install node.
Note: Although it is not recommended but if you must use the download option Download using your browser (HTTPS), instead of clicking the down arrow to the left of the package name, you must right-click on the package name and select the Save Link As.. option. If you click the download arrow, the browser might hang.
Extract the new spectrumscale self-extracting package by running the package name (for example, /tmp/Spectrum_Scale_Protocols_Standard-4.2.0.0_x86_64-Linux_install).
This creates a new directory structure (/usr/lpp/mmfs/4.2.0.0/).
Accept the license agreement. The system will guide you through next steps including:
- cluster installation and protocol deployment
- upgrading an existing cluster using the install toolkit
- adding nodes to an existing cluster using the install toolkit
- adding NSDs or file systems to an existing cluster using the install toolkit
If you are upgrading protocol nodes you need to stop the Performance Monitoring tool Object proxy service. Depending on the version of the Performance Monitoring tool you have installed, it will either be pmprovider.service or pmswiftd.service.
They can be stopped by issuing the following commands:
- systemctl stop pmswiftd.service
- systemctl stop pmprovider.service
Determine which of the following situations applies to you:
- If you already have GPFS installed and did not use the toolkit to do so, but you would like to upgrade using the toolkit, follow these steps:
  1. Configure your cluster definition file as explained in the following topics:
    - Defining configuration options for the spectrumscale installation toolkit
    - Setting up the install node
  2. Proceed to the "spectrumscale upgrade command" section that follows.
- If you have GPFS installed and have already used the install toolkit, follow these steps:
  1. Copy the cluster definition file from your /usr/lpp/mmfs/4.1.1.x/installer/configuration/clusterdefinition.txt file to the newly extracted /usr/lpp/mmfs/4.2.0.0/installer/configuration/clusterdefinition.txt directory.
  2. Ensure that the cluster definition file is accurate and reflects the current state of your Spectrum Scale cluster.
  3. Proceed to the "spectrumscale upgrade command" section that follows.
- If you have no existing GPFS cluster and would like to use the install toolkit to install both GPFS and an update, follow these steps:
  1. Copy the /usr/lpp/mmfs/4.2.0.0/installer directory (for example, /usr/lpp/mmfs/4.2.0.0/installer over the top of the /usr/lpp/mmfs/4.2.0.0/installer directory.
  2. Complete defining your cluster definition file as explained in the following topics:
    - Defining configuration options for the spectrumscale installation toolkit
    - Setting up the install node
  3. Run the spectrumscale install command to install GPFS. (See Installing GPFS.)
  4. Copy the configuration/clusterdefinition.txt file from the location where you just ran the spectrumscale install command to 4.2.0.0/installer/configuration (for example, 4.2.0.0/installer/configuration).
  5. Add nodes to ensure that the gpfs.base rpm is installed.
  6. Proceed to upgrade GPFS (for example, to 4.x.x.x); see the "The spectrumscale upgrade command" section that follows.
  7. Proceed to deploy protocols (if required) as explained in Deploying protocols.

The spectrumscale upgrade command

The ./spectrumscale upgrade command is used to perform upgrades and has these options:

-ve: Shows the current version of install packages and the available version to upgrade to.
-pr: Performs health checks and ensures all prerequisites are met on the cluster prior to upgrade.
-po: Performs health checks on the cluster after the upgrade has been completed.
-f: Bypasses the message that will appear when upgrading components, that warns that there may be an outage when upgrading SMB and the performance monitoring tool.

As with all spectrumscale commands, use the -v flag for verbose output. The -v flag must be placed immediately after ./spectrumscale.

For example: ./spectrumscale -v upgrade

Note: The upgrade command can be used to upgrade GPFS from version 4.1.0 and later.

Running upgrade prechecks

For any cluster with existing Performance Monitoring collectors, the upgrade prechecks might find that a reconfiguration is required. To enable the install toolkit to perform the reconfiguration, issue the following command:

./spectrumscale config perfmon -r on

To keep your existing algorithms, issue the following command:

./spectrumscale config perfmon -r off

Before upgrading, run upgrade prechecks by issuing the following command:

/usr/lpp/mmfs/4.2.0.x/installer/spectrumscale upgrade -pr

Upgrading all components

Note: Before upgrading all components, run the upgrade precheck command.

All components can be upgraded at once using the following command:

./spectrumscale upgrade

This command can still be used even if all protocols are not enabled; however, the packages for other protocols will be upgraded also (but not started).

This will cause a brief outage of the performance monitoring tool and SMB (if enabled). Read the following sections for more information about the impact of upgrade on each component.

Once upgrade is complete, run the spectrumscale upgrade -ve command to show the versions of the packages installed to ensure that they are all at the desired level.

Proceed to the next step, migration to a new level of GPFS. See this topic for instructions Completing the migration to a new level of IBM Spectrum Scale.

Failure recovery

Should a failure occur during an upgrade, examine the log at the location provided in the output from the install toolkit to determine the cause. More information on each spectrumscale error will be provided in the output from the install toolkit.

Certain failures will cause the upgrade process to stop. In this case, the cause of the failure should be addressed on the node on which it occurred. Once the problem has been addressed, the upgrade command can be run again. This will not affect any nodes that were upgraded successfully and will continue once the point of failure has been reached.

An example of a failure during upgrade is:

A protocol is enabled in the configuration file, but is not running on a node.: Using the mmces service list command on the node that failed will highlight which process is not running (the output from the install toolkit will also report which component failed). Make sure the component is started on all nodes with mmces service start nfs | smb | obj, or alternatively disable the protocol in the configuration using spectrumscale disable nfs|smb|object if the component has been intentionally stopped.

Start of change For information on troubleshooting installation. deployment, and upgrade related issues, see Resolving most frequent problems related to installation, deployment, and upgrade in IBM Spectrum Scale: Problem Determination Guide. End of change

Upgrade considerations for IBM Spectrum Scale for object storage

If object authentication is configured with AD or LDAP and anonymous bind is not used during object authentication configuration, use the following command after upgrading IBM Spectrum Scale for object storage from 4.1.x to 4.2.x:
```
mmobj config change --ccrfile keystone.conf --section ldap --property password --value ldapbindpassword
```
After upgrading IBM Spectrum Scale for object storage, some migration steps need to be performed to enable features such as compression for existing file systems. For information about these migration steps, see Completing the migration to a new level of IBM Spectrum Scale.
If the MAX_FILE_SIZE parameter is not changed in your setup, verify that its value is correct (5 TiB) and change it if required.
1. Verify if the value of the MAX_FILE_SIZE parameter is 5497558138882 using the following command:
```
mmobj config list --ccrfile swift.conf --section 'swift-constraints' \
--property max_file_size
```
  The system displays the following output:
```
max_file_size = 5497558138882
```
2. If MAX_FILE_SIZE is not correct (5 TiB), set MAX_FILE_SIZE using the following command:
```
mmobj config change --ccrfile swift.conf --section 'swift-constraints' \
--property max_file_size --value 5497558138882
```