WebSphere MQ Linux uninstallation of v7.1/7.5 GSKIT fix pack package changes the entire system filesystem's file ownership and permissions.

Content

******************************************************************************************************
Users affected

This document applies to users of WebSphere MQ v7.1 or v7.5, who have installed the MQSeriesGSKit package from a WebSphere MQ fixpack, without installing the corresponding MQSeriesGSKit package from the main WebSphere MQ product installer.

Users who are unsure if their installation is affected can download and run the tool supplied below to determine if they need to take preventative action.

******************************************************************************************************

On all architectures of the Linux platform (x86, x86-64, PowerPC, zLinux), it is possible to have the WebSphere MQ package installation configuration in such as state that when the GSKit component is removed, the product installation scripts attempt to change the attributes of all files and directories on the system in following irreversible way:

1. Change the ownership of all files and directories to be owned by the user 'mqm' and the group 'mqm. This is the equivalent of the command:
   chown -h -R mqm:mqm /
   
2. Remove the write attribute from all files and directories.
   This is the equivalent of the command:
   chmod -R u-w,g-w,o-w /

This occurs due to a defect in the post-uninstallation script of the GSKit package of the fix pack packages, which have a package name of the following form:

MQSeriesGSKit-Uxxxxxx

where 'xxxxxx' is replaced by a 6-digit number, which corresponds to a specific platform architecture and fix pack version:

Linux (x86)
7.1.0.1 : MQSeriesGSKit-U850304-7.1.0-1.i386.rpm
7.1.0.2 : MQSeriesGSKit-U853049-7.1.0-2.i386.rpm
7.5.0.1 : MQSeriesGSKit-U200355-7.5.0-1.i386.rpm

Linux (x86-64)
7.1.0.1 : MQSeriesGSKit-U850308-7.1.0-1.x86_64.rpm
7.1.0.2 : MQSeriesGSKit-U853053-7.1.0-2.x86_64.rpm
7.5.0.1 : MQSeriesGSKit-U200357-7.5.0-1.x86_64.rpm

Linux (PowerPC)
7.1.0.1 : MQSeriesGSKit-U850306-7.1.0-1.ppc64.rpm
7.1.0.2 : MQSeriesGSKit-U853051-7.1.0-2.ppc64.rpm
7.5.0.1 : MQSeriesGSKit-U200358-7.5.0-1.ppc64.rpm

Linux (zLinux)
7.1.0.1 : MQSeriesGSKit-U850305-7.1.0-1.s390x.rpm
7.1.0.2 : MQSeriesGSKit-U853050-7.1.0-2.s390x.rpm
7.5.0.1 : MQSeriesGSKit-U200356-7.5.0-1.s390x.rpm


This defect is documented further under APAR IC93233:
http://www.ibm.com/support/docview.wss?uid=swg1IC93233



The installation configuration state for which this problem occurs is when the fix pack package has been installed onto a system, where the base (non-fix pack) package has not been installed for the GSKit component.

For example, consider a Linux (x86-64) system which is installed with three WebSphere MQ components, the Runtime, Server and JRE. Listing the 'MQSeries' packages that are in the package database shows:

# rpm -qa | grep MQSeries
MQSeriesRuntime-7.5.0-0.x86_64
MQSeriesServer-7.5.0-0.x86_64
MQSeriesJRE-7.5.0-0.x86_64


If the the system then has the 7.5.0.1 fix pack applied for the Runtime, Server, JRE and GSKit components, it becomes susceptible to the problem:

# rpm -i MQSeriesRuntime-U200357-7.5.0-1.x86_64.rpm \
         MQSeriesServer-U200357-7.5.0-1.x86_64.rpm \
         MQSeriesJRE-U200357-7.5.0-1.x86_64.rpm \
         MQSeriesGSKit-U200357-7.5.0-1.x86_64.rpm


After this command has completed, the system has the following set of packages installed:

# rpm -qa | grep MQSeries
MQSeriesJRE-U200357-7.5.0-1.x86_64
MQSeriesRuntime-7.5.0-0.x86_64
MQSeriesServer-7.5.0-0.x86_64
MQSeriesJRE-7.5.0-0.x86_64
MQSeriesServer-U200357-7.5.0-1.x86_64
MQSeriesGSKit-U200357-7.5.0-1.x86_64
MQSeriesRuntime-U200357-7.5.0-1.x86_64


Note that the MQSeriesGSKit fix pack package is present (highlighted in the above list), but there is no corresponding base package, that is to say there is no MQSeriesGSKit package installed without a 'U-number'.

If the GSKit 7.5.0.1 FixPack fileset is then removed from this system, the post-uninstall script will attempt to recursively modify the entire root ('/') filesystem, altering the ownership and file write attributes. As some files on the filesystem cannot have their file permissions altered, error messages of the following form may have been observed when this package is removed:

# rpm -e MQSeriesGSKit-U200357-7.5.0-1.x86_64
/var/tmp/rpm-tmp.E7x8VE: line 140: cd: /opt/mqm/gskit8: No such file or directory
chown: changing ownership of `./proc/sys/kernel/sched_child_runs_first': Operation not permitted
chown: changing ownership of `./proc/sys/kernel/sched_min_granularity_ns': Operation not permitted
chown: changing ownership of `./proc/sys/kernel/sched_latency_ns': Operation not permitted
... ... ...



The error messages seen after the first error "cd: /opt/mqm/gskit8: No such file or directory" will vary. The effects of this problem can be seen by listing the root filesystem after uninstalling the fix pack fileset:

# ls -la /
total 148
dr-xr-xr-x.  28 mqm mqm  4096 Jun 24 17:26 .
dr-xr-xr-x.  28 mqm mqm  4096 Jun 24 17:26 ..
dr-xr-xr-x.   2 mqm mqm  4096 Jun 24 16:48 bin
dr-xr-xr-x.   4 mqm mqm  4096 Jun 24 15:31 boot
drwxr-xr-x.  19 mqm mqm  3680 Jun 24 15:51 dev
dr-xr-xr-x. 125 mqm mqm 12288 Jun 26 10:02 etc
... ... ...



The problem occurs for two reasons:

1. It should not have been possible to install the GSKit FixPack RPM package onto a system which did not have the base 7.1.0.0/7.5.0.0 version of the RPM already installed on the system. It was intended that the package prerequisite checking prevented this invalid installation configuration.
   
   
2. During the post-uninstall script for the GSKit FixPack RPM, an attempt was made to change to a directory which did not exist as the base GSKit RPM package is not installed. The success of this change directory command was not checked, resulting in a recursive 'chmod' and 'chown' operation running from the root filesystem.

**** WARNING ****
IF YOUR SYSTEM IS CURRENTLY IN THIS CONFIGURATION, DO NOT REMOVE THE GSKIT PACKAGE WITHOUT FIRST UNDERSTANDING THE CONSEQUENCES.

You can determine if you are susceptible to this problem by examining your installed packages, prior to removing the GSKit component.

For WebSphere MQ v7.5, you are looking to see if your system has WebSphere MQ 7.5.0.0 installed without the GSKit component, but has the WebSphere MQ 7.5.0.1 fix pack applied including the GSKit component. This can be determined by one result being returned from the following RPM query command:

# rpm -qa | grep MQSeriesGSKit
MQSeriesGSKit-U200357-7.5.0-1.x86_64


Removing this package for the system in this state will result in the system's file permissions and ownership being modified.

When the above RPM query is run, a system which is not susceptible to the problem will return more than one result, one of which includes is the base package name, the package which does not have the "Uxxxxxx" component in its name, as in the following example:

# rpm -qa | grep MQSeriesGSKit
MQSeriesGSKit-7.5.0-0.x86_64
MQSeriesGSKit-U200357-7.5.0-1.x86_64


For WebSphere MQ v7.1, it is slightly more complicated to determine if the system is susceptible, because WebSphere MQ v7.1.0.1 was delivered both as a manufacturing refresh and a fix pack, and that there have been two fix packs which have this defect in. The output from the above RPM query will depend upon which version of v7.1.0.1 you have installed, the manufacturing refresh or the fix pack.

Again, you are looking to find the presence of the MQSeriesGSKit fix pack package, the name with the U-number, without the base package. For example, the following queries all return results which indicate a susceptible system:

# rpm -qa | grep MQSeriesGSKit
MQSeriesGSKit-U850308-7.1.0-1.x86_64


# rpm -qa | grep MQSeriesGSKit
MQSeriesGSKit-U853053-7.1.0-2.x86_64.rpm


# rpm -qa | grep MQSeriesGSKit
MQSeriesGSKit-U850308-7.1.0-1.x86_64.rpm
MQSeriesGSKit-U853053-7.1.0-2.x86_64.rpm



The following queries return results which indicate a system which is not susceptible to the problem:

# rpm -qa | grep MQSeriesGSKit
MQSeriesGSKit-7.1.0-0.x86_64
MQSeriesGSKit-U850308-7.1.0-1.x86_64


# rpm -qa | grep MQSeriesGSKit
MQSeriesGSKit-7.1.0-0.x86_64
MQSeriesGSKit-U853053-7.1.0-2.x86_64.rpm


# rpm -qa | grep MQSeriesGSKit
MQSeriesGSKit-7.1.0-0.x86_64
MQSeriesGSKit-U850308-7.1.0-1.x86_64.rpm
MQSeriesGSKit-U853053-7.1.0-2.x86_64.rpm


# rpm -qa | grep MQSeriesGSKit
MQSeriesGSKit-7.1.0-1.x86_64
MQSeriesGSKit-U853053-7.1.0-2.x86_64.rpm


# rpm -qa | grep MQSeriesGSKit
MQSeriesGSKit-7.1.0-1.x86_64



Note that if you have a system with multiple versions or instances of WebSphere MQ, then the results returned by the RPM query will vary to include all the packages from each installation, and you will need to distinguish by version and the package name if your system is susceptible.

To help with this, the following shell script can be run to identify if the system is susceptible to the problem:

dspmqinfo_IC93233.sh

Download the script and put on the Linux system which has the WebSphere MQ installation. Make the script executable, and run using the commands:

chmod a+x dspmqinfo_IC93233.sh
./dspmqinfo_IC93233.sh

The script will assess the status of the WebSphere MQ installation(s), and provide output in the form of text in the terminal. If the script finds a potential problem, it will not rectify the problem, but will provide instructions on what to, utilising the "Solution 2" method detailed below.

For example, on a Linux x86-64 system which has the following RPM packages installed:

MQSeriesRuntime-7.1.0-0.x86_64
MQSeriesJRE-7.1.0-0.x86_64
MQSeriesGSKit-U853053-7.1.0-2.x86_64
MQSeriesRuntime-U853053-7.1.0-2.x86_64
MQSeriesJRE-U853053-7.1.0-2.x86_64


which represents a vulnerable configuration, running the script will show the following output:

# ./dspmqinfo_IC93233.sh
Querying with the package management database which WebSphere MQ
packages are installed on the system...

Located WebSphere MQ GSKit fix pack package: MQSeriesGSKit-U853053-7.1.0-2.x86_64
Searching for a base package of the form: MQSeriesGSKit-7.1

*******************************************************************************
WARNING: Unable to find a base package corresponding to the fix pack package:

MQSeriesGSKit-U853053-7.1.0-2.x86_64

This system may be susceptible to the problem described in APAR IC93233.

DO NOT UNINSTALL this package until you taken preventative measures!

Run the following command to take preventative measures:

touch /opt/mqm/gskit8/IC93233.fix

then run this script again to confirm the preventative measure(s) were taken.
*******************************************************************************





There are two alternative solutions to mitigate against this problem, should you find that your system is in this susceptible state:

• Solution 1 - Remove the package using --noscripts
  Use the '--noscripts' parameter option when uninstalling the package, for example on WebSphere MQ v7.5.0.1:
  
  rpm -e --noscripts MQSeriesGSKit-U200357-7.5.0-1.x86_64
  
  for the Linux (x86-64) platform.
  
  Using this option will prevent the post-uninstall script from running, which prevents the problem from happening. After using this option for this package, before attempting to reinstall the GSKit functionality, it is advised that the all the components of WebSphere MQ are removed and reinstalled, ensuring that when all of the WebSphere MQ packages associated with this installation have been removed, the /opt/mqm (or other location where the product was requested to be installed to) directory is either not present or empty.
  
  
  
• Solution 2 - Create a uniquely named file within the GSKit directory structure
  Creating a file within the WebSphere MQ installation in the directory structure that the MQSeriesGSKit package installed its files, prevents the problem from happening. For example, on a default location installation use the command:
  
  touch /opt/mqm/gskit8/IC93233.fix
  
  Then the MQSeriesGSKit fix pack package can be safely removed in the usual manner, upon which a message will be displayed by RPM to indicate that it could not remove the /opt/mqm/gskit8 directory:
  
  # rpm -e MQSeriesGSKit-U200357-7.5.0-1.x86_64
  rmdir: failed to remove `/opt/mqm/gskit8': Directory not empty
  
  
  This "Directory not empty" message is expected and can be safely ignored. The directory can be safely removed once the WebSphere MQ installation has been removed from the system.
  
  If the installation location has been changed, replace the "/opt/mqm" component of this 'touch' command with the alternative installation location.