IBM Support

Troubleshooting and manual installation of the ESX/ESXi kernel module (VIB) for Virtual Server Protection for VMware

Technote (troubleshooting)


How to manually install the Virtual Server Protection module and confirm its presence.


IBM Security Virtual Server Protection for VMware (Virtual Server Protection agent) uses a kernel module loaded on the ESX/ESXi host. This module plays a role in firewall and IPS function and is required for Virtual Server Protection agent operation. VMware calls this type of module a "fastpath" module. The installation package containing the fastpath module is known as a VIB ("vSphere Installation Bundle").

Under normal circumstances, functions in the Virtual Server Protection agent's provVSetup menu manage installing and removing the module on the host. In some cases, such as certain upgrade scenarios, certain patches containing an updated fastpath module, or any time there is a problem with the module, manual steps might be needed to ensure the proper module version is installed and properly loaded on the host. This article details these steps.

Diagnosing the problem

While logged in as root user (or equivalent), execute commands from the ESX/ESXi command line. Commands executed in the Virtual Server Protection agent also require the root account. Commands are case sensitive.

Note: Some commands used for ESX/ESXi 4.1 are different from those used for ESXi 5.0. It is expected that the information for ESXi 5.0 will also be valid for ESXi 5.1. Additionally, the actual module used for the 4.1 versions is different than the module used for the 5.x versions.

  1. To check the current state of the module, on the Virtual Server Protection agent, run the following command:

    grep Version /var/iss/engine1.log

    This output indicates the version of the loaded fastpath module on the host (the VSP fastpath module is named ibm-iss-vmkmod) and the version of the IPS engine running within the Virtual Server Protection agent. Expected output looks like this:

    2012/08/23 11:44:40.457 T:3083393728 Version of ibm-iss-vmkmod: []  Version of IPS Engine: []

    This output indicates that the fastpath module is loaded on the ESX(i) host and its version matches that of the IPS engine. If you are performing an action involving an upgrade to the module, you can compare the output before and after the upgrade to confirm the updated module is loaded.

    Note: If the above grep command does not produce any output at all, it is possible that the engine1.log file has been rotated and no longer contains the relevant entry, which is only logged when the IPS engine starts. Issue the following command to restart the services, and then after waiting several minutes, re-run the grep command.

    service issDaemon restart

  2. On the ESX(i) host, check for the following two items:
    • The installed state of the VIB
    • Whether or not the module is actually loaded.

  3. To check the VIB state on ESX(i) 4.1:

    esxupdate --vib-view query | grep ibm-iss-vmkmod

    Output indicating the module is installed:

    cross_ibm-iss-vmkmod_400.                                           installed     2012-08-29T13:42:58.239760-04:00

    If the module is not installed, the output will be blank. If the module has been installed previously, but now is not, it is expected to be listed as either "uninstalled" or "retired".

    Note: If the state includes the term "staged", for example "staged, installed", then the host must be restarted after which the state should become permanent (no longer "staged"). This is not expected to happen following the steps in this document.

    On ESX only (not ESXi), you might see more than one version of the module installed at the same time. This will occur after an upgrade from VSP 1.1 to If this is the case, you must uninstall all instances of the module if you need to remove it.

  4. To check the VIB state on ESXi 5.0:

    esxcli software vib list | grep ibm-iss-vmkmod

    Output indicating the module is installed:

    proventiaServerV-ibm-iss-vmkmod5.0                        IBM     PartnerSupported  2012-08-22

  5. To check if the module is loaded (for both ESX/ESXi 4.1 and 5.0):

    vmkload_mod -l | grep ibm-iss-vmkmod

    Expected output if the module is loaded:

    ibm-iss-vmkmod           1    104

    Note: If the module is not loaded, then there will be no output.

Resolving the problem

The following steps include the procedure for removing any installed VIB, followed by the manual installation of a new VIB. For this procedure, suspend or power off any active virtual machines on the host or migrate them to another host.

If this procedure is performed in conjunction with a Virtual Server Protection patch installation or upgrade that includes an updated fastpath module, ensure the Virtual Server Protection agent update is installed first. This ensues that the updated VIB is in place to be copied from VSP to the ESX(i) host.

  1. Copy the VIB from VSP to the host for manual installation at the following location:

    /etc/iss/drivers/proventiaServerV-ibm-iss-vmkmod4.1.vib (for ESX/ESXi 4.1)
    /etc/iss/drivers/proventiaServerV-ibm-iss-vmkmod5.0.vib (for ESXi 5.0)

    This can be done using SCP if the host has SSH enabled and permits root logins with SSH. If there are restrictions preventing that, you can use the datastore browser within the vSphere client to upload the VIB file to a datastore accessible by the host.

  2. On the ESX(i) host, verify the installed state of the VIB (see previous section).

  3. If the VIB is installed, remove it.

    On ESX(i) 4.1:

    esxupdate remove -b `esxupdate --vib-view query | egrep "ibm-iss-vmkmod.*[ \t]installed" | awk '{print $1}'`

    On ESXi 5.0:

    esxcli software vib remove -n IBM:proventiaServerV-ibm-iss-vmkmod5.0

  4. Remove any remaining DVFilter IP address from esx.conf on the host using this command:

    sed -i -e '/DVFilterBindIpAddress/d' /etc/vmware/esx.conf

  5. Restart the ESX(i) host

  6. Using the commands in the previous section, verify that the VIB is no longer installed and that the module is not loaded. If either one is present, return to Step 3, and run the uninstallation command two times before proceeding.

  7. Install the VIB on the host (assuming for this example the VIB was updated to the root of "datastore1"):

    For ESX(i) 4.1:

    esxupdate update -b /vmfs/volumes/datastore1/proventiaServerV-ibm-iss-vmkmod4.1.vib --nosigcheck

    For ESXi 5.0:

    esxcli software vib install --viburl=file:/vmfs/volumes/datastore1/proventiaServerV-ibm-iss-vmkmod5.0.vib --no-sig-check

    Note: ONLY if you have installed a patch for the Virtual Server Protection agent containing a VMware Certified kernel module, omit the "--nosigcheck" / "--no-sig-check" option.

  8. On the ESX(i) host, verify the installed state of the VIB (see previous section). The VIB should be installed and the ibm-iss-vmkmod module should be loaded.

  9. Start the Virtual Server Protection agent and verify the output of the following command:

    grep vmkmod /var/iss/engine1.log

    This will show the versions of IPS engine and the fastpath module. They will normally match. Output should be similar to this:

    2012/08/22 17:02:58.346 T:3083586240 Version of ibm-iss-vmkmod: []  Version of IPS Engine: []

Document information

More support for: IBM Security Virtual Server Protection for VMware

Software version:

Operating system(s): Firmware

Reference #: 1610899

Modified date: 14 September 2012