Establishing an encryption-enabled environment

Establishing an encryption-ready environment requires the sequence of activities summarized here. This summary covers a basic setup with a single encrypted fileset.

Considerations on FIPS compliance

To retain compliance to FIPS 140-2, do the following:

Initial setup by administrator

During initial setup, perform the following steps:
  1. Install IBM® Security Key Lifecycle Manager (ISKLM) version 2.5.0.1 or later.

    Refer to the ISKLM installation guide.

    From the main page of the ISKLM web GUI, select Configuration and then Key Serving Parameters from the selector on the right. Then make sure that the check box next to Keep pending client device communication certificates is selected.

  2. Configure an SSL/KMIP server certificate on ISKLM.

    (Skip this step if you have already configured the SSL/KMIP server certificate).

    1. Click Configuration in the menu bar at the top, and select SSL/KMIP from the menu on the left.
    2. To create a new self-signed certificate, click the Create self-signed certificate radio button.
    3. Enter the appropriate details for the certificate to be created, and click OK.
  3. Export the SSL/KMIP server certificate from ISKLM.
    This step is required to obtain a trusted copy of the server certificate.
    1. Identify the certificate label for the SSL/KMIP server certificate currently in use.
    2. Click Advanced Configuration from the menu bar at the top, and select Server Certificates from the dropdown menu.
    3. Select the certificate identified as being "In Use," click Modify, and make note of the certificate label.
    4. From an administrator console on the ISKLM server, change the directory to the bin directory in WAS_HOME:
      • Windows systems:
        drive:\Program Files (x86)\IBM\WebSphere\AppServer\bin
      • Other systems such as Linux:
        /opt/IBM/WebSphere/AppServer/bin
    5. Type the following:
      • On Windows systems:
        wsadmin -username SKLMAdmin -password mypwd -lang jython
      • On other systems such as AIX® or Linux:
        ./wsadmin.sh -username SKLMAdmin -password mypwd -lang jython
    6. Identify the correct UUID of the certificate:
      1. First issue the following print command, replacing YOUR_LABEL with the certificate label obtained in the previous step:
        print AdminTask.tklmCertList('[-alias YOUR_LABEL]')
        The system will respond with output similar to the following:
        CTGKM0001I Command succeeded.
        uuid = CERTIFICATE-7005029a-831d-405f-af30-4bf0177909de
        alias = server
        key store name = defaultKeyStore
        key state = ACTIVE
        issuer name = CN=server
        subject name = CN=server
        creation date = 13/03/2014 16:27:13 Eastern Daylight Time
        expiration date = 09/03/2015 07:12:30 Eastern Daylight Time
        serial number = 1394363550
      2. Reissue the print command, this time specifying the UUID identified during the previous step:
        print AdminTask.tklmCertExport('[-uuid
        CERTIFICATE-7005029a-831d-405f-af30-4bf0177909de -format base64 -fileName
        /root/srvcert]')

        The certificate will be exported in the file specified after -fileName (in this example, /root/srvcert).

    7. Copy the file into a temporary directory on the GPFS node you wish to configure for encryption (for example, to /tmp/srvcert).
  4. Create a new GPFS device group.
    1. From the main page of the ISKLM web GUI, select Advanced Configuration and then Device Group from the menu bar at the top.
    2. On the next page, click the Create button.
    3. In the Create Device Group window, select the GPFS device family and enter an appropriate name (for example, "GPFS_TENANT1"); make note of this name, as it will be required later on.
    4. Once the device group has been created, the system will prompt you to add devices and keys. However, the next steps in this procedure will generate keys explicitly, so now just click Close to return to the main page.
    5. On the main page, select the GPFS tenant device group, click Go to..., and select Manage keys and devices from the dropdown menu.
    6. On the next screen, click Add and select Key from the dropdown menu.
      The system prompts you to specify the following:
      • the number of keys to be created
      • the three-letter prefix to be added to their name

        The name is the ISKLM internal name and is not used for GPFS encryption.

      Make note of the key UUID (in the example, KEY-326a1906-be46-4983-a63e-29f005fb3a15), as it will be required later on.

    7. Enable the option Hold new certificate requests pending my approval by selecting it from the dropdown menu in the lower part of the page.
  5. Configure the RKM backend on GPFS.
    1. Create a key store containing the server certificate and a new key pair and certificate that will be used by the client.
      1. To do so, from an administrator console on the GPFS node you wish to be configured with encryption, create the subdirectory /var/mmfs/etc/RKMcerts.
      2. Issue the following command, replacing cname_test with a string identifying the GPFS node or the tenant performing encryption, label_test with a string that identifies the client certificate in the key store, and a_password with a password protecting the secret key material in the key store file:
        mmauth gencert --cname cname_test --label a_label --cert /tmp/srvcert --out
        /var/mmfs/etc/RKMcerts/ISKLM.p12 --pwd a_password

        Upon success, the new key store is created (for example, in /var/mmfs/etc/RKMcerts/ISKLM.p12).

        Note: Start of changeThe created key store must be record locked when the GPFS daemon starts. If the key store files are stored on an NFS mount, it is possible for the encryption initialization process to hang, due to a bug that affects the way NFS handles record locking. If this happens, upgrade your version of NFS or store your key store file on a local file system. If an upgrade is not possible and no local file system is available, use a RAM drive to store the key store files.End of change
      3. Open an editor to create the /var/mmfs/etc/RKM.conf file, as follows:
        ISKLM_srv {
        type = ISKLM
        kmipServerUri = tls://raclette.zurich.ibm.com:5696
        keyStore = /var/mmfs/etc/RKMcerts/ISKLM.p12
        passphrase = a_password
        clientCertLabel = a_label
        tenantName = GPFS_TENANT1
        }
        Choose an appropriate name for the RKM backend stanza (ISKLM_srv in the example) and ensure that the fields are filled in correctly:
        type
        Must be set to ISKLM.
        kmipServerUri
        Contains the DNS name or IP address of the ISKLM server and the KMIP SSL port (you can find the latter in the main page of the ISKLM web GUI; 5696 is the default).
        keyStore
        Contains the path to the key store file created in the previous step.
        passphrase and clientCertLabel
        Contain the password and label specified in the command line upon creation of the key store.
        tenantName
        Contains the name of the device group created in ISKLM.
  6. Set up an encryption policy.
    1. Create a policy, instructing GPFS to encrypt all files in the file system with an FEK and wrap the FEK with the MEK created previously; for example:
      RULE 'p1' SET POOL 'system' # one placement rule is required at all times
      
      RULE 'Encrypt all files in filesystem with rule E1'
            SET ENCRYPTION 'E1'
            WHERE NAME LIKE '%'
      
      RULE 'simpleEncRule' ENCRYPTION 'E1' IS
            ALGO 'DEFAULTNISTSP800131A'
            KEYS('KEY-326a1906-be46-4983-a63e-29f005fb3a15:ISKLM_srv')
    2. Create the Keyname string in the policy statement as follows:
      KeyID
      The key ID copied from the ISKLM web GUI.
      RkmID
      The name of the RKM backend stanza in the /var/mmfs/etc/RKM.conf file.
    3. After installing the rule with mmchpolicy, you need to import the client certificate into ISKLM. To do that, attempt the creation of an encrypted file to trigger a KMIP request, for example:
      [root@baden ~]# touch /gpfs0/test
      touch: cannot touch `/gpfs0/test': Permission denied
      [root@baden ~]# tail -n 2 /var/adm/ras/mmfs.log.latest
      Thu Mar 20 14:00:55.029 2014: [E] Unable to open encrypted file: inode 46088,
      Fileset fs1, File System gpfs0.
      Thu Mar 20 14:00:55.030 2014: [E] Error: key
      'KEY-326a1906-be46-4983-a63e-29f005fb3a15:ISKLM_srv' could not be fetched (RKM
      reported error -1004).

      The request will initially fail because ISKLM does not yet trust the client certificate; however, the certificate will be placed in a list of pending certificates.

    4. From the main page of the ISKLM web GUI, select Pending client device communication certificates.
    5. Identify the client certificate and click View.
    6. Carefully check that the certificate being imported matches the one created in the previous step, then click Accept and Trust
    7. On the resulting screen, give a name to the certificate, then click Accept and Trust.
    8. Attempt again the file creation that previously failed to verify that the setup is complete:
      [root@baden ~]# touch /gpfs0/test
      [root@baden ~]# mmlsattr -n gpfs.Encryption /gpfs0/test
      file name: /gpfs0/test
      gpfs.Encryption: "EAGC????f?????????????? ??????w?^??>???????????? ?L4??
      _-???V}f???X????,?G?<sH??0?)??M?????)?KEY-326a1906-be46-4983-a63e-29f005fb3a15?
      isklmsrv?)?KEY-6aaa3451-6a0c-4f2e-9f30-d443ff2ac7db?RKMKMIP3?"
      EncPar 'AES:256:XTS:FEK:HMACSHA512'
              type: wrapped FEK WrpPar 'AES:KWRAP' CmbPar 'XORHMACSHA512'
                      KEY-326a1906-be46-4983-a63e-29f005fb3a15:isklmsrv

    All files created from this point on will be encrypted with an FEK, which in turn will be wrapped with the chosen MEK.

    Security note: The contents of /var/mmfs/etc/RKM.conf and of the key store file (/var/mmfs/etc/RKMcerts/ISKLM.p12 in the example) are extremely security-sensitive. Make sure file permissions are appropriately set, for example:
    -rw-------. 1 root root 2446 Mar 20 12:15 /var/mmfs/etc/RKM.conf
    drw-------. 2 root root 4096 Mar 20 13:47 /var/mmfs/etc/RKMcerts
    -rw-------. 1 root root 3988 Mar 20 13:47 /var/mmfs/etc/RKMcerts/ISKLM.p12
    Also, make sure the passphrase is not leaked through other means (for example, the shell history). Finally, note that GPFS neither manages nor replicates these files, so make sure you take appropriate precautions to ensure that they are not lost or corrupted.

Enabling encryption on other nodes

If you want to replicate the same encryption configuration on another node, you can simply replicate the content of /var/mmfs/etc on the latter. You may also choose to have different encryption configurations on different nodes; to do so, follow the steps described previously. Note the following:
  • The content of /var/mmfs/etc/RKM.conf may be different across nodes.
  • Different nodes may have different key stores.
  • Installing a new encryption policy removes the previous one; as a consequence, all policy statements for the file system must be collected into a single file that can then be installed in the file system.

The scenario described here implies that the ability to create, read or write files from a particular node depends on the content of /var/mmfs/etc/RKM.conf and on which key stores are deployed on that node. For example, within a given cluster, read/write operations may succeed on one node and fail on another due to the fact that the right key store or /var/mmfs/etc/RKM.conf entry, or both, are missing.