Establishing an encryption-ready environment requires the sequence
of activities summarized here. This summary covers a basic setup with
a single encrypted fileset.
Considerations on FIPS compliance
To retain
compliance to FIPS 140-2, do the following:
- Ensure that the fips configuration parameter
is turned to on on the ISKLM before generating
server certificates and MEKs. (See the ISKLM installation guide for
more information.)
- Ensure that GPFS™ is running
with the FIPS1402mode configuration variable
set to yes before creating the key
store file.
Initial setup by administrator
During initial
setup, perform the following steps:
- Install IBM® Security
Key Lifecycle Manager (ISKLM) version 2.5.0.1 or later.
Refer
to the ISKLM installation guide.
From the main page of the
ISKLM web GUI, select Configuration and
then Key Serving Parameters from the selector
on the right. Then make sure that the check box next to Keep
pending client device communication certificates is
selected.
- Configure an SSL/KMIP server certificate on ISKLM.
(Skip
this step if you have already configured the SSL/KMIP server certificate).
- Click Configuration in the menu bar
at the top, and select SSL/KMIP from the menu on the left.
- To create a new self-signed certificate, click the Create
self-signed certificate radio button.
- Enter the appropriate details for the certificate to be created,
and click OK.
- Export the SSL/KMIP server certificate from ISKLM.
This
step is required to obtain a trusted copy of the server certificate.
- Identify the certificate label for the SSL/KMIP server certificate
currently in use.
- Click Advanced Configuration from the
menu bar at the top, and select Server Certificates from
the dropdown menu.
- Select the certificate identified as being "In Use," click Modify,
and make note of the certificate label.
- From an administrator console on the ISKLM server, change the
directory to the bin directory in WAS_HOME:
- Type the following:
- Identify the correct UUID of the certificate:
- First issue the following print command,
replacing YOUR_LABEL with the certificate
label obtained in the previous step:
print AdminTask.tklmCertList('[-alias YOUR_LABEL]')
The
system will respond with output similar to the following:
CTGKM0001I Command succeeded.
uuid = CERTIFICATE-7005029a-831d-405f-af30-4bf0177909de
alias = server
key store name = defaultKeyStore
key state = ACTIVE
issuer name = CN=server
subject name = CN=server
creation date = 13/03/2014 16:27:13 Eastern Daylight Time
expiration date = 09/03/2015 07:12:30 Eastern Daylight Time
serial number = 1394363550
- Reissue the print command, this time
specifying the UUID identified during the previous step:
print AdminTask.tklmCertExport('[-uuid
CERTIFICATE-7005029a-831d-405f-af30-4bf0177909de -format base64 -fileName
/root/srvcert]')
The certificate will
be exported in the file specified after -fileName (in
this example, /root/srvcert).
- Copy the file into a temporary directory on the GPFS node you wish to configure for encryption
(for example, to /tmp/srvcert).
- Create a new GPFS device
group.
- From the main page of the ISKLM web GUI, select Advanced
Configuration and then Device Group from
the menu bar at the top.
- On the next page, click the Create button.
- In the Create Device Group window, select
the GPFS device family and enter an appropriate
name (for example, "GPFS_TENANT1"); make note of this name, as it
will be required later on.
- Once the device group has been created, the system will prompt
you to add devices and keys. However, the next steps in this procedure
will generate keys explicitly, so now just click Close to
return to the main page.
- On the main page, select the GPFS tenant
device group, click Go to..., and select Manage
keys and devices from the dropdown menu.
- On the next screen, click Add and select Key from
the dropdown menu.
The system prompts you to specify the following:
Make note of the key UUID (in the example, KEY-326a1906-be46-4983-a63e-29f005fb3a15),
as it will be required later on.
- Enable the option Hold new certificate requests pending
my approval by selecting it from the dropdown menu in
the lower part of the page.
- Configure the RKM backend on GPFS.
- Create a key store containing the server certificate and a new
key pair and certificate that will be used by the client.
- To do so, from an administrator console on the GPFS node you wish to be configured with encryption,
create the subdirectory /var/mmfs/etc/RKMcerts.
- Issue the following command, replacing cname_test with
a string identifying the GPFS node
or the tenant performing encryption, label_test with
a string that identifies the client certificate in the key store,
and a_password with a password protecting
the secret key material in the key store file:
mmauth gencert --cname cname_test --label a_label --cert /tmp/srvcert --out
/var/mmfs/etc/RKMcerts/ISKLM.p12 --pwd a_password
Upon
success, the new key store is created (for example, in /var/mmfs/etc/RKMcerts/ISKLM.p12).
Note: The created key store must be record locked when the GPFS daemon starts. If the key store
files are stored on an NFS mount, it is possible for the encryption
initialization process to hang, due to a bug that affects the way
NFS handles record locking. If this happens, upgrade your version
of NFS or store your key store file on a local file system. If an
upgrade is not possible and no local file system is available, use
a RAM drive to store the key store files.
- Open an editor to create the /var/mmfs/etc/RKM.conf file,
as follows:
ISKLM_srv {
type = ISKLM
kmipServerUri = tls://raclette.zurich.ibm.com:5696
keyStore = /var/mmfs/etc/RKMcerts/ISKLM.p12
passphrase = a_password
clientCertLabel = a_label
tenantName = GPFS_TENANT1
}
Choose an appropriate name for the RKM backend stanza
(
ISKLM_srv in the example) and ensure that the
fields are filled in correctly:
- type
- Must be set to ISKLM.
- kmipServerUri
- Contains the DNS name or IP address of the ISKLM server and the
KMIP SSL port (you can find the latter in the main page of the ISKLM
web GUI; 5696 is the default).
- keyStore
- Contains the path to the key store file created in the previous
step.
- passphrase and clientCertLabel
- Contain the password and label specified in the command line upon
creation of the key store.
- tenantName
- Contains the name of the device group created in ISKLM.
- Set up an encryption policy.
- Create a policy, instructing GPFS to
encrypt all files in the file system with an FEK and wrap the FEK
with the MEK created previously; for example:
RULE 'p1' SET POOL 'system' # one placement rule is required at all times
RULE 'Encrypt all files in filesystem with rule E1'
SET ENCRYPTION 'E1'
WHERE NAME LIKE '%'
RULE 'simpleEncRule' ENCRYPTION 'E1' IS
ALGO 'DEFAULTNISTSP800131A'
KEYS('KEY-326a1906-be46-4983-a63e-29f005fb3a15:ISKLM_srv')
- Create the Keyname string in the policy
statement as follows:
- KeyID
- The key ID copied from the ISKLM web GUI.
- RkmID
- The name of the RKM backend stanza in the /var/mmfs/etc/RKM.conf file.
- After installing the rule with mmchpolicy,
you need to import the client certificate into ISKLM. To do that,
attempt the creation of an encrypted file to trigger a KMIP request,
for example:
[root@baden ~]# touch /gpfs0/test
touch: cannot touch `/gpfs0/test': Permission denied
[root@baden ~]# tail -n 2 /var/adm/ras/mmfs.log.latest
Thu Mar 20 14:00:55.029 2014: [E] Unable to open encrypted file: inode 46088,
Fileset fs1, File System gpfs0.
Thu Mar 20 14:00:55.030 2014: [E] Error: key
'KEY-326a1906-be46-4983-a63e-29f005fb3a15:ISKLM_srv' could not be fetched (RKM
reported error -1004).
The request will initially fail
because ISKLM does not yet trust the client certificate; however,
the certificate will be placed in a list of pending certificates.
- From the main page of the ISKLM web GUI, select Pending
client device communication certificates.
- Identify the client certificate and click View.
- Carefully check that the certificate being imported matches the
one created in the previous step, then click Accept and
Trust
- On the resulting screen, give a name to the certificate, then
click Accept and Trust.
- Attempt again the file creation that previously failed to verify
that the setup is complete:
[root@baden ~]# touch /gpfs0/test
[root@baden ~]# mmlsattr -n gpfs.Encryption /gpfs0/test
file name: /gpfs0/test
gpfs.Encryption: "EAGC????f?????????????? ??????w?^??>???????????? ?L4??
_-???V}f???X????,?G?<sH??0?)??M?????)?KEY-326a1906-be46-4983-a63e-29f005fb3a15?
isklmsrv?)?KEY-6aaa3451-6a0c-4f2e-9f30-d443ff2ac7db?RKMKMIP3?"
EncPar 'AES:256:XTS:FEK:HMACSHA512'
type: wrapped FEK WrpPar 'AES:KWRAP' CmbPar 'XORHMACSHA512'
KEY-326a1906-be46-4983-a63e-29f005fb3a15:isklmsrv
All files created from this point on will be encrypted
with an FEK, which in turn will be wrapped with the chosen MEK.
Security note: The contents of
/var/mmfs/etc/RKM.conf and
of the key store file (
/var/mmfs/etc/RKMcerts/ISKLM.p12 in
the example) are extremely security-sensitive. Make sure file permissions
are appropriately set, for example:
-rw-------. 1 root root 2446 Mar 20 12:15 /var/mmfs/etc/RKM.conf
drw-------. 2 root root 4096 Mar 20 13:47 /var/mmfs/etc/RKMcerts
-rw-------. 1 root root 3988 Mar 20 13:47 /var/mmfs/etc/RKMcerts/ISKLM.p12
Also,
make sure the passphrase is not leaked through other means (for example,
the shell history). Finally, note that GPFS neither
manages nor replicates these files, so make sure you take appropriate
precautions to ensure that they are not lost or corrupted.
Enabling encryption on other nodes
If you
want to replicate the same encryption configuration on another node,
you can simply replicate the content of
/var/mmfs/etc on
the latter. You may also choose to have different encryption configurations
on different nodes; to do so, follow the steps described previously.
Note the following:
- The content of /var/mmfs/etc/RKM.conf may
be different across nodes.
- Different nodes may have different key stores.
- Installing a new encryption policy removes the previous one; as
a consequence, all policy statements for the file system must be collected
into a single file that can then be installed in the file system.
The scenario described here implies that the ability to
create, read or write files from a particular node depends on the
content of /var/mmfs/etc/RKM.conf and on which
key stores are deployed on that node. For example, within a given
cluster, read/write operations may succeed on one node and fail on
another due to the fact that the right key store or /var/mmfs/etc/RKM.conf entry,
or both, are missing.