FIPS 140 mode and IP security

The Internet Key Exchange daemon (IKED), the network security services daemon (NSSD), and the TCP/IP stack components perform a wide variety of cryptographic operations for IP Security. The IKED and the NSSD manage cryptographic keys and digital signatures and perform hashes for authentication. The IKED and the TCP/IP stacks perform encryption and decryption of messages that flow over the IPSec tunnels. Architectural enhancements to the IKE protocols periodically introduce new cryptographic algorithms for performing hashes and encryption.

Federal Information Processing Standards (FIPS) document 140 (FIPS 140) provides a higher degree of assurance of the integrity of cryptographic operations by placing restrictions on the cryptographic components and the operations performed by these components. Weaker algorithms are forbidden and all the operations must be performed by cryptographic modules that are contained within a well defined cryptographic boundary.

You can configure the IKED, the NSSD, and the TCP/IP stack components to operate in FIPS 140 mode. When you do, you restrict the cryptographic algorithms they support, and you modify their interactions with each other and with the other hardware and software components of the z/OS® system related to cryptography, such as Integrated Cryptographic Services Facility (ICSF) and System SSL.

In FIPS 140 mode, the IKED, the NSSD, and the TCP/IP stacks enforce the following restrictions on the cryptographic algorithms that can be used for IP security:

You cannot use the DES encryption algorithm.
You cannot use the HMAC-MD5, HMAC-MD5-96, AES128-XCBC, and AES128-XCBC-96 algorithms for authentication or pseudo-random function.
You cannot use Diffie-Hellman groups 1, 2, and 5.
You cannot use certificates for RSA signature authentication that have key lengths less than 1024 bits.
You cannot use pre-shared keys whose length is less than half the key size of the chosen HowToAuthMsgs (IKEv1) or PseudoRandomFunction (IKEv2) algorithm.

When the FIPS 140 mode is configured for a TCP/IP stack, the Policy Agent enforces some of the FIPS 140-related restrictions when it parses the IP security policy files. Other restrictions are enforced when dynamic tunnels are being activated, after the FIPS 140 mode of all of the relevant software components (the IKED and the NSSD) is known.

You configure FIPS 140 mode independently for each of the IKED, the NSSD, and the TCP/IP stack components. ICSF and System SSL must also be configured to support FIPS 140. If you use FIPS 140 mode in some components and not others, the resulting system might not operate in FIPS 140 mode. The components that are configured to use FIPS 140 mode can use cryptographic services only from components that are also operating in FIPS 140 mode, so the FIPS 140 mode mismatch can cause functional problems.

If a TCP/IP stack is configured to use FIPS 140 mode, but the IKED is not, the IKED does not perform any dynamic VPN tunnel activations for the stack. The IKED performs many cryptographic operations on behalf of the stack during tunnel activation.
Rule: Whenever the stack is configured for FIPS 140 mode, also configure FIPS 140 mode for the IKED.
If the IKED and the TCP/IP stacks it supports are configured to use FIPS 140 mode, but the NSSD is not, the IKED cannot use the NSS IPSec certificate service for its stacks. Because the IKED must use the NSS IPSec certificate service to create and verify digital signatures for IKEv2 tunnels, this FIPS 140 mode mismatch prevents activation of any IKEv2 tunnels that use digital signature authentication.
Rule: Whenever the IKED is configured for FIPS 140 mode, also configure FIPS 140 mode for the NSSD.
If you have a sysplex that is configured for Sysplex-Wide Security Associations (SWSA), and the distributor stack is not configured to use FIPS 140 mode, then it will not be able to successfully distribute tunnels to target stacks that are configured in FIPS 140 mode.
Rule: Whenever you have target stacks that are configured in FIPS 140 mode, first configure FIPS 140 mode for the distributing stack.

When possible, you should enable FIPS 140 mode for the IKED, the NSSD, and the TCP/IP stacks all at once. If you must implement FIPS 140 support in stages, enable FIPS 140 mode in the components in the following order:

Configure FIPS 140 mode in the NSSD. If the NSSD is configured in FIPS 140 mode and the IKED and the TCP/IP stacks are not, the IKED still uses the NSS IPSec certificate service provided by the NSSD. Note that the NSSD creates and verifies signatures only for certificates that conform to FIPS 140 restrictions, even if the IPSec client is not operating in FIPS 140 mode.
Configure FIPS 140 mode in the IKED. When the NSSD and the IKED are both in FIPS 140 mode, but the stacks are not, dynamic VPN tunnels can be activated and data can flow, as long as those tunnels follow the FIPS 140 cryptographic algorithm restrictions.
If you are using SWSA in a sysplex, configure FIPS 140 mode in the distributor stack of the sysplex.
Configure FIPS 140 mode in all other TCP/IP stacks.

Enabling FIPS 140 mode on a system can affect performance. For example, you might have to change from using a weak encryption algorithm to using one that requires more processing to perform. Even if no algorithm changes are necessary, the IKED, the NSSD, and the TCP/IP stacks perform their cryptography operations in a different way when FIPS 140 mode is enabled than when it is not enabled, because FIPS 140 imposes additional self-verification requirements and access restrictions, and because hardware accelerated implementations of some cryptographic operations might not be available in FIPS 140 mode.