Possible client outage for IBM® WebSphere® Application Server V6.1 if using default self-signed certificate expiration with Application Server V6.1 through V126.96.36.199, including z/OS® when using the self signed certificate configuration.
All Application Server V6.1 platforms (V6.1 through 188.8.131.52), including z/OS when using the self-signed certificate configuration.
Note: Typically, the z/OS platform configurations use the internal RACF CA certificates which would not have this problem.
WebSphere Application Server V6.1 introduced advanced certificate management capabilities:
- When a new profile is created the configuration process generates a unique self-signed certificate for each Application Server profile. Before applying this APAR Fix, the certificate lifetime is good for one year from when it is created (security best practice).
- A certificate expiration monitor thread is setup to keep track of the server certificates to prevent expirations from causing server outages. The Certificate Expiration Monitor, which runs every 28 days, by default, can do the following:
- Check to determine if any server certificates in the keystore configurations will expire within the 60 day (default) threshold.
- Send a notification to the system logs (default) or send a email to a pre-configured email address once this threshold is reached.
- Automatically replace the server's self-signed certificate with a new self-signed certificate (default).
- Update the runtime dynamically by reloading any server sockets or channels with the new certificate (default).
Due to this dynamic replacement of the Server's expiring certificates, given the default settings, your certificate will be automatically replaced when the Server's personal certificate end date is within 60 days (default) of the running of the Certification Expiration Monitor (which by default runs every 28 days). This can be a problem because:
- WebSphere Application Server does not send prenotification, to system logs, prior to automatic replacement of the self signed certificate.
- Any clients that currently have signers for that server will no longer have the correct signers, meaning the clients can no longer talk to the server if SSL is enabled.
For organizations running the web server on an unmanaged node, while making SSL connections from the plug-in to the application server (the most common configuration), this will result in a production outage as the Web server plug-in will lose connectivity to the Web container.
For IBM WebSphere Application Server for Distributed V6.1 through 184.108.40.206:
- Apply Cumulative Fix 1 (if not already at V220.127.116.11) or later, then apply APAR PK42863.
Important: If you have already applied PK34093 or Cumulative Fix 7(V18.104.22.168) or later, (which contains PK34093) at Recommended Fixes for WebSphere Application Server, and CIP or i5/OS (OS400) are not involved, you need not install PK42863 in addition.
For IBM WebSphere Application Server for i5/OS V6.1 through 22.214.171.124:
- Apply Cumulative Fix 3 or 5 (if not already at V126.96.36.199 or 188.8.131.52), or later, and then apply APAR PK42863.
Important: If you have already applied PK34093 or Cumulative Fix 7 (V184.108.40.206) or later, (which contains PK34093) at Recommended Fixes for WebSphere Application Server, you will need to install APAR PK42863 in addition.
For IBM WebSphere Application Server for z/OS V220.127.116.11 through 18.104.22.168:
- Apply APAR PK34093 by installing PTFs for V22.214.171.124, or later, available at APAR/PTF table for WebSphere Application Server V6.1 for z/OS.
Note: If you happen to need to install the fix using CIP, you will need to acquire an APAR fix for PK42863. If you are not installing the fix using CIP, PTFs for V126.96.36.199 or later will resolve this issue.
Overall APAR Solution Note: APAR PK42863 is a replacement fix for PK34093. However, PK42863 is only needed to resolve an issue with customers installing with CIP or installing on i5/OS (OS400).
Applying this APAR, or its Cumulative Fix containing it, will do the following:
(Important: To benefit from the part of this change that extends the certificate's expiration to 15 years, you must first apply the APAR fix before creating the profile. If you have an existing profile, created prior to applying the APAR fix, then you need only manage the certificates (replace the existing certificate) prior to letting them expire.)
- Send prenotification warnings starting 90 days before self-signed certificate replacement (90 days by default).
- This prenotification gives the administrator an opportunity to perform the replacement at a more opportune time where the client keystores can be updated. By default these warnings will start 90 days before self-signed certificate replacement occurs.
- Self-signed certificates will now have a default of 15 years from the date the profile is created.
- This reduces the administrative burden for the default self-signed certificate configuration to reduce the potential for these kinds of issues. Administrators need to assess the risk of having such a long certificate life-time and may choose to replace the default self-signed certificate with one with a shorter lifetime. These certificates are still unique to the profile and more secure than the default certificates used in previous releases of Application Server.
Additionally you may also wish to do the following, in concert with applying the APAR, or instead of applying the APAR:
- You can disable the auto-replacement and just get the notifications (warnings about certificates about to expire) requiring you to manually replace your certificates:
- These functions can be disabled in the Administrative Console using the following links:
- Security > SSL certificate and key management, deselect “Dynamically update the run time when SSL configuration changes occur."
- Security > SSL certificate and key management > Manage certificate expiration, deselect “Automatically replace expiring self-signed certificates."
- You can consider using CA issued personal certificates in your configuration. The auto-replace function only replaces self-signed certificates that are about to expire, but still provides expiration notifications for all certificates in the server keystores. When a CA issued personal certificate is replaced by an administrator, it is typically replaced with the same root CA certificate and thus clients will continue to work without any intervention.
- If you can, replace the default self-signed certificates this way so that trust is preserved even when a personal certificate needs to be replaced. This provides the most convenience with personal certificates, allowing the personal certificate to continue to be created with the recommended one-year lifetime and be replaced without causing any client trust issues. The CA signing certificates are typically valid for many years.
- If using a CA is not possible for generating personal certificates, you could also consider replacing the default self-signed certificate with a new self-signed certificate that will last longer than one-year to avoid maintenance downtime while replacing the self-signed certificates.
If the automatic replacement already occurred to resolve the client SSL communication failure, you need to ensure the client trust store contains the correct signer certificate. For the plug-in client, when the Server's self-signed certificate gets replaced, the correct signer is placed in the plug-in keystore
profile_root\config\cells\cell_name\nodes\node_name\servers\webserver1\plugin-key.kdb. The plug-in keystores containing the new signers will need to get manually copied to the correct plug-in location. Until this occurs, the Web server plug-in cannot resume successful SSL communications to the Server Web container. After the keystore containing the new signers is copied to the correct plug-in location, the Web server will need to be restarted before it can resume successful SSL communications to the Web container.
For more details, please see this documentation at:
- WebSphere Application Server V6.1 default self-signed configuration (location of key and trust stores)
- Information Center link describing how to replace a self-signed certificate (and associated signers) from within the Administrative Console
- Information Center link describing Certificate Expiration Monitor capabilities
- IBM WebSphere Developer Technical Journal: SSL, certificate, and key management enhancements for even stronger security in WebSphere Application Server v6.1
28 July 2007
|Added reference link to IBM WebSphere Developer Technical Journal article.|
18 May 2007
|Updated to address additional situations.|
11 May 2007
| Added ZE APAR PK42863 as an additional APAR, needed instead of PK34093
in certain situations.
27 April 2007
|Added Cross Reference for IBM WebSphere Application Server for z/OS|
13 April 2007
|Application Servers||WebSphere Application Server for z/OS||Security||OS/390, Solaris, z/OS, Windows||188.8.131.52, 184.108.40.206, 220.127.116.11, 18.104.22.168, 22.214.171.124, 126.96.36.199, 6.1||All Editions|
|Application Servers||WebSphere Application Server - Express||Security||AIX, HP-UX, i5/OS, Linux, Linux pSeries, Linux Red Hat - pSeries, Linux Red Hat - zSeries, Linux zSeries, OS/400, Solaris, Windows||6.1||All Editions|
|Application Servers||Runtimes for Java Technology||Java SDK|
|Application Servers||WebSphere Application Server for z/OS||z/OS||6.1|