Possible client outage for WebSphere Application Server V6.1 if using default self-signed certificate expiration (APAR PK34093 or PK42863)

Flash (Alert)


Abstract

Possible client outage for IBM® WebSphere® Application Server V6.1 if using default self-signed certificate expiration with Application Server V6.1 through V6.1.0.5, including z/OS® when using the self signed certificate configuration.

Content

Versions affected:

All Application Server V6.1 platforms (V6.1 through 6.1.0.6), including z/OS when using the self-signed certificate configuration.

Note: Typically, the z/OS platform configurations use the internal RACF CA certificates which would not have this problem.

Background:

WebSphere Application Server V6.1 introduced advanced certificate management capabilities:

  • When a new profile is created the configuration process generates a unique self-signed certificate for each Application Server profile. Before applying this APAR Fix, the certificate lifetime is good for one year from when it is created (security best practice).
  • A certificate expiration monitor thread is setup to keep track of the server certificates to prevent expirations from causing server outages. The Certificate Expiration Monitor, which runs every 28 days, by default, can do the following:
    • Check to determine if any server certificates in the keystore configurations will expire within the 60 day (default) threshold.
    • Send a notification to the system logs (default) or send a email to a pre-configured email address once this threshold is reached.
    • Automatically replace the server's self-signed certificate with a new self-signed certificate (default).
    • Update the runtime dynamically by reloading any server sockets or channels with the new certificate (default).
Important: The Certificate Expiration Monitor does not handle replacing client self-signed certificates, nor is it capable of sending client's the new signer certificate needed for trust. Specifically, if the client is a Web server plug-in, it will not be able to securely communicate with the application server after self-signed certificate replacement.

Problem Description:

Due to this dynamic replacement of the Server's expiring certificates, given the default settings, your certificate will be automatically replaced when the Server's personal certificate end date is within 60 days (default) of the running of the Certification Expiration Monitor (which by default runs every 28 days). This can be a problem because:

  • WebSphere Application Server does not send prenotification, to system logs, prior to automatic replacement of the self signed certificate.
  • Any clients that currently have signers for that server will no longer have the correct signers, meaning the clients can no longer talk to the server if SSL is enabled.

For organizations running the web server on an unmanaged node, while making SSL connections from the plug-in to the application server (the most common configuration), this will result in a production outage as the Web server plug-in will lose connectivity to the Web container.


Solutions:

For IBM WebSphere Application Server for Distributed V6.1 through 6.1.0.5:

  • Apply Cumulative Fix 1 (if not already at V6.1.0.1) or later, then apply APAR PK42863.

Important: If you have already applied PK34093 or Cumulative Fix 7(V6.1.0.7) or later, (which contains PK34093) at Recommended Fixes for WebSphere Application Server, and CIP or i5/OS (OS400) are not involved, you need not install PK42863 in addition.
-----------------------------

For IBM WebSphere Application Server for i5/OS V6.1 through 6.1.0.5:

  • Apply Cumulative Fix 3 or 5 (if not already at V6.1.0.3 or 6.1.0.5), or later, and then apply APAR PK42863.

Important: If you have already applied PK34093 or Cumulative Fix 7 (V6.1.0.7) or later, (which contains PK34093) at Recommended Fixes for WebSphere Application Server, you will need to install APAR PK42863 in addition.
-----------------------------

For IBM WebSphere Application Server for z/OS V6.1.0.3 through 6.1.0.6:


Note: If you happen to need to install the fix using CIP, you will need to acquire an APAR fix for PK42863. If you are not installing the fix using CIP, PTFs for V6.1.0.7 or later will resolve this issue.

Overall APAR Solution Note: APAR PK42863 is a replacement fix for PK34093. However, PK42863 is only needed to resolve an issue with customers installing with CIP or installing on i5/OS (OS400).
-----------------------------

Applying this APAR, or its Cumulative Fix containing it, will do the following:

(Important: To benefit from the part of this change that extends the certificate's expiration to 15 years, you must first apply the APAR fix before creating the profile. If you have an existing profile, created prior to applying the APAR fix, then you need only manage the certificates (replace the existing certificate) prior to letting them expire.)

  • Send prenotification warnings starting 90 days before self-signed certificate replacement (90 days by default).
    • This prenotification gives the administrator an opportunity to perform the replacement at a more opportune time where the client keystores can be updated. By default these warnings will start 90 days before self-signed certificate replacement occurs.

  • Self-signed certificates will now have a default of 15 years from the date the profile is created.
    • This reduces the administrative burden for the default self-signed certificate configuration to reduce the potential for these kinds of issues. Administrators need to assess the risk of having such a long certificate life-time and may choose to replace the default self-signed certificate with one with a shorter lifetime. These certificates are still unique to the profile and more secure than the default certificates used in previous releases of Application Server.

Additionally you may also wish to do the following, in concert with applying the APAR, or instead of applying the APAR:
  • You can disable the auto-replacement and just get the notifications (warnings about certificates about to expire) requiring you to manually replace your certificates:
    • These functions can be disabled in the Administrative Console using the following links:
      1. Security > SSL certificate and key management, deselect “Dynamically update the run time when SSL configuration changes occur."
      2. Security > SSL certificate and key management > Manage certificate expiration, deselect “Automatically replace expiring self-signed certificates."
    Note: While not directly related to the issue here, if the node synchronization feature is disabled or the default synchronization period has been extended, you should disable the dynamic SSL configuration changes as previously described. Not synchronizing the configuration out to the nodes using the default interval could cause loss of administrative communications after a self-signed certificate replacement.

OR...

  • You can consider using CA issued personal certificates in your configuration. The auto-replace function only replaces self-signed certificates that are about to expire, but still provides expiration notifications for all certificates in the server keystores. When a CA issued personal certificate is replaced by an administrator, it is typically replaced with the same root CA certificate and thus clients will continue to work without any intervention.
  • If you can, replace the default self-signed certificates this way so that trust is preserved even when a personal certificate needs to be replaced. This provides the most convenience with personal certificates, allowing the personal certificate to continue to be created with the recommended one-year lifetime and be replaced without causing any client trust issues. The CA signing certificates are typically valid for many years.
  • If using a CA is not possible for generating personal certificates, you could also consider replacing the default self-signed certificate with a new self-signed certificate that will last longer than one-year to avoid maintenance downtime while replacing the self-signed certificates.

Recovery:

If the automatic replacement already occurred to resolve the client SSL communication failure, you need to ensure the client trust store contains the correct signer certificate. For the plug-in client, when the Server's self-signed certificate gets replaced, the correct signer is placed in the plug-in keystore
profile_root\config\cells\cell_name\nodes\node_name\servers\webserver1\plugin-key.kdb. The plug-in keystores containing the new signers will need to get manually copied to the correct plug-in location. Until this occurs, the Web server plug-in cannot resume successful SSL communications to the Server Web container. After the keystore containing the new signers is copied to the correct plug-in location, the Web server will need to be restarted before it can resume successful SSL communications to the Web container.

Additional documentation:

For more details, please see this documentation at:

Change History
28 July 2007
Added reference link to IBM WebSphere Developer Technical Journal article.
18 May 2007
Updated to address additional situations.
11 May 2007
Added ZE APAR PK42863 as an additional APAR, needed instead of PK34093
in certain situations.
27 April 2007
Added Cross Reference for IBM WebSphere Application Server for z/OS
13 April 2007
Initial publication

Cross Reference information
Segment Product Component Platform Version Edition
Application Servers WebSphere Application Server for z/OS Security OS/390, Solaris, z/OS, Windows 6.1.0.6, 6.1.0.5, 6.1.0.4, 6.1.0.3, 6.1.0.2, 6.1.0.1, 6.1 All Editions
Application Servers WebSphere Application Server - Express Security AIX, HP-UX, i5/OS, Linux, Linux pSeries, Linux Red Hat - pSeries, Linux Red Hat - zSeries, Linux zSeries, OS/400, Solaris, Windows 6.1 All Editions
Application Servers Runtimes for Java Technology Java SDK
Application Servers WebSphere Application Server for z/OS z/OS 6.1

Rate this page:

(0 users)Average rating

Add comments

Document information


More support for:

WebSphere Application Server
General

Software version:

6.1, 6.1.0.1, 6.1.0.2, 6.1.0.3, 6.1.0.5

Operating system(s):

AIX, HP-UX, IBM i, Linux, Solaris, Windows

Software edition:

Advanced, Base, Developer, Express, Network Deployment

Reference #:

1258427

Modified date:

2007-04-27

Translate my page

Machine Translation

Content navigation