IBM Support

PI70402: SAML WEB SSO OUTOFMEMORY IN KEYSTOREMANAGER

Subscribe

You can track all active APARs for this component.

APAR status

  • Closed as program error.

Error description

  • In SAML Web SSO, the customer has trustStore set to
    "CellDefaultTrustStore".  After some time, an OutOfMemory
    condition occurs.
    
    The class
    "com.ibm.ws.wssecurity.wssapi.token.impl.KeyStoreManager",
    occupies 1,257,927,688 (81.79%) bytes.
    
    3707 [9/28/16 13:53:19:625 UTC] 0000027a WasKeyStoreUt >
    getKeyStore(String keyStoreRef[CellDefaultTrustStore]) Entry
    [9/28/16 13:54:40:024 UTC] 0000027a webapp        E
    com.ibm.ws.webcontainer.webapp.WebApp logServletError SRVE0293E:
    [Servlet Error]-[IBMWebSphereSamlACSListenerServlet]:
    java.lang.OutOfMemoryError: Java heap space
    at java.lang.Class.getConstructorsImpl(Native Method)
    at java.lang.Class.getConstructors(Class.java:568)
    at com.ibm.crypto.provider.bd.newInstance(Unknown Source)
    ...
    

Local fix

  • Please ensure following property can be set as workaround.
    
    sso_<id>.sp.keyStore="name=CellDefaultTrustStore
    managementScope=(cell):myCellName"
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED:  All users of IBM WebSphere Application      *
    *                  Server                                      *
    ****************************************************************
    * PROBLEM DESCRIPTION: OutOfMemory may occur in SAML Web SSO   *
    *                      when trustStore property has no         *
    *                      management scope                        *
    ****************************************************************
    * RECOMMENDATION:  Install a fix pack that contains this       *
    *                  APAR.                                       *
    ****************************************************************
    If the SAML Web SSO TAI sso_<id>.sp.trustStore custom property
    is configured either without specifying a management scope or
    with a management scope that does not match the current
    management scope of the keystore that WebSphere security finds
    in its cache, a memory leak in the
    com.ibm.ws.wssecurity.wssapi.token.impl.KeyStoreManager class
    will occur.
    

Problem conclusion

  • The SAML code relies on the base security code to cache
    managed keystores. The SAML code will cache private keys
    retrieved from the keystores and the cache key for these keys
    includes a hash of the keystore object from which it was
    obtained.
    
    When SAML attempts to retrieve a keystore that either has no
    management scope or has the wrong management scope, instead of
    returning the keystore that is in its cache, base security is
    reading the keystore from the disk and returning a new
    keystore object. Although we technically have the same
    keystore from the disk and the same key from the keystore,
    since the Java object for the keystore is different, so the
    cache key is different.
    
    Since SAML keeps generating unique cache keys for the same
    keystore/key, the same private key keeps getting cached over
    and over using different cache keys each time. Given enough
    requests, an OutOfMemory condition will occur.
    
    The base security code is updated to return the keystore from
    the cache if there is no management scope specified instead
    of creating a new object using the keystore on the disk.
    
    If the management scope that is specified is not correct, then
    the OutOfMemory condition may still occur.  The resolution to
    this problem is to either remove or correct the management
    scope.  Example:
    
    name=myKeyStoreRef managementScope=(cell):myCell:(node):myNode
    
    
    The fix for this APAR is currently targeted for inclusion in
    fix packs 7.0.0.43, 8.0.0.13, 8.5.5.12 and 9.0.0.2.  Please
    refer to the Recommended Updates page for delivery
    information:
    http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980
    

Temporary fix

Comments

APAR Information

  • APAR number

    PI70402

  • Reported component name

    WEBS APP SERV N

  • Reported component ID

    5724H8800

  • Reported release

    850

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2016-10-07

  • Closed date

    2016-11-15

  • Last modified date

    2016-11-15

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    WEBS APP SERV N

  • Fixed component ID

    5724H8800

Applicable component levels

  • R700 PSY

       UP

  • R800 PSY

       UP

  • R850 PSY

       UP

  • R900 PSY

       UP



Document information

More support for: WebSphere Application Server
General

Software version: 850

Reference #: PI70402

Modified date: 15 November 2016