IBM Support

PK69110: THE SERVANT ABENDS CAUSED BY HUNG THREADS IN JDBC DRIVER.

Fixes are available

7.0.0.3: WebSphere Application Server V7.0 Fix Pack 3 for IBM i
7.0.0.3: WebSphere Application Server V7.0 Fix Pack 3 for AIX
7.0.0.3: WebSphere Application Server V7.0 Fix Pack 3 for HP-UX
7.0.0.1: WebSphere Application Server V7.0 Fix Pack 1 for AIX
7.0.0.3: Java SDK 1.6 SR4 Cumulative Fix for WebSphere Application Server
7.0.0.1: WebSphere Application Server V7.0 Fix Pack 1 for Windows
7.0.0.3: WebSphere Application Server V7.0 Fix Pack 3 for Solaris
7.0.0.1: WebSphere Application Server V7.0 Fix Pack 1 for HP-UX
7.0.0.1: WebSphere Application Server V7.0 Fix Pack 1 for Linux
7.0.0.3: WebSphere Application Server V7.0 Fix Pack 3 for Linux
7.0.0.1: WebSphere Application Server V7.0 Fix Pack 1 for Solaris
7.0.0.5: WebSphere Application Server V7.0 Fix Pack 5 for AIX
7.0.0.3: WebSphere Application Server V7.0 Fix Pack 3 for Windows
7.0.0.5: WebSphere Application Server V7.0 Fix Pack 5 for IBM i
7.0.0.5: WebSphere Application Server V7.0 Fix Pack 5 for Windows
7.0.0.5: WebSphere Application Server V7.0 Fix Pack 5 for HP-UX
7.0.0.5: Java SDK 1.6 SR5 Cumulative Fix for WebSphere Application Server
7.0.0.5: WebSphere Application Server V7.0 Fix Pack 5 for Solaris
7.0.0.5: WebSphere Application Server V7.0 Fix Pack 5 for Linux
Java SDK 1.5 SR10 Cumulative Fix for WebSphere Application Server
7.0.0.7: WebSphere Application Server V7.0 Fix Pack 7 for IBM i
7.0.0.7: WebSphere Application Server V7.0 Fix Pack 7 for AIX
7.0.0.7: WebSphere Application Server V7.0 Fix Pack 7 for Windows
7.0.0.7: WebSphere Application Server V7.0 Fix Pack 7 for HP-UX
7.0.0.7: Java SDK 1.6 SR6 Cumulative Fix for WebSphere Application Server
7.0.0.7: WebSphere Application Server V7.0 Fix Pack 7 for Solaris
7.0.0.7: WebSphere Application Server V7.0 Fix Pack 7 for Linux
7.0.0.9: WebSphere Application Server V7.0 Fix Pack 9 for IBM i
7.0.0.9: WebSphere Application Server V7.0 Fix Pack 9 for Windows
7.0.0.9: WebSphere Application Server V7.0 Fix Pack 9 for AIX
7.0.0.9: WebSphere Application Server V7.0 Fix Pack 9 for HP-UX
7.0.0.9: Java SDK 1.6 SR7 Cumulative Fix for WebSphere Application Server
7.0.0.9: WebSphere Application Server V7.0 Fix Pack 9 for Solaris
7.0.0.9: WebSphere Application Server V7.0 Fix Pack 9 for Linux
6.1.0.31: Java SDK 1.5 SR11 Cumulative Fix for WebSphere Application Server
7.0.0.11: WebSphere Application Server V7.0 Fix Pack 11 for IBM i
7.0.0.11: WebSphere Application Server V7.0 Fix Pack 11 for Windows
7.0.0.11: WebSphere Application Server V7.0 Fix Pack 11 for HP-UX
7.0.0.11: WebSphere Application Server V7.0 Fix Pack 11 for AIX
7.0.0.11: Java SDK 1.6 SR7 Cumulative Fix for WebSphere Application Server
7.0.0.11: WebSphere Application Server V7.0 Fix Pack 11 for Solaris
7.0.0.11: WebSphere Application Server V7.0 Fix Pack 11 for Linux
6.1.0.33: Java SDK 1.5 SR12 FP1 Cumulative Fix for WebSphere
6.1.0.29: Java SDK 1.5 SR11 Cumulative Fix for WebSphere Application Server
7.0.0.13: WebSphere Application Server V7.0 Fix Pack 13 for AIX
7.0.0.13: WebSphere Application Server V7.0 Fix Pack 13 for HP-UX
7.0.0.13: WebSphere Application Server V7.0 Fix Pack 13 for IBM i
7.0.0.13: WebSphere Application Server V7.0 Fix Pack 13 for Linux
7.0.0.13: WebSphere Application Server V7.0 Fix Pack 13 for Solaris
7.0.0.13: WebSphere Application Server V7.0 Fix Pack 13 for Windows
7.0.0.13: Java SDK 1.6 SR8FP1 Cumulative Fix for WebSphere Application Server
6.1.0.35: Java SDK 1.5 SR12 FP2 Cumulative Fix for WebSphere
7.0.0.15: WebSphere Application Server V7.0 Fix Pack 15 for AIX
7.0.0.15: Java SDK 1.6 SR9 Cumulative Fix for WebSphere Application Server
7.0.0.15: WebSphere Application Server V7.0 Fix Pack 15 for HP-UX
7.0.0.15: WebSphere Application Server V7.0 Fix Pack 15 for IBM i
7.0.0.15: WebSphere Application Server V7.0 Fix Pack 15 for Linux
7.0.0.15: WebSphere Application Server V7.0 Fix Pack 15 for Solaris
7.0.0.15: WebSphere Application Server V7.0 Fix Pack 15 for Windows
6.1.0.37: Java SDK 1.5 SR12 FP3 Cumulative Fix for WebSphere
7.0.0.17: WebSphere Application Server V7.0 Fix Pack 17
7.0.0.17: Java SDK 1.6 SR9 FP1 Cumulative Fix for WebSphere Application Server
6.1.0.39: Java SDK 1.5 SR12 FP4 Cumulative Fix for WebSphere Application Server
7.0.0.19: WebSphere Application Server V7.0 Fix Pack 19
6.1.0.41: Java SDK 1.5 SR12 FP5 Cumulative Fix for WebSphere Application Server
7.0.0.21: WebSphere Application Server V7.0 Fix Pack 21
7.0.0.1: Java SDK 1.6 SR3 Cumulative Fix for WebSphere Application Server
6.1.0.43: Java SDK 1.5 SR13 Cumulative Fix for WebSphere Application Server
7.0.0.23: WebSphere Application Server V7.0 Fix Pack 23
7.0.0.25: WebSphere Application Server V7.0 Fix Pack 25
6.1.0.45: Java SDK 1.5 SR14 Cumulative Fix for WebSphere Application Server
7.0.0.27: WebSphere Application Server V7.0 Fix Pack 27
7.0.0.29: WebSphere Application Server V7.0 Fix Pack 29
6.1.0.47: WebSphere Application Server V6.1 Fix Pack 47
6.1.0.47: Java SDK 1.5 SR16 Cumulative Fix for WebSphere Application Server
7.0.0.31: WebSphere Application Server V7.0 Fix Pack 31
7.0.0.27: Java SDK 1.6 SR13 FP2 Cumulative Fix for WebSphere Application Server
7.0.0.33: WebSphere Application Server V7.0 Fix Pack 33
7.0.0.35: WebSphere Application Server V7.0 Fix Pack 35
Obtain the fix for this APAR.

Subscribe

You can track all active APARs for this component.

APAR status

  • Closed as program error.

Error description

  • The servant abends caused by hung threads in JDBC Driver.
    After 5 minutes of seeing the hang, the control region (CR)
    kills the servant.
    
    The method com.ibm.ws.rsadapter.spi.WSManagedConnectionFactor
    Impl$Equals.checkPrivateCredentials() in J2C component is
    causing a deadlock.  This method checkPrivateCredentials() is
    expected to be invoked in multi threaded environment.
    
    A javacore taken at the time of the deadlock will show the
    following:
    
    1LKDEADLOCK    Deadlock detected !!!
    NULL           ---------------------
    NULL
    2LKDEADLOCKTHR  Thread "ORB.thread.pool : 0"
    (0x00000000412BCC00)
    3LKDEADLOCKWTR    is waiting for:
    4LKDEADLOCKMON      sys_mon_t:0x00000000826641D0 infl_mon_t:
    0x0000000082664240:
    4LKDEADLOCKOBJ
    java/util/Collections$SynchronizedSet@00000000031930E0/000000000
    31930EC:
    3LKDEADLOCKOWN    which is owned by:
    2LKDEADLOCKTHR  Thread "ORB.thread.pool : 4"
    (0x0000000042945800)
    3LKDEADLOCKWTR    which is waiting for:
    4LKDEADLOCKMON      sys_mon_t:0x00000200121AC420 infl_mon_t:
    0x00000200121AC490:
    4LKDEADLOCKOBJ
    java/util/Collections$SynchronizedSet@000000000329CFC8/000000000
    329CFD4:
    3LKDEADLOCKOWN    which is owned by:
    2LKDEADLOCKTHR  Thread "ORB.thread.pool : 0"
    (0x00000000412BCC00)
    
    
    The stack traces of the two threads involved in the deadlock
    show this:
    
    "ORB.thread.pool : 0" TID:0x00000000412BCC00,
    j9thread_t:0x00000200081FD4E0, state:B, prio=5
           (native thread ID:0x5B85, native priority:0x5, native
    policy:UNKNOWN)
        at
    java/util/Collections$SynchronizedCollection.size(Collections.ja
    va:478(Compiled Code))
        at java/util/AbstractSet.equals(AbstractSet.java:51(Compiled
    Code))
        at
    java/util/Collections$SynchronizedSet.equals(Collections.java:83
    1(Compiled Code))
        at
    com/ibm/ws/rsadapter/spi/WSManagedConnectionFactoryImpl$Equals.c
    heckPrivateCredentials(WSManagedConnectionFactoryImpl.java:3227(
    Compiled Code))
        at
    com/ibm/ws/rsadapter/spi/WSManagedConnectionFactoryImpl$Equals.r
    un(WSManagedConnectionFactoryImpl.java:3187(Compiled Code))
        at
    com/ibm/ws/security/util/AccessController.doPrivileged(AccessCon
    troller.java:63(Compiled Code))
        at
    com/ibm/ws/rsadapter/spi/WSManagedConnectionFactoryImpl.matchMan
    agedConnections(WSManagedConnectionFactoryImpl.java:2087(Compile
    d Code))
        at
    com/ibm/ejs/j2c/FreePool.getMCWrapperFromMatch(FreePool.java:179
    8(Compiled Code))
        at
    com/ibm/ejs/j2c/FreePool.getFreeConnection(FreePool.java:873(Com
    piled Code))
        at
    com/ibm/ejs/j2c/PoolManager.reserve(PoolManager.java:2153(Compil
    ed Code))
    ...
    
    
    "ORB.thread.pool : 4" TID:0x0000000042945800,
    j9thread_t:0x00000200098C5CE0, state:B, prio=5
           (native thread ID:0x18ED, native priority:0x5, native
    policy:UNKNOWN)
        at
    java/util/Collections$SynchronizedCollection.size(Collections.ja
    va:478(Compiled Code))
        at java/util/AbstractSet.equals(AbstractSet.java:51(Compiled
    Code))
        at
    java/util/Collections$SynchronizedSet.equals(Collections.java:83
    1(Compiled Code))
        at
    com/ibm/ws/rsadapter/spi/WSManagedConnectionFactoryImpl$Equals.c
    heckPrivateCredentials(WSManagedConnectionFactoryImpl.java:3227(
    Compiled Code))
        at
    com/ibm/ws/rsadapter/spi/WSManagedConnectionFactoryImpl$Equals.r
    un(WSManagedConnectionFactoryImpl.java:3187(Compiled Code))
        at
    com/ibm/ws/security/util/AccessController.doPrivileged(AccessCon
    troller.java:63(Compiled Code))
        at
    com/ibm/ws/rsadapter/spi/WSManagedConnectionFactoryImpl.matchMan
    agedConnections(WSManagedConnectionFactoryImpl.java:2087(Compile
    d Code))
        at
    com/ibm/ejs/j2c/FreePool.getMCWrapperFromMatch(FreePool.java:179
    8(Compiled Code))
        at
    com/ibm/ejs/j2c/FreePool.getFreeConnection(FreePool.java:873(Com
    piled Code))
        at
    com/ibm/ejs/j2c/PoolManager.reserve(PoolManager.java:2153(Compil
    ed Code))
    ...
    

Local fix

Problem summary

  • ****************************************************************
    * USERS AFFECTED:  WebSphere Application Server V6.1.0         *
    *                  for z/OS & distributed platforms JCA        *
    *                  and JDBC                                    *
    ****************************************************************
    * PROBLEM DESCRIPTION: Selectable fix for servant abends       *
    *                      caused by hung threads in WebSphere     *
    *                      Application Server adapter component.   *
    ****************************************************************
    * RECOMMENDATION:                                              *
    ****************************************************************
    Method com.ibm.ws.rsadapter.spi.WSManagedConnectionFactor
    Impl$Equals.checkPrivateCredentials() or
    checkPublicCrentials() in adapter component is
    causing a deadlock.  If a customer has java cores that contain
    this deadlock they need to enable this fix.  To enable
    the fix, set custom property "useRRASetEquals" of type
    java.lang.String to true on the datasource.
    

Problem conclusion

  • The JDK's implementation of Subject.equals() is
    synchronized. This fix implements an unsynchronized equals.
    Only enable this fix if the java cores contain a deadlock in
    checkPrivateCredentials() or checkPublicCredentials().
    
    If a customer has java cores that contain this deadlock they
    need to enable this fix.  To enable the fix, set custom
    property "useRRASetEquals" of type java.lang.String to true on
    the datasource.
    
    APAR PK69110 is currently targeted for inclusion in Service
    Level (Fix Pack) 6.1.0.23 of WebSphere Application Server V6.1
    for z/OS.
    
    Please refer to URL:
    //www.ibm.com/support/docview.wss?rs=404&uid=swg27006970
    for Fix Pack availability.
    

Temporary fix

Comments

APAR Information

  • APAR number

    PK69110

  • Reported component name

    WEBSPHERE FOR Z

  • Reported component ID

    5655I3500

  • Reported release

    610

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt

  • Submitted date

    2008-07-16

  • Closed date

    2008-10-29

  • Last modified date

    2012-05-29

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

    PK75897

Fix information

  • Fixed component name

    WEBSPHERE FOR Z

  • Fixed component ID

    5655I3500

Applicable component levels

  • R610 PSY UK44403

       UP09/03/21 P F903

Fix is available

  • Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.



Document information

More support for: WebSphere Application Server for z/OS
General

Software version: 6.1

Reference #: PK69110

Modified date: 29 May 2012


Translate this page: