IBM Support

PM49608: IN A CLUSTER ENVIRONMENT WITH TWO ADJUNCT REGIONS WHEN SHUTTING ONE ADJUNCT DOWN, THE SECOND ADJUNCT HANGS.

Fixes are available

8.0.0.3: WebSphere Application Server V8.0 Fix Pack 3
7.0.0.23: WebSphere Application Server V7.0 Fix Pack 23
8.0.0.4: WebSphere Application Server V8.0 Fix Pack 4
7.0.0.25: WebSphere Application Server V7.0 Fix Pack 25
8.0.0.5: WebSphere Application Server V8.0 Fix Pack 5
7.0.0.27: WebSphere Application Server V7.0 Fix Pack 27
8.0.0.6: WebSphere Application Server V8.0 Fix Pack 6
7.0.0.29: WebSphere Application Server V7.0 Fix Pack 29
8.0.0.7: WebSphere Application Server V8.0 Fix Pack 7
8.0.0.8: WebSphere Application Server V8.0 Fix Pack 8
7.0.0.31: WebSphere Application Server V7.0 Fix Pack 31
7.0.0.27: Java SDK 1.6 SR13 FP2 Cumulative Fix for WebSphere Application Server
7.0.0.33: WebSphere Application Server V7.0 Fix Pack 33
8.0.0.9: WebSphere Application Server V8.0 Fix Pack 9
7.0.0.35: WebSphere Application Server V7.0 Fix Pack 35
8.0.0.10: WebSphere Application Server V8.0 Fix Pack 10
7.0.0.37: WebSphere Application Server V7.0 Fix Pack 37
8.0.0.11: WebSphere Application Server V8.0 Fix Pack 11
7.0.0.39: WebSphere Application Server V7.0 Fix Pack 39
8.0.0.12: WebSphere Application Server V8.0 Fix Pack 12
7.0.0.41: WebSphere Application Server V7.0 Fix Pack 41
8.0.0.13: WebSphere Application Server V8.0 Fix Pack 13
7.0.0.43: WebSphere Application Server V7.0 Fix Pack 43
8.0.0.14: WebSphere Application Server V8.0 Fix Pack 14
7.0.0.45: WebSphere Application Server V7.0 Fix Pack 45
8.0.0.15: WebSphere Application Server V8.0 Fix Pack 15
7.0.0.23: Java SDK 1.6 SR10 FP1 Cumulative Fix for WebSphere
7.0.0.25: Java SDK 1.6 SR11 Cumulative Fix for WebSphere Application Server
7.0.0.27: Java SDK 1.6 SR12 Cumulative Fix for WebSphere Application Server
7.0.0.29: Java SDK 1.6 SR13 FP2 Cumulative Fix for WebSphere Application Server
7.0.0.45: Java SDK 1.6 SR16 FP60 Cumulative Fix for WebSphere Application Server
7.0.0.31: Java SDK 1.6 SR15 Cumulative Fix for WebSphere Application Server
7.0.0.35: Java SDK 1.6 SR16 FP1 Cumulative Fix for WebSphere Application Server
7.0.0.37: Java SDK 1.6 SR16 FP3 Cumulative Fix for WebSphere Application Server
7.0.0.39: Java SDK 1.6 SR16 FP7 Cumulative Fix for WebSphere Application Server
7.0.0.41: Java SDK 1.6 SR16 FP20 Cumulative Fix for WebSphere Application Server
7.0.0.43: Java SDK 1.6 SR16 FP41 Cumulative Fix for WebSphere Application Server

Subscribe

You can track all active APARs for this component.

APAR status

  • Closed as program error.

Error description

  • The customer has a WAS cluster with two adjunct regions. The
    problem occurred when the customer was closing the first
    adjunct, the second one is taking all the queued messages and
    hangs.
    .
    The change has analyzed the javacores provided by the customer
    and they are seeing a deadlock that is being reported with the
    following stack:
    .
    
    3XMTHREADINFO      "Default : N" J9VMThread:X,
    j9thread_t:X,
    java/lang/Thread:X,
    state:B, prio=5
    3XMTHREADINFO1            (native thread ID:X, native
    priority:0x5, native policy:UNKNOWN)
    3XMTHREADINFO3           Java callstack:
    4XESTACKTRACE                at
    com/ibm/ws/sib/ra/inbound/impl/SibRaCommonEndpointActivation.cl
    earTimer(SibRaCommonEndpointActivation.java:443)
    --> waiting to get a lock on timerLock
    4XESTACKTRACE                at
    com/ibm/ws/sib/ra/inbound/impl/SibRaCommonEndpointActivation.dr
    opConnection(SibRaCommonEndpointActivation.java:1416)
    --> obtained a lock on _connections
    4XESTACKTRACE                at
    com/ibm/ws/sib/ra/inbound/impl/SibRaCommonEndpointActivation.se
    ssionError(SibRaCommonEndpointActivation.java:1329)
    4XESTACKTRACE                at
    com/ibm/ws/sib/ra/inbound/impl/SibRaMessagingEngineConnection$S
    ibRaConnectionEventThread.executeCallback(SibRaMessagingEngineC
    onnection.java:1423)
    4XESTACKTRACE                at
    com/ibm/ws/sib/ra/inbound/impl/SibRaMessagingEngineConnection$S
    ibRaConnectionEventThread.run(SibRaMessagingEngineConnection.ja
    va:1377)
    4XESTACKTRACE                at
    com/ibm/ws/util/ThreadPool$Worker.run(ThreadPool.java:1550)
    3XMTHREADINFO3         -->  No native callstack available
    for this thread
    
    
    3XMTHREADINFO      "Thread-2462" J9VMThread:X,
    j9thread_t:X,java/lang/Thread:X
    40,state:B, prio=5
    3XMTHREADINFO1            (native thread ID:X, native
    priority:0x5, native policy:UNKNOWN)
    3XMTHREADINFO3           Java callstack:
    4XESTACKTRACE                at
    com/ibm/ws/sib/ra/inbound/impl/SibRaCommonEndpointActivation.co
    nnectUsingTrmWithTargetData(SibRaCommonEndpointActivation.java:
    655(Compiled Code)) ---> waiting to get a lock on _connections
    4XESTACKTRACE                at
    com/ibm/ws/sib/ra/inbound/impl/SibRaCommonEndpointActivation.co
    nnectUsingTrm(SibRaCommonEndpointActivation.java:582(Compiled
    Code))
    4XESTACKTRACE                at
    com/ibm/ws/sib/ra/inbound/impl/SibRaCommonEndpointActivation.co
    nnect(SibRaCommonEndpointActivation.java:486(Compiled Code))
    4XESTACKTRACE                at
    com/ibm/ws/sib/ra/inbound/impl/SibRaCommonEndpointActivation.ch
    eckMEs(SibRaCommonEndpointActivation.java:366(Compiled Code))
    4XESTACKTRACE                at
    com/ibm/ws/sib/ra/inbound/impl/SibRaCommonEndpointActivation.ti
    merLoop(SibRaCommonEndpointActivation.java:325(Compiled Code))
    4XESTACKTRACE                at
    com/ibm/ws/sib/ra/inbound/impl/SibRaCommonEndpointActivation$1.
    run(SibRaCommonEndpointActivation.java:413(Compiled Code))
    --> holding a timerLock
    4XESTACKTRACE                at
    java/util/Timer$TimerImpl.run(Timer.java:291(Compiled Code))
    3XMTHREADINFO3    No native callstack available for this thread
    .
    

Local fix

Problem summary

  • ****************************************************************
    * USERS AFFECTED:  Users of the default messaging provider     *
    *                  for                                         *
    *                  IBM WebSphere Application Server            *
    ****************************************************************
    * PROBLEM DESCRIPTION: In a WebSphere Application Server       *
    *                      Cluster of two adjunct regions on a     *
    *                      z/OS environment configuration, a       *
    *                      deadlock was detected when one of the   *
    *                      adjunct regions was closed.             *
    ****************************************************************
    * RECOMMENDATION:                                              *
    ****************************************************************
    In a WebSphere Application Server Cluster of two adjunct
    regions on a z/OS environment configuration, a deadlock was
    detected with the following thread stack due to inconsistency
    in the hierarchy of obtaining locks in different code paths.
    

Problem conclusion

  • The code has been modified to ensure that the locks are
    obtained in the same hierarchy in different code paths and
    hence avoiding the possibility of occurrence of a deadlock.
    
    The fix for this APAR is currently targeted for inclusion in
    fix pack 7.0.0.23 and 8.0.0.3.  Please refer to the
    Recommended Updates page for delivery information:
    http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980
    

Temporary fix

Comments

APAR Information

  • APAR number

    PM49608

  • Reported component name

    WAS SIB & SIBWS

  • Reported component ID

    620800101

  • Reported release

    300

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt

  • Submitted date

    2011-10-07

  • Closed date

    2011-12-08

  • Last modified date

    2012-03-09

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

    PM50402

Fix information

  • Fixed component name

    WAS SIB & SIBWS

  • Fixed component ID

    620800101

Applicable component levels

  • R300 PSY

       UP

  • R800 PSY

       UP



Document information

More support for: WebSphere Application Server
Service Integration Technology

Software version: 7.0

Reference #: PM49608

Modified date: 09 March 2012