PM49608: IN A CLUSTER ENVIRONMENT WITH TWO ADJUNCT REGIONS WHEN SHUTTING ONE ADJUNCT DOWN, THE SECOND ADJUNCT HANGS.

Fixes are available

APAR status

Closed as program error.

Error description

The customer has a WAS cluster with two adjunct regions. The
problem occurred when the customer was closing the first
adjunct, the second one is taking all the queued messages and
hangs.
.
The change has analyzed the javacores provided by the customer
and they are seeing a deadlock that is being reported with the
following stack:
.

3XMTHREADINFO      "Default : N" J9VMThread:X,
j9thread_t:X,
java/lang/Thread:X,
state:B, prio=5
3XMTHREADINFO1            (native thread ID:X, native
priority:0x5, native policy:UNKNOWN)
3XMTHREADINFO3           Java callstack:
4XESTACKTRACE                at
com/ibm/ws/sib/ra/inbound/impl/SibRaCommonEndpointActivation.cl
earTimer(SibRaCommonEndpointActivation.java:443)
--> waiting to get a lock on timerLock
4XESTACKTRACE                at
com/ibm/ws/sib/ra/inbound/impl/SibRaCommonEndpointActivation.dr
opConnection(SibRaCommonEndpointActivation.java:1416)
--> obtained a lock on _connections
4XESTACKTRACE                at
com/ibm/ws/sib/ra/inbound/impl/SibRaCommonEndpointActivation.se
ssionError(SibRaCommonEndpointActivation.java:1329)
4XESTACKTRACE                at
com/ibm/ws/sib/ra/inbound/impl/SibRaMessagingEngineConnection$S
ibRaConnectionEventThread.executeCallback(SibRaMessagingEngineC
onnection.java:1423)
4XESTACKTRACE                at
com/ibm/ws/sib/ra/inbound/impl/SibRaMessagingEngineConnection$S
ibRaConnectionEventThread.run(SibRaMessagingEngineConnection.ja
va:1377)
4XESTACKTRACE                at
com/ibm/ws/util/ThreadPool$Worker.run(ThreadPool.java:1550)
3XMTHREADINFO3         -->  No native callstack available
for this thread


3XMTHREADINFO      "Thread-2462" J9VMThread:X,
j9thread_t:X,java/lang/Thread:X
40,state:B, prio=5
3XMTHREADINFO1            (native thread ID:X, native
priority:0x5, native policy:UNKNOWN)
3XMTHREADINFO3           Java callstack:
4XESTACKTRACE                at
com/ibm/ws/sib/ra/inbound/impl/SibRaCommonEndpointActivation.co
nnectUsingTrmWithTargetData(SibRaCommonEndpointActivation.java:
655(Compiled Code)) ---> waiting to get a lock on _connections
4XESTACKTRACE                at
com/ibm/ws/sib/ra/inbound/impl/SibRaCommonEndpointActivation.co
nnectUsingTrm(SibRaCommonEndpointActivation.java:582(Compiled
Code))
4XESTACKTRACE                at
com/ibm/ws/sib/ra/inbound/impl/SibRaCommonEndpointActivation.co
nnect(SibRaCommonEndpointActivation.java:486(Compiled Code))
4XESTACKTRACE                at
com/ibm/ws/sib/ra/inbound/impl/SibRaCommonEndpointActivation.ch
eckMEs(SibRaCommonEndpointActivation.java:366(Compiled Code))
4XESTACKTRACE                at
com/ibm/ws/sib/ra/inbound/impl/SibRaCommonEndpointActivation.ti
merLoop(SibRaCommonEndpointActivation.java:325(Compiled Code))
4XESTACKTRACE                at
com/ibm/ws/sib/ra/inbound/impl/SibRaCommonEndpointActivation$1.
run(SibRaCommonEndpointActivation.java:413(Compiled Code))
--> holding a timerLock
4XESTACKTRACE                at
java/util/Timer$TimerImpl.run(Timer.java:291(Compiled Code))
3XMTHREADINFO3    No native callstack available for this thread
.

Local fix

Problem summary

****************************************************************
* USERS AFFECTED:  Users of the default messaging provider     *
*                  for                                         *
*                  IBM WebSphere Application Server            *
****************************************************************
* PROBLEM DESCRIPTION: In a WebSphere Application Server       *
*                      Cluster of two adjunct regions on a     *
*                      z/OS environment configuration, a       *
*                      deadlock was detected when one of the   *
*                      adjunct regions was closed.             *
****************************************************************
* RECOMMENDATION:                                              *
****************************************************************
In a WebSphere Application Server Cluster of two adjunct
regions on a z/OS environment configuration, a deadlock was
detected with the following thread stack due to inconsistency
in the hierarchy of obtaining locks in different code paths.

Problem conclusion

The code has been modified to ensure that the locks are
obtained in the same hierarchy in different code paths and
hence avoiding the possibility of occurrence of a deadlock.

The fix for this APAR is currently targeted for inclusion in
fix pack 7.0.0.23 and 8.0.0.3.  Please refer to the
Recommended Updates page for delivery information:
http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980

Temporary fix

Comments

APAR Information

APAR number
PM49608
Reported component name
WAS SIB & SIBWS
Reported component ID
620800101
Reported release
300
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt
Submitted date
2011-10-07
Closed date
2011-12-08
Last modified date
2012-03-09

APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:

PM50402

Fix information

Fixed component name
WAS SIB & SIBWS
Fixed component ID
620800101

Applicable component levels

R300 PSY
UP
R800 PSY
UP

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSEQTP","label":"WebSphere Application Server"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"7.0","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
28 October 2021

Tips

PM49608: IN A CLUSTER ENVIRONMENT WITH TWO ADJUNCT REGIONS WHEN SHUTTING ONE ADJUNCT DOWN, THE SECOND ADJUNCT HANGS.

Fixes are available

Subscribe

APAR status

Closed as program error.

Error description

Local fix

Problem summary

Problem conclusion

Temporary fix

Comments

APAR Information

APAR number

Reported component name

Reported component ID

Reported release

Status

PE

HIPER

Special Attention

Submitted date

Closed date

Last modified date

APAR is sysrouted FROM one or more of the following:

APAR is sysrouted TO one or more of the following:

Fix information

Fixed component name

Fixed component ID

Applicable component levels

R300 PSY

R800 PSY

Document Information

Share your feedback

Need support?