A message is stuck due to an event sequencing lock that cannot be released during failover for WebSphere Process Server (WPS)

Technote (troubleshooting)


Problem(Abstract)

During a failover, a message that has been locked can prevent the processing of both that message and subsequent incoming messages.

Symptom

The SIB queue reveals the following information:

<msgstore:item id="331000003" state="Locked" size="12101"
backoutCount="0" unlockCount="0" position="1" linkState="Linked"
cursorCount="1" sihc="9170917" refCount="0"
refCountDecreasing="false"><msgstore:class>com.ibm.ws.sib.processor.impl
.store.items.MessageItem</msgstore:class><msgstore:priority>0</msgstore:
priority><msgstore:storageStrategy>StoreAlways</msgstore:storageStrategy
><msgstore:expiryTime>0</msgstore:expiryTime><msgstore:sequence>1</msgst
ore:sequence><msgstore:tranID>000001337c402587000000010000034e887c19a830
86fe58a4c8a9d199d943a240b7ef63000001337c402587000000010000034e887c19a830
86fe58a4c8a9d199d943a240b7ef63000000010000000000000000000000000002</msgs
tore:tranID><msgstore:JsMessage>com.ibm.ws.sib.mfp.impl.JsSdoMessageImpl
</msgstore:JsMessage><msgstore:systemMessageId>37F61C8BB95BB151_16650000
4</msgstore:systemMessageId><msgstore:APIMessageId>null</msgstore:APIMes
sageId><msgstore:APICorrelId>null</msgstore:APICorrelId><msgstore:putTim
e>Mon Nov 07 13:18:58 JST
2011</msgstore:putTime><msgstore:data>com.ibm.ws.sib.mfp.impl.JsSdoMessa
geImpl@16e416e4{SysMsgId=37F61C8BB95BB151_166500004}</msgstore:data></ms
gstore:item>



The server log file lists the following messages:

[11/11/16 20:24:28:343 JST] 00000065 ESAsyncConsum 3          
com.ibm.wbiserver.sequencing.service.messaging.
ESAsyncConsumerCallBackImpl                                                                      

invokeLkmgrLock This request message msgId=[37F61C8BB95BB151_176500001]
needs ES. But it did not get the lock, hence it is saved ES db.        
[11/11/16 20:24:28:343 JST] 00000065 ESAsyncConsum <                    
com.ibm.wbiserver.sequencing.service.messaging.
ESAsyncConsumerCallBackImpl                                                                      

invokeLkmgrLock RETURN                                                  
[11/11/16 20:24:28:343 JST] 00000065 ESAsyncConsum <                    
com.ibm.wbiserver.sequencing.service.messaging.
ESAsyncConsumerCallBackImpl                                                                      

consumeMessages RETURN    


Cause

During a failover, the server first stops one node and then switches to a new node while the transaction roll back occurs. But, if the original roll back cannot be completed, a message can remain locked in the SIB queue and the event sequencing lock cannot be released automatically. New incoming messages cannot obtain a lock because the lock on the first message has not been released. Thus, messages are not processed after the failover.


Resolving the problem

To resolve this issue, use the esAdmin command to manually release the lock on the stuck message, as follows:

  1. Find the unique lock ID related to the stuck message, using either of these commands:
    esAdmin listAll  
    esAdmin listLocks [ModuleName]

  2. Release the lock ID, using the following command:
    esAdmin unlock lockId

    where lockId is the unique lock ID returned by the esAdmin listLocks command or the esAdmin listAll command.

For further information, see the Listing, releasing, and deleting locks topic in the WebSphere Process Server Information Center.

Rate this page:

(0 users)Average rating

Document information


More support for:

WebSphere Process Server
Service Component Architecture

Software version:

6.1, 6.2, 7.0

Operating system(s):

AIX, HP-UX, Linux, Linux zSeries, Solaris, Windows, i5/OS, z/OS

Reference #:

1614881

Modified date:

2012-11-08

Translate my page

Machine Translation

Content navigation