PM62615: MESSAGES BUILD UP ON SERVICE INTEGRATION BUS TOPICSPACE

PM62615; 8.0.0.5: messages build up on websphere application server v7.0 service
8.5.0.2: WebSphere Application Server V8.5 Fix Pack 2
8.0.0.6: WebSphere Application Server V8.0 Fix Pack 6

APAR status

• Closed as program error.

Error description

• A JMS application publishes messages to a WebSphere
  Application Server v7.0 Service Integration Bus Topic.
  Messages on the topicspace start building up when the
  number of transactions per second is increased. Even
  when the transactions per second is ramped down, the
  messages are consumed very slowly.
  Messages reporting Hung threads and HMGR0152W messages are
  written to the SystemOut.log:
  
  WSVR0605W: Thread "SIBFAPInboundThreadPool : 3" (00000067)
  has been active for 618131 milliseconds and may be hung.
  There is/are 1 thread(s) in total in the server that may
  be hung.
  at java.lang.Object.wait(Native Method)
  at java.lang.Object.wait(Object.java:167)
  at com.ibm.ws.sib.jfapchannel.impl.rldispatcher.Receive
    ListenerDispatchBarrier.pass(ReceiveListenerDispatchBarrier.
    java:60)
  at com.ibm.ws.sib.jfapchannel.impl.rldispatcher.Receive
    ListenerDispatchQueue.enqueue(ReceiveListenerDispatch
    Queue.java:313)
  at com.ibm.ws.sib.jfapchannel.impl.rldispatcher.Receive
    ListenerDispatcher.queueInvocationCommon(ReceiveListener
    Dispatcher.java:571)
  at com.ibm.ws.sib.jfapchannel.impl.rldispatcher.Receive
    ListenerDispatcher.queueDataReceivedInvocation(Receive
    ListenerDispatcher.java:667)
  at com.ibm.ws.sib.jfapchannel.impl.InboundTransmissionParser.
    dispatchToConversationListenerMethod(InboundTransmission
    Parser.java:1157)
  at com.ibm.ws.sib.jfapchannel.impl.InboundTransmissionParser.
    dispatchToConversation(InboundTransmissionParser.java:996)
  at com.ibm.ws.sib.jfapchannel.impl.InboundTransmissionParser.
    parseConversationPayload(InboundTransmissionParser.java:
    732)
  at com.ibm.ws.sib.jfapchannel.impl.InboundTransmissionParser.
    parse(InboundTransmissionParser.java:293)
  at com.ibm.ws.sib.jfapchannel.impl.ConnectionReadCompleted
    Callback.complete(ConnectionReadCompletedCallback.java:249)
  at com.ibm.ws.sib.jfapchannel.framework.impl.CFWIOReadRequest
    Context$1.complete(CFWIOReadRequestContext.java:198)
  at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.
    futureCompleted(AioReadCompletionListener.java:165)
  at com.ibm.io.async.AbstractAsyncFuture.invokeCallback
    (AbstractAsyncFuture.java:217)
  at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions
    (AsyncChannelFuture.java:161)
  at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:
    138)
  at com.ibm.io.async.ResultHandler.complete(ResultHandler.
    java:204)
  at com.ibm.io.async.ResultHandler.runEventProcessingLoop
    (ResultHandler.java:816)
  at com.ibm.io.async.ResultHandler$2.run(ResultHandler.
    java:905)
  at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:
    1550)
  
  HMGR0152W: CPU Starvation detected. Current thread
  scheduling  delay is 31 seconds.
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED:  Users of the default messaging provider     *
  *                  for IBM WebSphere Application Server        *
  *                  version 7.0,8.0 and 8.5                     *
  ****************************************************************
  * PROBLEM DESCRIPTION: Messages stuck on Messaging engine      *
  *                      (ME) hosting publication point          *
  ****************************************************************
  * RECOMMENDATION:                                              *
  ****************************************************************
  This problem is seen in a publish subscribe environment. The
  messages were not flowing from the source ME (the point where
  the messages were published) to the destination ME (the point
  where the durable subscription home was defined). This was
  caused due to a gap in ticks( internal state of the ME) at the
  source ME (where the messages are stuck). The messages remain
  in pending acknowledgment mode. This problem comes because
  the source ME has just now discovered a subscriber is added at
  the destination ME for first time. On the discovery of this,
  at the source ME a gap is formed in tick range ( internal book
  keeping data structure which is used to transfer messages
  between MEs).
  

Problem conclusion

• The code has been modified to fill the gap in the tick range.
  This APAR will ensure that message flow between MEs .
  
  The fix for this APAR is currently targeted for inclusion in
  fix packs 7.0.0.27 ,8.0.0.7 and 8.5.0.2. Please refer to the
  Recommended Updates page for delivery information:
  http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  WAS SIB & SIBWS

• Fixed component ID

  620800101

Applicable component levels

• R300 PSY

     UP

• R800 PSY

     UP