IBM Support

PI17533: JMS MESSAGE PRODUCER AND CONSUMER APPLICATIONS FAIL TO PRODUCE AND CONSUME MESSAGES FROM SERVICE INTEGRATION BUS DESTINATION

Fixes are available

8.5.5.4: WebSphere Application Server V8.5.5 Fix Pack 4
PI17533;8.5.5: jms message producer and consumer applications fail to produce an
8.0.0.10: WebSphere Application Server V8.0 Fix Pack 10
7.0.0.37: WebSphere Application Server V7.0 Fix Pack 37
8.5.5.5: WebSphere Application Server V8.5.5 Fix Pack 5
8.5.5.6: WebSphere Application Server V8.5.5 Fix Pack 6
8.0.0.11: WebSphere Application Server V8.0 Fix Pack 11
8.5.5.7: WebSphere Application Server V8.5.5 Fix Pack 7
7.0.0.39: WebSphere Application Server V7.0 Fix Pack 39
8.5.5.8: WebSphere Application Server V8.5.5 Fix Pack 8
8.0.0.12: WebSphere Application Server V8.0 Fix Pack 12
8.5.5.9: WebSphere Application Server V8.5.5 Fix Pack 9
7.0.0.41: WebSphere Application Server V7.0 Fix Pack 41
8.5.5.10: WebSphere Application Server V8.5.5 Fix Pack 10
8.5.5.11: WebSphere Application Server V8.5.5 Fix Pack 11
8.0.0.13: WebSphere Application Server V8.0 Fix Pack 13
7.0.0.43: WebSphere Application Server V7.0 Fix Pack 43
8.5.5.12: WebSphere Application Server V8.5.5 Fix Pack 12
8.0.0.14: WebSphere Application Server V8.0 Fix Pack 14
8.5.5.13: WebSphere Application Server V8.5.5 Fix Pack 13
7.0.0.45: WebSphere Application Server V7.0 Fix Pack 45
8.0.0.15: WebSphere Application Server V8.0 Fix Pack 15
7.0.0.45: Java SDK 1.6 SR16 FP60 Cumulative Fix for WebSphere Application Server
7.0.0.37: Java SDK 1.6 SR16 FP3 Cumulative Fix for WebSphere Application Server
7.0.0.39: Java SDK 1.6 SR16 FP7 Cumulative Fix for WebSphere Application Server
7.0.0.41: Java SDK 1.6 SR16 FP20 Cumulative Fix for WebSphere Application Server
7.0.0.43: Java SDK 1.6 SR16 FP41 Cumulative Fix for WebSphere Application Server
8.5.5.14: WebSphere Application Server V8.5.5 Fix Pack 14

Subscribe

You can track all active APARs for this component.

APAR status

  • Closed as program error.

Error description

  • JMS application is configured to send messages to Websphere
    Application Server v7.0.x Service Integration Bus destination
    and an MDB is configured to consume messages from the
    destination.The message producer and consumer application
    suddenly stop producing and consuming messages from the
    destination.Javacores collected from the server hosting the
    applications and the server hosting the messaging engine
    show the threads in blocked state with the following stack:
    
    On the server hosting the messaging engine,7 out of 8
    SpillDispatcher threads were hung while adding messages to
    the filestore:
    
    at java/lang/Object.wait(Native Method)
    at java/lang/Object.wait(Object.java:167(Compiled Code))
    at
    com/ibm/ws/objectManager/AbstractSingleFileObjectStore.reserve(A
    bstractSingleFileObjectStore.java:718(Compiled Code))
    at
    com/ibm/ws/objectManager/ManagedObject.reserveSpaceInStore(Manag
    edObject.java:663(Compiled Code))
    at
    com/ibm/ws/objectManager/ManagedObject.getSerializedBytes(Manage
    dObject.java:637(Compiled Code))
    at
    com/ibm/ws/objectManager/InternalTransaction.add(InternalTransac
    tion.java:780(Compiled Code))
    at
    com/ibm/ws/objectManager/Transaction.add(Transaction.java:243(Co
    mpiled Code))
    at
    com/ibm/ws/sib/msgstore/persistence/objectManager/PersistableImp
    l.addToStore(PersistableImpl.java:454(Compiled Code))
    at
    com/ibm/ws/sib/msgstore/persistence/objectManager/BatchingContex
    tImpl.insert(BatchingContextImpl.java:205(Compiled Code))
    at
    com/ibm/ws/sib/msgstore/task/AddTask.persist(AddTask.java:371(Co
    mpiled Code))
    at
    com/ibm/ws/sib/msgstore/persistence/dispatcher/SpillDispatcher$S
    pillDispatcherThread.writeBatch(SpillDispatcher.java:1669(Compil
    ed Code))
    at
    com/ibm/ws/sib/msgstore/persistence/dispatcher/SpillDispatcher$S
    pillDispatcherThread.run(SpillDispatcher.java:711(Compiled
    Code))
    at java/lang/Thread.run(Thread.java:738(Compiled Code))
    
    
    On the server hosting the message producer and consumer
    applications 90+ threads were found with the following stack:
    
    Consumer threads:
    
    at java/lang/Object.wait(Native Method)
    at java/lang/Object.wait(Object.java:167(Compiled Code))
    at
    com/ibm/ws/sib/jfapchannel/impl/ExchangeReceiveListener.waitToCo
    mplete(ExchangeReceiveListener.java:227(Compiled Code))
    at
    com/ibm/ws/sib/jfapchannel/impl/ConversationImpl.exchange(Conver
    sationImpl.java:988(Compiled Code))
    at
    com/ibm/ws/sib/comms/common/JFAPCommunicator.jfapExchange(JFAPCo
    mmunicator.java:386(Compiled Code))
    at
    com/ibm/ws/sib/comms/client/ConsumerSessionProxy.performReceive(
    ConsumerSessionProxy.java:914(Compiled Code))
    at
    com/ibm/ws/sib/comms/client/ConsumerSessionProxy._receiveWithWai
    t(ConsumerSessionProxy.java:825(Compiled Code))
    at
    com/ibm/ws/sib/comms/client/ConsumerSessionProxy.receiveWithWait
    (ConsumerSessionProxy.java:733(Compiled Code))
    at
    com/ibm/ws/sib/api/jms/impl/JmsMsgConsumerImpl.receiveInboundMes
    sage(JmsMsgConsumerImpl.java:1114(Compiled Code))
    at
    com/ibm/ws/sib/api/jms/impl/JmsMsgConsumerImpl.receive(JmsMsgCon
    sumerImpl.java:481(Compiled Code))
    at
    com/ibm/ws/sib/api/jms/impl/JmsQueueReceiverImpl.receive(JmsQueu
    eReceiverImpl.java(Compiled Code))
    
    Producer threads:
    
    at
    com/ibm/ws/sib/comms/client/ConnectionProxy._createProducerSessi
    on(ConnectionProxy.java:1007(Compiled Code))
    at
    com/ibm/ws/sib/comms/client/ConnectionProxy.createProducerSessio
    n(ConnectionProxy.java:932(Compiled Code))
    at
    com/ibm/ws/sib/api/jms/impl/JmsMsgProducerImpl.<init>(JmsMsgProd
    ucerImpl.java:369(Compiled Code))
    at
    com/ibm/ws/sib/api/jms/impl/JmsQueueSenderImpl.<init>(JmsQueueSe
    nderImpl.java:60(Compiled Code))
    at
    com/ibm/ws/sib/api/jms/impl/JmsQueueSessionImpl.instantiateProdu
    cer(JmsQueueSessionImpl.java:224(Compiled Code))
    at
    com/ibm/ws/sib/api/jms/impl/JmsSessionImpl.createProducer(JmsSes
    sionImpl.java:866(Compiled Code))
    

Local fix

Problem summary

  • ****************************************************************
    * USERS AFFECTED:  Users of the default messaging provider     *
    *                  for IBM WebSphere Application Server V7.0,  *
    *                  V8.0 and V8.5                               *
    ****************************************************************
    * PROBLEM DESCRIPTION: Message producer application suddenly   *
    *                      stops producing messages to the         *
    *                      destination because dispacher threads   *
    *                      are hung.                               *
    ****************************************************************
    * RECOMMENDATION:                                              *
    ****************************************************************
    When persisting messages into the filestore, an object manager
    needs to ensure free space in the filestore to store the
    messages. So the "SpillDispatcher" thread is indefinitely
    waiting in AbstractSingleFileObjectStore.reserve(). Another
    thread notifies the waiting thread based on a condition which
    is controlled by a boolan variable which is not thread safe.
    So the waiting "SpillDispatcher" thread may not get notified
    and waits forever and this may eventually happen to all the
    dispacther threads.
    When all the dispacther threads are hung, the message producer
    hangs as there are no dispacther threads to persist messages.
    Consumer application waits on receiveWithWait() because there
    are no messages in the destination.
    This is a rare problem due to timing especially when there are
    more CPUs. This problem is applicable if file store is being
    used as a message store.
    

Problem conclusion

  • Source code has been modified to ensure that the boolean
    variable is thread safe.
    
    The fix for this APAR is currently targeted for inclusion in
    fix packs 7.0.0.37, 8.0.0.10, and 8.5.5.4. Please refer to the
    Recommended Updates page for delivery information:
    http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980
    

Temporary fix

Comments

APAR Information

  • APAR number

    PI17533

  • Reported component name

    WAS SIB & SIBWS

  • Reported component ID

    620800101

  • Reported release

    300

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt

  • Submitted date

    2014-05-08

  • Closed date

    2014-08-18

  • Last modified date

    2014-08-18

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    WAS SIB & SIBWS

  • Fixed component ID

    620800101

Applicable component levels

  • R800 PSY

       UP

  • R850 PSY

       UP



Document information

More support for: WebSphere Application Server
Service Integration Technology

Software version: 7.0

Reference #: PI17533

Modified date: 18 August 2014