IBM Support

IT18135: A JMS application thread hangs closing a MessageConsumer when the queue manager is also quiescing

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • A hang condition may occur on an application thread that is
    using the MQ V7.5 classes for JMS application and attempting to
    close a JMS MessageConsumer object, which is associated with a
    JMS MessageListener for asynchronous message delivery, at the
    same time as the MQ classes for JMS receive a notification from
    the queue manager that is shutting down or quiescing.
    
    An FFDC file is generated reporting a
    java.lang.NullPointerException within the MQ classes for JMS
    code:
    
    
    Source Class  :-com.ibm.mq.jmqi.remote.impl.RemoteDispatchThread
    Source Method :- run()
    ProbeID       :- 01
    Thread        :- name=DispatchThread:
    [com.ibm.mq.jmqi.remote.impl.RemoteSession[
    connectionId=414D5143444D4F4D5445535430332020D29C045801270020]]
    priority=5
    group=JMSCCThreadPoolccl=sun.misc.Launcher$AppClassLoader@18b4aa
    c2
    
                                          Data
                                          ----
    |   Description        :-  Unexpected error
    |   Exception Summary  :-  java.lang.NullPointerException[null]
    |   Exception message  :-  <null>
    
                                      Stack trace
                                      -----------
    
    Stack trace to show the location of the FFST call
    |   FFST Location  :-  java.lang.Exception
    |   	at
    com.ibm.msg.client.commonservices.trace.Trace.getCurrentPosition
    (Trace.java:2019)
    |   	at
    com.ibm.msg.client.commonservices.trace.Trace.createFFSTString(T
    race.java:1958)
    |   	at
    com.ibm.msg.client.commonservices.trace.Trace.ffstInternal(Trace
    .java:1847)
    |   	at
    com.ibm.msg.client.commonservices.trace.Trace.ffst(Trace.java:16
    60)
    |   	at
    com.ibm.mq.jmqi.remote.impl.RemoteDispatchThread.run(RemoteDispa
    tchThread.java:391)
    |   	at
    com.ibm.msg.client.commonservices.workqueue.WorkQueueItem.runTas
    k(WorkQueueItem.java:214)
    
    
    A Javacore (thread dump) showing threads within the JVM will
    show the hung application thread that is in an API call to
    MQQueueReceiver.close() with the following Java call stack:
    
    "THREADNAME" daemon prio=5 os_prio=0 tid=0x00007fa0dc003000
    nid=0x33fe in Object.wait()
      java.lang.Thread.State: WAITING (on object monitor)
    	at java.lang.Object.wait(Native Method)
    	at java.lang.Object.wait(Object.java:502)
    	at
    com.ibm.mq.jmqi.remote.api.RemoteHconn.waitOnDispatchEvent(Remot
    eHconn.java:502)
    	- locked <0x00000000feecdb88> (a
    com.ibm.mq.jmqi.remote.api.RemoteHconn$DispatchEventSync)
    	at
    com.ibm.mq.jmqi.remote.api.RemoteHconn.dispatchThreadExchange(Re
    moteHconn.java:545)
    	at
    com.ibm.mq.jmqi.remote.api.RemoteFAP.MQCTL(RemoteFAP.java:2562)
    	at
    com.ibm.mq.jmqi.monitoring.JmqiInterceptAdapter.MQCTL(JmqiInterc
    eptAdapter.java:216)
    	at
    com.ibm.msg.client.wmq.internal.WMQConsumerOwnerShadow.suspendAs
    yncService(WMQConsumerOwnerShadow.java:453)
    	at
    com.ibm.msg.client.wmq.internal.WMQSession.suspendAsyncService(W
    MQSession.java:1499)
    	at
    com.ibm.msg.client.wmq.internal.WMQConsumerShadow.close(WMQConsu
    merShadow.java:326)
    	- locked <0x00000000feefadb8> (a java.lang.Object)
    	at
    com.ibm.msg.client.wmq.internal.WMQMessageConsumer.close(WMQMess
    ageConsumer.java:378)
    	at
    com.ibm.msg.client.jms.internal.JmsMessageConsumerImpl.close(Jms
    MessageConsumerImpl.java:266)
    	- locked <0x00000000feefad18> (a
    com.ibm.msg.client.jms.internal.State)
    	at
    com.ibm.msg.client.jms.internal.JmsMessageConsumerImpl.close(Jms
    MessageConsumerImpl.java:185)
    	at
    com.ibm.mq.jms.MQQueueReceiver.close(MQQueueReceiver.java:84)
    

Local fix

  • Three local workarounds are available:
    
    Option 1
    ----------
    Use a transacted JMS Session within the application to create
    the JMS QueueReceiver objects that are then used with the JMS
    MessageListeners.
    
    For example, rather than calling:
      JmsQueueConnectionImpl.createQueueSession(
        false,
        Session.AUTO_ACKNOWLEDGE)
    
    for a non-transacted JMS Session, pass the boolean value:
    
      true
    
    as the first argument to ensure a transacted JMS Session is used
    as messages that are delivered to the application's JMS
    MessageListener are consumed under an MQ syncpoint.
    
    
    Option 2
    ----------
    Use the IBM MQ V8.0.0.5 (or later) classes for JMS.
    
    Some of the MQ V8.0.0.5 classes for JMS code was refactored in
    this version under APAR IT11291 which prevents the
    java.lang.NullPointerException occurring, which leads to the
    hung application thread performing the JMS
    MessageConsumer.close() call.
    
    
    Option 3
    ----------
    Kill and restart the application.
    

Problem summary

  • ****************************************************************
    USERS AFFECTED:
    This issue affects users of the WebSphere MQ V7.5 classes for
    JMS who are consuming messages asynchronously using JMS
    MessageListeners.
    
    This issue does not affect users of the IBM MQ V8.0.0.5 (and
    later) classes for JMS, after APAR IT11291.
    
    
    Platforms affected:
    MultiPlatform
    
    ****************************************************************
    PROBLEM DESCRIPTION:
    When an MQ classes for JMS application attempts to close either
    a JMS MessageConsumer or JMS QueueReceiver object that has a JMS
    MessageListener configured, such that messages are consumed
    asynchronously, then the close API call will block until any
    in-progress or pending message delivery to the JMS
    MessageListener has completed.  This also includes any messages
    that have just been received under an MQ syncpoint but have not
    yet been delivered to the MessageListener.
    
    The MQ classes for JMS use an internal thread, known as the
    "DispatchThread" to deliver messages to application's JMS
    MessageListener.  As part of stopping a MessageConsumer or
    QueueReceiver, the MQ classes for JMS notify the DispatchThread
    of the pending close request to ensure it completes delivery of
    pending messages consumed under syncpoint.  The application
    thread blocks and waits for the DispatchThread to notify it that
    the message delivery work has completed before it stops, which
    allows the close processing to continue.
    
    In the case where a request to shutdown (quiesce) the queue
    manager has been made, the queue manager notifies the MQ classes
    for JMS that it is ending and is no longer accepting new
    connections.  A "notification" type message is put on an
    internal queue (known as a ProxyQueue), which is processed by
    the DispatchThread.  When the application thread notifies the
    DispatchThread of the pending MessageConsumer close call, the
    DispatchThread checks for messages that have just consumed under
    an MQ syncpoint that have also been put to this internal queue.
     It finds that the queue is not empty (as it contains the
    notification from the queue manager) but the DispatchThread
    assumes the internal queue contains a valid MQ message with MQMD
    header.
    
    If the JMS Session is not transacted (i.e., false was passed as
    the first argument in the Connection.createSession(boolean, int)
    API call), then whether or not the message was consumed under an
    MQ syncpoint depends on the persistence of the message.
    
    In this case, the DispatchThread attempted to get the MQMD of
    the message in order to check the value of the MQMD's
    "Persistence" field.  However, because the queue contained a
    notification from the queue manager and not an MQ message, then
    no MQMD existed.  Because of the null MQMD, the operation to
    query the Persistence threw a java.lang.NullPointerException
    which caused the DispatchThread to generate an FFDC and end.
    This meant that it did not notify the waiting application
    thread, which was closing the MessageConsumer, and so this
    thread hung indefinitely.
    

Problem conclusion

  • The MQ classes for JMS have been updated such that any message
    (either an actual MQ message or a notification) held on the
    internal ProxyQueue is delivered to the application's JMS
    MessageListener before a MessageConsumer.close() call can
    complete, regardless of whether the message is persistent,
    non-persistent or consumed under syncpoint.  Therefore, the
    DispatchThread no longer needs to check the Persistence field of
    the MQMD when making this determination and can continue to
    process either actual MQ messages or notifications correctly,
    before notifying the waiting application thread closing the
    MessageConsumer when message delivery has completed.
    
    Please note that the changes in the IBM MQ V8 and V9 product
    releases made under this APAR are only internal serviceability
    changes and not functional changes.
    
    ---------------------------------------------------------------
    The fix is targeted for delivery in the following PTFs:
    
    Version    Maintenance Level
    v7.5       7.5.0.8
    v8.0       8.0.0.7
    v9.0 CD    9.0.2
    v9.0 LTS   9.0.0.1
    
    The latest available maintenance can be obtained from
    'WebSphere MQ Recommended Fixes'
    http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037
    
    If the maintenance level is not yet available information on
    its planned availability can be found in 'WebSphere MQ
    Planned Maintenance Release Dates'
    http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006309
    ---------------------------------------------------------------
    

Temporary fix

Comments

APAR Information

  • APAR number

    IT18135

  • Reported component name

    WMQ BASE MULTIP

  • Reported component ID

    5724H7241

  • Reported release

    750

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    YesHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2016-11-28

  • Closed date

    2016-12-21

  • Last modified date

    2017-06-01

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    WMQ BASE MULTIP

  • Fixed component ID

    5724H7241

Applicable component levels

  • R750 PSY

       UP

[{"Line of Business":{"code":"LOB45","label":"Automation"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSFKSJ","label":"WebSphere MQ"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"7.5"}]

Document Information

Modified date:
26 August 2021