IBM Support

IT30234: Message delivery to an MDB stops when using the MQ resource adapter, later resulting in the JEE server hanging on shutdown

Subscribe to this APAR

By subscribing, you receive periodic emails alerting you to the status of the APAR, along with a link to the fix after it becomes available. You can track this item individually or track all items by product.

Notify me when this APAR changes.

Notify me when an APAR for this component changes.

 

APAR status

  • Closed as program error.

Error description

  • When using an Activation Specification defined against the IBM
    MQ resource adapter deployed into a JEE application server,
    concurrent message delivery to a message-driven bean application
    declines until no further messages are delivered.
    
    If an attempt is subsequently made to stop the application or
    shutdown the application server, the shutdown processing hangs.
     A javacore from the application server when the hang occurs
    shows a thread, which is attempting to deactivate the endpoint
    associated with the application, stuck in a conditional wait
    state trying to close a ServerSession.
    
    For example, if the issue occurs while the IBM MQ resource
    adapter has been deployed into a WebSphere Libery server, then a
    Javacore taken when the application server is trying to shutdown
    would show the following thread in a conditional wait state:
    
    "Default Executor-thread-111" J9VMThread:0x00000000013E7700,
    omrthread_t:0x00007F0B10015938,
    java/lang/Thread:0x00000007F9A02340, state:CW, prio=5
    Waiting on:
    com/ibm/mq/connector/inbound/ServerSessionImpl$IsInUseLock@0x000
    000070FCB3BB0 Owned by: <unowned>
      Heap bytes allocated since last GC cycle=0 (0x0)
      Java callstack:
          at java/lang/Object.wait(Native Method)
          at java/lang/Object.wait(Object.java:189(Compiled Code))
          at
    com/ibm/mq/connector/inbound/ServerSessionImpl.close(ServerSessi
    onImpl.java:303)
          at
    com/ibm/mq/connector/inbound/ServerSessionPoolImpl.closeInternal
    (ServerSessionPoolImpl.java:777)
          at
    com/ibm/mq/connector/inbound/ServerSessionPoolImpl.close(ServerS
    essionPoolImpl.java:711)
          at
    com/ibm/mq/connector/inbound/MessageEndpointDeployment.stopDeliv
    eryASF(MessageEndpointDeployment.java:655)
          at
    com/ibm/mq/connector/inbound/MessageEndpointDeployment.stop(Mess
    ageEndpointDeployment.java:538)
          at
    com/ibm/mq/connector/ResourceAdapterImpl.endpointDeactivation(Re
    sourceAdapterImpl.java:682)
          at
    com/ibm/ws/jca/service/EndpointActivationService.endpointDeactiv
    ation(EndpointActivationService.java:653)
          at
    com/ibm/ws/jca/service/JCAQuiesceListener.serverStopping(JCAQuie
    sceListener.java:69)
          at
    com/ibm/ws/runtime/update/internal/RuntimeUpdateManagerImpl$3.ru
    n(RuntimeUpdateManagerImpl.java:362)
          at
    com/ibm/ws/threading/internal/ExecutorServiceImpl$RunnableWrappe
    r.run(ExecutorServiceImpl.java:239(Compiled Code))
          at
    java/util/concurrent/ThreadPoolExecutor.runWorker(ThreadPoolExec
    utor.java:1160(Compiled Code))
          at
    java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExe
    cutor.java:635)
          at java/lang/Thread.run(Thread.java:812)
    

Local fix

Problem summary

  • ****************************************************************
    USERS AFFECTED:
    This issue affects message-driven bean applications that use the
    IBM MQ resource adapter in a JEE environment.
    
    
    Platforms affected:
    MultiPlatform
    
    ****************************************************************
    PROBLEM DESCRIPTION:
    In IBM MQ V9.1 the IBM MQ resource adapter used a background
    task to retrieve a:
    
        javax.jms.ServerSession
    
    from a:
    
        javax.jms.ServerSessionPool
    
    This background task was executed in a separate thread from the
    "browsing thread", which was notified of an available message
    for the message-driven bean (MDB) application.  If the
    background task took longer than 120 seconds to return with a
    ServerSession from the ServerSessionPool then the browsing
    thread would give up waiting, discard the message reference it
    currently held and return to the queue manager to request
    another message reference.  Note that the queue manager may
    still return the same  reference if the message to which it
    belongs is still available on the MDB's input queue.
    
    After the browsing thread gave up waiting, the background task
    that was attempting to retrieve a ServerSession was not
    cancelled.  When it later completed, it would have obtained a
    ServerSession from the ServerSessionPool but there would have
    been no browsing thread to hand it to.  The result, was that the
    ServerSession would never be released back to the
    ServerSessionPool for reuse.  If this happen repeatedly, then
    the number of ServerSession available in the pool would be
    reduced over time.  This would have meant that fewer MDB
    instances could run in parallel until no more messages could be
    delivered at all, which would be the case when all the
    ServerSessions had been removed and never returned to the
    ServerSessionPool.
    
    If this occurred when using a Activation Specification to
    deliver messages to an MDB, then the application server would
    hang on shutdown.  When the messaging endpoint, created against
    the IBM MQ resource adapter, was requested to shutdown, an
    attempt was made to close the Activation Specification's
    ServerSessionPool.  The shutdown thread would wait until all
    ServerSession objects had been released back to the
    ServerSessionPool, so as not to close a ServerSession while it
    was still in use by an MDB instance.  Because the ServerSessions
    would never be returned to the ServerSessionPool, the shutdown
    thread would wait indefinitely.
    

Problem conclusion

  • The IBM MQ resource adapter has been updated such that the
    browsing thread, which handles references to messages available
    for delivery to message-driven bean applications, requests a
    javax.jms.ServerSession from a javax.jms.ServerSessionPool
    directly without using a background task and separate thread.
    This is the behaviour that was present in the IBM MQ resource
    adapter prior to version 9.0.1.0.
    
    ---------------------------------------------------------------
    The fix is targeted for delivery in the following PTFs:
    
    Version    Maintenance Level
    v9.1 CD    9.1.4
    v9.1 LTS   9.1.0.4
    
    The latest available maintenance can be obtained from
    'WebSphere MQ Recommended Fixes'
    http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006037
    
    If the maintenance level is not yet available information on
    its planned availability can be found in 'WebSphere MQ
    Planned Maintenance Release Dates'
    http://www-1.ibm.com/support/docview.wss?rs=171&uid=swg27006309
    ---------------------------------------------------------------
    

Temporary fix

Comments

APAR Information

  • APAR number

    IT30234

  • Reported component name

    IBM MQ BASE MP

  • Reported component ID

    5724H7271

  • Reported release

    910

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    YesHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2019-09-10

  • Closed date

    2019-09-30

  • Last modified date

    2020-04-14

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    IBM MQ BASE MP

  • Fixed component ID

    5724H7271

Applicable component levels

[{"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSYHRD","label":"IBM MQ"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
15 April 2020