IBM Support

PM41418: WEBSPHERE APPLICATION SERVER MESSAGING ENGINE HANGS AND WSVR0605W MESSAGES REPORTED IN SYSTEMOUT.LOG

Fixes are available

7.0.0.21: WebSphere Application Server V7.0 Fix Pack 21
8.0.0.2: WebSphere Application Server V8.0 Fix Pack 2
8.0.0.3: WebSphere Application Server V8.0 Fix Pack 3
7.0.0.23: WebSphere Application Server V7.0 Fix Pack 23
8.0.0.4: WebSphere Application Server V8.0 Fix Pack 4
7.0.0.25: WebSphere Application Server V7.0 Fix Pack 25
8.0.0.5: WebSphere Application Server V8.0 Fix Pack 5
7.0.0.27: WebSphere Application Server V7.0 Fix Pack 27
8.0.0.6: WebSphere Application Server V8.0 Fix Pack 6
7.0.0.29: WebSphere Application Server V7.0 Fix Pack 29
8.0.0.7: WebSphere Application Server V8.0 Fix Pack 7
8.0.0.8: WebSphere Application Server V8.0 Fix Pack 8
7.0.0.31: WebSphere Application Server V7.0 Fix Pack 31
7.0.0.27: Java SDK 1.6 SR13 FP2 Cumulative Fix for WebSphere Application Server
7.0.0.33: WebSphere Application Server V7.0 Fix Pack 33
8.0.0.9: WebSphere Application Server V8.0 Fix Pack 9
7.0.0.35: WebSphere Application Server V7.0 Fix Pack 35
8.0.0.10: WebSphere Application Server V8.0 Fix Pack 10
7.0.0.37: WebSphere Application Server V7.0 Fix Pack 37
8.0.0.11: WebSphere Application Server V8.0 Fix Pack 11
7.0.0.39: WebSphere Application Server V7.0 Fix Pack 39
8.0.0.12: WebSphere Application Server V8.0 Fix Pack 12
7.0.0.41: WebSphere Application Server V7.0 Fix Pack 41
8.0.0.13: WebSphere Application Server V8.0 Fix Pack 13
7.0.0.43: WebSphere Application Server V7.0 Fix Pack 43
8.0.0.14: WebSphere Application Server V8.0 Fix Pack 14
7.0.0.45: WebSphere Application Server V7.0 Fix Pack 45
8.0.0.15: WebSphere Application Server V8.0 Fix Pack 15
7.0.0.21: Java SDK 1.6 SR9 FP2 Cumulative Fix for WebSphere
7.0.0.23: Java SDK 1.6 SR10 FP1 Cumulative Fix for WebSphere
7.0.0.25: Java SDK 1.6 SR11 Cumulative Fix for WebSphere Application Server
7.0.0.27: Java SDK 1.6 SR12 Cumulative Fix for WebSphere Application Server
7.0.0.29: Java SDK 1.6 SR13 FP2 Cumulative Fix for WebSphere Application Server
7.0.0.45: Java SDK 1.6 SR16 FP60 Cumulative Fix for WebSphere Application Server
7.0.0.31: Java SDK 1.6 SR15 Cumulative Fix for WebSphere Application Server
7.0.0.35: Java SDK 1.6 SR16 FP1 Cumulative Fix for WebSphere Application Server
7.0.0.37: Java SDK 1.6 SR16 FP3 Cumulative Fix for WebSphere Application Server
7.0.0.39: Java SDK 1.6 SR16 FP7 Cumulative Fix for WebSphere Application Server
7.0.0.41: Java SDK 1.6 SR16 FP20 Cumulative Fix for WebSphere Application Server
7.0.0.43: Java SDK 1.6 SR16 FP41 Cumulative Fix for WebSphere Application Server

Subscribe

You can track all active APARs for this component.

APAR status

  • Closed as program error.

Error description

  • A deadlock occurs in a messaging engine using a filestore.
    Javacores collected from the JVM hosting the messaging engine
    show deadlock between two threads with the following stacks:
    
    Thread1 :
    at
    com/ibm/ws/objectManager/ManagedObject.freeLatestSerializedBytes
    (ManagedObject.java:765(Compiled Code))
    at
    com/ibm/ws/objectManager/SingleFileObjectStore.write(SingleFileO
    bjectStore.java:595(Compiled Code))
    at
    com/ibm/ws/objectManager/AbstractSingleFileObjectStore.updateDir
    ectory(AbstractSingleFileObjectStore.java:1222(Compiled Code))
    at
    com/ibm/ws/objectManager/AbstractSingleFileObjectStore.flush(Abs
    tractSingleFileObjectStore.java:1124(Compiled Code))
    at
    com/ibm/ws/objectManager/ObjectManagerState.performCheckpoint(Ob
    jectManagerState.java:923(Compiled Code))
    at
    com/ibm/ws/objectManager/ObjectManagerState.access$500(ObjectMan
    agerState.java:52(Compiled Code))
    at
    com/ibm/ws/objectManager/ObjectManagerState$CheckpointHelper.run
    (ObjectManagerState.java:3212)
    at java/lang/Thread.run(Thread.java:736)
    
    Thread 2
    at
    com/ibm/ws/objectManager/AbstractSingleFileObjectStore.reserve(A
    bstractSingleFileObjectStore.java:752(Compiled Code))
    at
    com/ibm/ws/objectManager/ManagedObject.commit(ManagedObject.java
    :1462(Compiled Code))
    at
    com/ibm/ws/objectManager/LinkedList$Link.commit(LinkedList.java:
    3267(Compiled Code))
    at
    com/ibm/ws/objectManager/InternalTransaction.commit(InternalTran
    saction.java:1949(Compiled Code))
    at
    com/ibm/ws/objectManager/Transaction.commit(Transaction.java:539
    (Compiled Code))
    at
    com/ibm/ws/sib/msgstore/persistence/objectManager/BatchingContex
    tImpl.executeBatch(BatchingContextImpl.java:621(Compiled Code))
    at
    com/ibm/ws/sib/msgstore/persistence/objectManager/PersistentMess
    ageStoreImpl.commit(PersistentMessageStoreImpl.java:1729(Compile
    d Code))
    at
    com/ibm/ws/sib/msgstore/transactions/MSDelegatingLocalTransactio
    n.commit(MSDelegatingLocalTransaction.java:402(Compiled Code))
    at
    com/ibm/ws/sib/comms/server/clientsupport/StaticCATTransaction.r
    cvCommitTransaction(StaticCATTransaction.java:279(Compiled
    Code))
    at
    com/ibm/ws/sib/comms/server/clientsupport/ServerTransportReceive
    Listener.dataReceived(ServerTransportReceiveListener.java:509(Co
    mpiled Code))
    at
    com/ibm/ws/sib/jfapchannel/impl/rldispatcher/ConversationReceive
    ListenerDataReceivedInvocation.invoke(ConversationReceiveListene
    rDataReceivedInvocation.java:195(Compiled Code))
    at
    com/ibm/ws/sib/jfapchannel/impl/rldispatcher/ReceiveListenerDisp
    atchQueue.run(ReceiveListenerDispatchQueue.java:505(Compiled
    Code))
    at
    com/ibm/ws/util/ThreadPool$Worker.run(ThreadPool.java:1563(Compi
    led Code))
    
    Thread 2 may have the second topmost stack similar to:
    
    com/ibm/ws/objectManager/ManagedObject.backout
    

Local fix

  • No workaround is available.
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED:  Users of the default messaging provider     *
    *                  for IBM WebSphere Application Server        *
    *                  configured with a filestore.                *
    ****************************************************************
    * PROBLEM DESCRIPTION: A deadlock is possible during           *
    *                      concurrent checkpoint and transaction   *
    *                      commit or rollback.                     *
    ****************************************************************
    * RECOMMENDATION:                                              *
    ****************************************************************
    Message processing hangs because the filestore is deadlocked.
    An unlikely timing condition causes the deadlock.  The
    filestore log must be checkpointing and writing a
    ManagedObject to the store file, while at the same time that
    ManagedObject is being committed or backed out.
    

Problem conclusion

  • The code has been changed to prevent the ABBA deadlock by
    dropping one of the locks before acquiring the other in commit
    and backout.
    
    The fix for this APAR is currently targeted for inclusion in
    fix pack 7.0.0.21 and 8.0.0.3.  Please refer to the Recommended
    Updates page for delivery information:
    
    http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980
    

Temporary fix

Comments

APAR Information

  • APAR number

    PM41418

  • Reported component name

    PLAT MSG COM

  • Reported component ID

    620800101

  • Reported release

    300

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt

  • Submitted date

    2011-06-13

  • Closed date

    2011-07-30

  • Last modified date

    2011-07-30

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    PLAT MSG COM

  • Fixed component ID

    620800101

Applicable component levels

  • R300 PSY

       UP



Document information

More support for: WebSphere Application Server
Service Integration Technology

Software version: 7.0

Reference #: PM41418

Modified date: 30 July 2011