IBM Support

PH05716: MIXED TRANSACTION OUTCOME POSSIBLE IF LOGS STORED IN A DATABASE ARE RECLAIMED DURING TIMING WINDOW USING

Subscribe to this APAR

By subscribing, you receive periodic emails alerting you to the status of the APAR, along with a link to the fix after it becomes available. You can track this item individually or track all items by product.

Notify me when this APAR changes.

Notify me when an APAR for this component changes.

APAR status

  • Closed as program error.

Error description

  • An application server that propagates transactions to a
    backend application server running in a cluster with
    transactional high availability enabled and the transaction
    logs stored in a relational database receives repeated
    failures to commit transactions when invoking the commit
    method on the backend server's subordinate transaction
    coordinator.
    
    The front end application server receives the exception
    org.omg.CORBA.OBJECT_NOT_EXIST with minor code 0.
    
    The callstack from the backend server is of the form:
    
    >> SERVER (id=abababab, host=somehost) TRACE START:
    >>    org.omg.CORBA.OBJECT_NOT_EXIST:   vmcid: 0x0  minor
    code: 0  completed: No
    >>  at
    com.ibm.ws.Transaction.JTS.JTSServantManager.keyToObject(JTSServ
    antManager.java:283)
    >>  at com.ibm.ejs.oa.EJSOAImpl.keyToObject(EJSOAImpl.java:553)
    >>  at
    com.ibm.ejs.oa.EJSRootOAImpl.keyToObject(EJSRootOAImpl.java:277)
    >>  at
    com.ibm.rmi.corba.ObjectManager.lookupServant(ObjectManager.java
    :104)
    >>  at
    com.ibm.CORBA.iiop.ServerDelegate.getServantForRequest(ServerDel
    egate.java:394)
    >>  at
    com.ibm.CORBA.iiop.ServerDelegate.dispatch(ServerDelegate.java:4
    61)
    >>  at com.ibm.rmi.iiop.ORB.process(ORB.java:616)
    ...
    
    The front end application server will report a
    HeuristicMixedException to the application.
    
    Although the transaction committed, the subordinate branch in
    the backend server rolled back and a mixed transactional
    outcome occurred.
    
    Investigation shows that:
    1 - The backend server had previously started to recover a
    peer server's transaction logs (stored in a database) and
    generated a CWRLS0013I message indicating this.
    2 - The peer had later reclaimed its transaction logs
    3 - The backend server had not generated a CWRLS0014I message
    to indicate that it had finished processing the peer's
    transaction logs.
    

Local fix

Problem summary

  • ****************************************************************
    * USERS AFFECTED:  WebSphere Application Server users of       *
    *                  transactional high availability             *
    ****************************************************************
    * PROBLEM DESCRIPTION: Transactions repeatedly fail to commit  *
    *                      with OBJECT_NOT_EXIST minor code 0.     *
    *                      Transaction outcome is mixed.           *
    ****************************************************************
    * RECOMMENDATION:                                              *
    ****************************************************************
    A timing window existed when using transactional high
    availability with the transaction service logs stored in a
    relational database whereby an application server performing
    peer recovery would not correctly clear up state when handing
    back the logs.
    This resulted in WLM misrouting requests for the second phase
    of the transaction 2PC protocol (ie commit) to the
    application server that were intended for the peer application
    server, the prepare phase having been routed correctly.
    More importantly, it also resulted in the incorrect processing
    of those requests resulting in a
    org.omg.CORBA.OBJECT_NOT_EXIST Exception with minor code 0
    being thrown.  This Exception and minor code indicates that the
    target transaction does not exist (as opposed to indicating
    that the transaction was not located in the target runtime).
    This results in a javax.transaction.HeuristicMixedException in
    the front end application server and the transaction is
    forgotten.  Later when the subordinate transaction, which is
    in-doubt, attempts to determine the transaction outcome via
    replay_completion it is told that the transaction does not
    exist and therefore, in accordance with the standard 2PC
    presumed abort optimization, the subordinate transaction
    branch is rolled back.
    

Problem conclusion

  • The synchronization of the transaction service code was
    modified so that the timing window did not occur and any state
    associated with peer recovery, including WLM routing for the
    transaction service endpoints, was correctly cleaned up.  The
    code that processed inbound protocol requests was also
    modified to perform additional checks which would result in a
    org.omg.CORBA.TRANSIENT being thrown in certain circumstances.
    
    
    The fix for this APAR is currently targeted for inclusion in
    fix packs 8.5.5.16 and 9.0.5.0.  Please refer to the
    Recommended Updates page for delivery information:
    http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980
    

Temporary fix

Comments

APAR Information

  • APAR number

    PH05716

  • Reported component name

    WEBS APP SERV N

  • Reported component ID

    5724H8800

  • Reported release

    850

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2018-11-27

  • Closed date

    2019-06-25

  • Last modified date

    2019-06-25

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    WEBS APP SERV N

  • Fixed component ID

    5724H8800

Applicable component levels



Document information

More support for: WebSphere Application Server
General

Software version: 850

Reference #: PH05716

Modified date: 25 June 2019