IBM Support

PM43823: BBOT0004E: RRS SERVICE ATREND FAILED WITH RETURN CODE=931 ALSO, BBO0006E, REASON=C9C2120C WEBSPHERE Z/OS 6.1.0.33

A fix is available

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • Reported on 6.1.0.33 (AM34450)
    
    Failure would resemble this:
     Trace: 2011/04/18 06:06:01.913 01 t=nnnnnn c=UNK key=P8
    (13007002)
       ThreadId: 00000058
       FunctionName: com.ibm.ws390.tx.NativeTransactionContext
       SourceId: com.ibm.ws390.tx.NativeTransactionContext
       Category: AUDIT
       Extended Message: BBOT0004E: RRS SERVICE ATREND FAILED WITH
    RETURN CODE=931.
     Trace: 2011/04/18 06:06:05.649 01 t=nnnnnn c=UNK key=P8
    (0000000A)
       Description: Log Boss/390 Error
       from filename: ./bbooboai.cpp
       at line: 5731
       error message: BBOO0006E WEBSPHERE FOR Z/OS PROCESS servant
    ENDED ABNORMALLY, REASON=C9C2120C.
    
    Additionally, protocol_http_timeout_output_recovery=SESSION
    
    There should also be HTTP REQUEST TIMEOUTs.
    

Local fix

  • As a workaround, investigate why the HTTP requests are timing
    out and consider changing the current HTTP timeout value.
    Because the timeouts occur, having then
    protocol_http_timeout_output_recovery=SESSION causes this abend.
    
    This setting is documented in the Info Center and does note:
    "Using the session setting might result in a loss of resources
    if the dispatched HTTP request loops or becomes inactive."
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED:  All users of IBM WebSphere Application      *
    *                  Server V7.0                                 *
    ****************************************************************
    * PROBLEM DESCRIPTION: Servant region terminates after         *
    *                      receiving a 931 return code from        *
    *                      RRS' ATREND service call.               *
    ****************************************************************
    * RECOMMENDATION:                                              *
    ****************************************************************
    The servant region terminates after having received a 931
    invalid return code from RRS' ATREND service call.
    The invalid return is triggered by the WebSphere Application
    Server transaction manager's attempt to end the Unit
    of Recovery (UR) a second time. This condition is prompted
    by the following: High HTTP workload,
    protocol_http_timeout_output_recovery setting set
    to SESSION, and an insufficient HTTP timeout value for
    high volume workloads such that numerous HTTP
    timeouts are encountered prior to the termination
    of the servant region.
    The following output can be seen in the controller region's
    sysprint log immediately prior to the time of the
    servant region failure.
    Trace: xxxx/xx/xx xx:xx:xx.xxx xx t=xxxxxx c=UNK key=xx
    Description: Throw CORBA system exception
    exception id: CORBA::INTERNAL
    minor code: c9c21210
    from filename: ./bbooboat.cpp
    at line: 911
    Trace: xxxx/xx/xx xx:xx:xx.xxx xx t=xxxxxx c=UNK key=xx
    Description: Java exception created
    from filename: ./bboocjex.cpp
    Method Name: createExceptionForJava(...)
    org.omg.CORBA.INTERNAL: Exception occured during native
    method call from JAVA  vmcid: 0xc9c21000  minor code: 210
    completed: No
    at com.ibm.ws390.tx.ControllerTransactionManagerImpl.
    contextToGtidRow(Native Method)
    at com.ibm.ws390.tx.ControllerTransactionManagerImpl.
    begin(ControllerTransactionManagerImpl.java:1588)
    at com.ibm.ws390.tx.ControllerTransactionManagerImpl.
    getControl(ControllerTransactionManagerImpl.java:1967)
    at com.ibm.ws390.tx.ControllerTransactionManagerImpl.
    enlistXaResources(ControllerTransactionManagerImpl.java:2827)
    at
    com.ibm.ws390.tx.ControllerTransactionManagerImpl.
    commit(ControllerTransactionManagerImpl.java:2316)
    The following can be seen in the servant region sysprint log
    at the time of the failure:
    Trace: xxxx/xx/xx xx:xx:xx.xxx xx t=xxxxxx c=UNK key=xx
    ThreadId: xxxxxxxx
    FunctionName: com.ibm.ws390.tx.NativeTransactionContext
    SourceId: com.ibm.ws390.tx.NativeTransactionContext
    Category: AUDIT
    ExtendedMessage: BBOT0004E: RRS SERVICE ATREND FAILED WITH
    RETURN CODE=931.
    

Problem conclusion

  • Code was added to prevent a second attempt to end the
    unit of recovery under the stated conditions.
    
    APAR PM43823 is currently targeted for inclusion in Service
    Level (Fix Pack) 7.0.0.21 of WebSphere Application Server V7.0.
    
    Please refer to URL:
    //www.ibm.com/support/docview.wss?rs=404&uid=swg27006970
    for Fix Pack availability.
    

Temporary fix

Comments

APAR Information

  • APAR number

    PM43823

  • Reported component name

    WEBSPHERE FOR Z

  • Reported component ID

    5655I3500

  • Reported release

    700

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt

  • Submitted date

    2011-07-15

  • Closed date

    2011-09-21

  • Last modified date

    2012-02-03

  • APAR is sysrouted FROM one or more of the following:

    PM40551

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    WEBSPHERE FOR Z

  • Fixed component ID

    5655I3500

Applicable component levels

  • R700 PSY UK74996

       UP12/01/18 P 1201

Fix is available

  • Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SS7K4U","label":"WebSphere Application Server for z\/OS"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"7.0","Edition":"","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
10 February 2022