PM17172: S0C4 IN METHODS SUCH AS ORB_REQUEST::GETACRWORKELEMENTPTR() DUE TO DOUBLE FREE OF BBOOORBR

Obtain the fix for this APAR.

APAR status

• Closed as program error.

Error description

• High volume load tests are performed, with over 500 RMI/IIOP
  requests per second being generated in serverX to another
  server serverY.
  
  If serverY is stopped or abended then the serverX controller
  region will ABEND with various symptoms, mainly S0C4, due to
  what appears to be a double free of a BBOOORBR control block.
  
  Some example call stacks that were encountered are:
  TCB Address: 007A9E88
  27 ORB_Request::getACRWorkElementPtr()    <--- exception here
        +0000000C BBGBOA
  28 Java_com_ibm_ws390_ziop_ZIOPCPPUtilities_process_1inbound_1r
  
  Traceback:
  
    DSA      Entry       E  Offset  Statement   Load Mod
  (jvm "take dump" methods removed)
    20       __zerro     +00000EE4              CELQLIB
    21       __zerros    +0000032E              CELQLIB
    22       CEEHDSP     +00003F64              CELQLIB
  CEEHDSP
    23       CEEOSIGJ    +0000096E              CELQLIB
  CEEOSIGJ
    24       CELQHROD    +00000248              CELQLIB
  CELQHROD
    25       CEEOSIGG    +00000000              CELQLIB
  CEEOSIGG
    26       CELQHROD    +00000248              CELQLIB
  CELQHROD
    27       ORB_Request::getACRWorkElementPtr()     <-- exception
                         +0000000C              BBGBOA
    28
  Java_com_ibm_ws390_ziop_ZIOPCPPUtilities_process_1inbound_1r
                         +00000F9C              BBGBOA
    29       RUNCALLINMETHOD
  
  Traceback:
  
  
    DSA      Entry       E  Offset  Statement   Load Mod
  ( ... "take dump" methods removed ... )
    12       __zerro     +00000EE4              CELQLIB
    13       __zerros    +0000032E              CELQLIB
    14       CEEHDSP     +00003F64              CELQLIB
  CEEHDSP
    15       CEEOSIGJ    +0000096E              CELQLIB
  CEEOSIGJ
    16       CELQHROD    +00000248              CELQLIB
  CELQHROD
    17       CEEOSIGG    +00000000              CELQLIB
  CEEOSIGG
    18       CELQHROD    +00000248              CELQLIB
  CELQHROD
    19
  Java_com_ibm_ws390_ziop_ZIOPCPPUtilities_process_1inbound_1r
                         +00000FCC              BBGBOA <-exception
  
  Traceback:
    DSA      Entry       E  Offset  Statement   Load Mod
  
   1        ORB_Request::~ORB_Request()     <-exception
                        +00000122              BBGBOA
   2        ORB_Request::~ORB_Request()
                        +00000D88              BBGBOA
   3        BBO_BOA::return_ORB_request(ORB_Request*)
                        +00000074              BBGBOA
   4
  Java_com_ibm_ws390_ziop_ZIOPCPPUtilities_process_1inbound_1r
                        +0000131E              BBGBOA
   5        RUNCALLINMETHOD
                        +00000000              *PATHNAM
   6        gpProtectedRunCallInMethod
                        +0000003E              *PATHNAM
   7        signalProtectAndRunGlue
                        +0000001C              *PATHNAM
   8        j9sig_protect
                        +00000356              *PATHNAM
   9        gpCheckCallin
                         +000000A4              *PATHNAM
  10       callStaticIntMethodA
                         +00000072              *PATHNAM
  11
   ZAioTCPChannelCppUtilities::readComplete(int,long,int,xrcd*)
                         +00000602              BBGCF
  12
   CF_TCP_Request::readComplete(int,CF_Connection*,int,xrcd*)
                         +0000003A              BBGCF
  13
   CF_TCP_Connection_Object_Manager::processAsyncReadComplete(a
                         +000010B6              BBGCF
  14
   ACR_ExecutionThread::processTcpAsyncReadComplete(acrwObj*)
                        +00000024              BBGBOA
   15
  ACR_ExecutionThread::RemoveAndProcessWork(ThreadCleanUp*)
                        +00002036              BBGBOA
  
  LOCAL FIX:
  none
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED:  All users of IBM WebSphere Application      *
  *                  Server V7.0                                 *
  ****************************************************************
  * PROBLEM DESCRIPTION: ABEND0C4/ABENDS0C4 in WebSphere         *
  *                      Application Server for z/OS Controller  *
  *                      during handling of outbound IIOP        *
  *                      requests under load.                    *
  ****************************************************************
  * RECOMMENDATION:                                              *
  ****************************************************************
  Storage corruption may occur in a WebSphere Application Server
  for z/OS Controller while handling a large number of outbound
  IIOP requests.  The server in question is acting as a client by
  sending an IIOP request to some other server.  If the backend
  server handling the IIOP request goes down for some reason, the
  frontend client server becomes exposed to a timing window that
  could result in storage corruption.
  Symptoms of storage corruption can vary, but the most common
  pattern is that the Controller will experience an ABEND0C4
  and a stack trace of the failing thread will show that the
  abend occurred in a function of an ORB_Request object, for
  example:
  
  ORB_Request::getACRWorkElementPtr()
  

Problem conclusion

• In some cases, a timing window allowed for a double delete of
  an ORB_Request object, causing corruption in the heap pool that
  contains those objects.  This timing window was fixed so that
  the double delete can no longer occur.
  
  APAR PM17172 is currently targeted for inclusion in Service
  Level (Fix Pack) 7.0.0.15 of WebSphere Application Server V7.0.
  
  Please refer to URL:
  //www.ibm.com/support/docview.wss?rs=404&uid=swg27006970
  for Fix Pack availability.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

  PM17170

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  WEBSPHERE FOR Z

• Fixed component ID

  5655I3500

Applicable component levels

• R700 PSY UK65060

     UP11/03/04 P F103

Fix is available

• Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.