PI29527: ABEND S878 WHEN HIGH NUMBER OF USERS FROM A LARGE SAF GROUP LOG IN.

8.5.5.7: WebSphere Application Server V8.5.5 Fix Pack 7
8.5.5.8: WebSphere Application Server V8.5.5 Fix Pack 8
8.5.5.9: WebSphere Application Server V8.5.5 Fix Pack 9
8.5.5.10: WebSphere Application Server V8.5.5 Fix Pack 10
8.5.5.11: WebSphere Application Server V8.5.5 Fix Pack 11
8.5.5.12: WebSphere Application Server V8.5.5 Fix Pack 12
8.5.5.13: WebSphere Application Server V8.5.5 Fix Pack 13
8.5.5.14: WebSphere Application Server V8.5.5 Fix Pack 14
8.5.5.15: WebSphere Application Server V8.5.5 Fix Pack 15
8.5.5.17: WebSphere Application Server V8.5.5 Fix Pack 17
8.5.5.20: WebSphere Application Server V8.5.5.20
8.5.5.18: WebSphere Application Server V8.5.5 Fix Pack 18
8.5.5.19: WebSphere Application Server V8.5.5 Fix Pack 19
8.5.5.16: WebSphere Application Server V8.5.5 Fix Pack 16
8.5.5.21: WebSphere Application Server V8.5.5.21

APAR status

• Closed as program error.

Error description

• When a large number of users access the Server,  the Servant
  terminated with an ABEND S878 after upgrading to z/OS 2.1.
  
  When BPX.DEFAULT.USER support is used, users of UNIX System
  Services can share a UID and GID and don't have to have a
  unique UID and GID. So a group could consist of many users but
  all users shared the same UID (Default User).
  The support for this FACILITY class was removed in z/OS 2.1,
  requiring a unique UID to be assigned to EACH user.
  
  When a user requests access to the address space, the getgrgid
  function call returns the group name and ALL members of this
  group.
  If the group consists of a large number of users, and many of
  them logged on concurrently it filled up the storage causing
  the abend S878.
  

Local fix

• Limit the number of users per group
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:  All users of IBM WebSphere Application      *
  *                  Server V8.5 for z/OS.                       *
  ****************************************************************
  * PROBLEM DESCRIPTION: Customer seeing ABEND878 during high    *
  *                      workload,when many users are            *
  *                      performing login.                       *
  ****************************************************************
  * RECOMMENDATION:                                              *
  ****************************************************************
  Storage below the line was exhausted. In the failing scenario
  the User Registry was localOS. Users belonged to a huge
  group. There were over 38,000 users in this group. During high
  workload, particualarly times when many users were performing
  login, an ABEND S878 occurred.
  The storage is in SP230 Key 0.
  The ABEND S878 thread looked like this:
  Dsa          Entry      Offset     Function
  ---          -----      ------     --------
  4d1befcda0   20327658   00000000   getgrgid
  4d1befcf60   7ad28d48   00000352
  SAFUserRegistryImpl::getGroupsForUser(JNIEnv_*,_jclass*,_jstring
  *,_jobje
  ct*)
  4d1befd120   7903b6f8   86fc4908   SIDECARINVOKEREFLECTMETHOD
  4d1befd8c0   785ad590   000000aa   JVM_InvokeMethod
  4d1befd9c0   7921ae60   000000cc   JVM_InvokeMethod
  4d1befdac0   21115f18   00000034   JVM_InvokeMethod
  4d1befdbc0   7a81bee8   0000002a
  Java_sun_reflect_NativeMethodAccessorImpl_invoke0
  4d1befdcc0   790354b0   86fcab50   RUNJAVATHREAD
  4d1befe460   790c4d00   0000010a   javaProtectedThreadProc
  4d1befe560   78f30430   0000077e   j9sig_protect
  4d1befed60   790c7950   00000072   javaThreadProc
  4d1befee60   224965d8   0000058e   thread_wrapper
  4d1beff360   20003900   00000fb2   CELQPCMM
  

Problem conclusion

• In a User Registry localOS environment security code was
  utilizing z/OS Services to read the entire group, including
  all its members, for each login. At z/OS 1.12 and higher, a
  new Service allows for retrieving group information without
  retrieving group member elements. This decreases the below the
  line storage requirements significantly. We have changed code
  to utilize this newer Service.
  
  APAR PI29527 is currently targeted for inclusion in Fix Pack
  8.5.5.7 of WebSphere Application Server V8.5.
  
  Please refer to the Recommended Updates page for delivery
  information:
  http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980
  
  In addition, please refer to URL:
  http://www.ibm.com/support/docview.wss?rs=404&uid=swg27006970
  for Fix Pack PTF information.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  WEBSPHERE FOR Z

• Fixed component ID

  5655I3500

Applicable component levels

• R850 PSY

     UP