OA46222: NEW FUNCTION - UPDATES TO USS_KERNEL_PVTSTG_THRESHOLD HEALTHCHECKER FOR MORE GRANULARITY DETECTING STORAGE USAGE

Obtain the fix for this APAR.

APAR status

• Closed as new function.

Error description

• HEALTHCHECKER USS_KERNEL_PVTSTG_THRESHOLD Triggers
  Customer is a heavy HFS filesystem type user. The heavy
  HFS usage causes higher than expected storage usage in
  the OMVS address space. The Healthchecker is issuing the
  MSGBPXH072E indicating that the USS_KERNEL_PVTSTG_THRESHOLD
  check triggered and yet stack cell storage availability is not
  an issue.
  
  PROBLEM DETAILS
  
  The Healthchecker (HC) check USS_KERNEL_PVTSTG_THRESHOLD divides
  the OMVS storage into areas.  80% of the available storage is
  designated for stack cell pool cells and the remaining 20% for
  kernel and system usage.
  
  To protect stack cells storage, the HC monitors the kernel and
  system usage storage area for growth over the 85% threshold by
  default.  When storage usage exceeds the defined or altered
  PVTSTG, the check triggers and message BPXH072E is issued.
  
  The high storage usage in the OMVS address space is caused by
  heavy HFS usage causing HFS cell pools to get allocated and
  stay around.  In the reported situation, stack cell usage is
  no where close to the limit and the customer often runs around
  103% without stack cell storage issues.
  
  VERIFICATION STEPS
  
  - Message BPXH072E is being issued
  - Stack Cell usage High Water Mark on the D OMVS,STORAGE
    is far from the MAXIMIM CELLS available
  STACK CELLS:
    CURRENT USAGE     MAXIMUM CELLS        HIGH WATER
         5330             19931               5422
  

Local fix

• Lower HFS activity and raise PVTSTG threshold from the
  USS_KERNEL_PVTSTG_THRESHOLD check
  

Problem summary

• ****************************************************************
  * USERS AFFECTED: All users of z/OS UNIX System Services for   *
  *                 HBB7790.                                     *
  ****************************************************************
  * PROBLEM DESCRIPTION: Provide new parameters for the          *
  *                      USS_KERNEL_PVTSTG_THRESHOLD Health      *
  *                      Check that will trigger the check to    *
  *                      monitor all kernel private storage      *
  *                      instead of just non-stack private       *
  *                      storage.                                *
  ****************************************************************
  * RECOMMENDATION:                                              *
  ****************************************************************
  Customers who have a UNIX workload with a small number of
  threads but high non-stack kernel storage usage (for example
  high HFS usage) receive misleading USS_KERNEL_PVTSTG_THRESHOLD
  check exceptions warning of a private storage shortage. The
  check currently monitors just non-stack private storage and
  issues an exception when the threshold of non-stack storage
  usage is exceeded.
  This APAR adds support for four new parameters to the
  USS_KERNEL_PVTSTG_THRESHOLD check, PVTSTG_NONE, PVTSTG_LOW,
  PVTSTG_MED and PVTSTG_HIGH. When any of the new parameters
  are specified the check will monitor all kernel private storage
  as a single resource. This will prevent customers from
  receiving misleading exceptions that indicate there is a
  kernel storage shortage.
  

Problem conclusion

Temporary fix

Comments

• Documentation Changes:
  
  IBM Health Checker for z/OS User's Guide
  SC23-6843-xx
  
  o Check Descriptions
    o IBM Health Checker for z/OS checks
      o z/OS UNIX System Services checks (IBMUSS)
        o USS_KERNEL_PVTSTG_THRESHOLD
  
  USS_KERNEL_PVTSTG_THRESHOLD
  
  Description:
      This check monitors the current usage of private below the
      bar storage of the UNIX System Services kernel against a
      suggested threshold.
  
  Reason for check:
      Exhausting the kernel private below-the-bar storage will
      cause unpredictable errors because resources can no longer
      be obtained, which could potentially bring down z/OS
      UNIX.
  
  z/OS releases the check applies to:
      z/OS V2R1 and later.
  
  User override of IBM values:
      Use the keywords in the following statement to override
      check values on either a POLICY statement in the HZSPRMxx
      parmlib member or on a MODIFY command.  This statement can
      be copied and modified to override the check defaults:
      UPDATE,
         CHECK(IBMUSS,USS_KERNEL_PVTSTG_THRESHOLD),
         INTERVAL(00:02),
         SEVERITY(HIGH),
         PARM('PVTSTG_HIGH(90%),PVTSTG_MED(85%),PVTSTG_LOW(80%)),
         DATE(20131005)
  
  Debug support:
      No
  
  Verbose support:
      No
  
  Parameters accepted:
      Yes, as follows:
  
      o When not using dynamic severity:
       - PVTSTG(pvtstg%) where an integer, 0-100, followed by a
         %, indicating the threshold percent for usage of kernel
         private storage below the bar that was available after
         kernel initialization.  An exception message is displayed
         when system values are equal to or greater than pvtstg%.
         The severity of the exception message depends on the
         system's % usage of kernel private storage that is below
         the bar.
         - A percentage of 80-84 is low severity.
         - A percentage of 85-89 is medium severity.
         - A percentage above 89 is high severity.
      o When using user-specified dynamic severity:
       - PVTSTG_HIGH(pvtstg%)
       - PVTSTG_MED(pvtstg%)
       - PVTSTG_LOW(pvtstg%)
       - PVTSTG_NONE(pvtstg%)
  
      pvtsg%
        An integer, 0-100 followed by a %, indicating the
        threshold percent for usage of kernel private storage
        below the bar that was available after kernel
        initialization.  When using dynamic severity, you can
        specify one or more thresholds.  If a threshold value for
        a given severity is not specified, no exception message
        is issued for that level.  If PVTSTG is specified along
        with any of the dynamic severity parameters, the
        PVTSTG value is ignored and the exception severity
        coincides with the specified parameters.
  
        Default: PVTSTG(85%)
  
      You can use synonyms for these parameters as follows:
  
      Parameter            Synonyms
      PVTSTG_HIGH          PVTSTG_H or PVTSTG_HI
      PVTSTG_MED           PVTSTG_M
      PVTSTG_LOW           PVTSTG_L
      PVTSTG_NONE          PVTSTG_NO or PVTSTG_NO
  
  Reference:
      None.
  
  Messages:
      This check issues the following exception message:
        o BPXH072E
      See the BPXH messages in z/OS MVS System Messages,
      Vol 3 (ASB-BPX)
  
  SECLABEL recommended for multilevel security users:
      SYSLOW - see z/OS Planning for Multilevel Security and the
      Common Criteria for information about using security labels.
  
  
  ================================================================
  z/OS MVS System Messages Volume 3 (ASB - BPX)
  SA38-0670-xx
  
  o BPXH messages
  
  BPXH072E The z/OS UNIX System Services kernel usage of
  monitored below the bar private storage has exceeded the check
  threshold of thresholdpvtstgpct%.  Currently allocpvtstg of
  maxpvtstg bytes (allocpvtstgpct%) of monitored private storage
  has been allocated.
  
  Explanation:  The USS_KERNEL_PVTSTG_THRESHOLD check determined
  that the percent of total allocated kernel private storage has
  exceeded the current check threshold.
  
  If only the PVTSTG parameter is specified for the check, then
  monitored private storage is considered to be all private
  storage below the bar that has not been allocated for stack
  cell pool cells.  During initialization, the kernel designates
  80% of the region as the maximum storage that can be allocated
  for stack cell pool cells and the remaining 20% as the maximum
  private storage.  If the allocated non-stack cell private
  storage exceeds 20% and overflows into the 80% designated for
  stack cells it will reduce the maximum number of stack cells
  that can be allocated.  The percent of monitored storage
  allocated can exceed 100% if it exceeds the 20% designated
  during kernel initialization and overflows into the 80%
  designated for stack cells.
  
  If any of the dynamic severity parameters are specified
  (PVTSTG_NONE,PVTSTG_LOW, PVTSTG_MED or PVTSTG_HIGH) for the
  check, then monitored private storage is considered to be all
  below the bar private storage.
  
  In the message text:
  
  allocpvtstg
  The number of bytes of monitored private storage below the bar
  that has been allocated in the z/OS UNIX System Services kernel
  address space.
  
  allocpvtstgpct
  If only the PVTSTG parameter has been specified for the check,
  this is the percentage of monitored (non-stack) kernel storage
  that has been allocated.  If the percentage exceeds 100%, it is
  not considered an error but will reduce the maximum number of
  stack cells that can be allocated and reduce the kernel thread
  capacity.  If any of the dynamic severity parameters have been
  specified, this is the percentage of kernel private storage
  that has been allocated (allocpvtstg/maxpvtstg).
  
  maxpvtstg
  If only the PVTSTG parameter has been specified for the check,
  this is the maximum number of bytes of below the bar private
  storage that can be allocated without overflowing into the
  storage designated for stack cells at kernel initialization.
  Overflowing into the stack cell designated area will reduce the
  maximum number of stack cells that can be allocated and will
  reduce the kernel thread capacity.  This value is approximately
  20% of the kernel region size.
  
  If any of the dynamic severity parameters have been specified,
  this is the total number of bytes of below the bar private
  storage that can be allocated.
  
  thresholdpvtstgpct
  The current setting of the monitored private storage usage
  threshold.  Th USS_KERNEL_PVTSTG_THRESHOLD check will only
  issue this message when the percent of allocated storage
  exceeded this value.  The IBM default threshold is 85%.
  
  The IBM default threshold percentage value can be overridden.
  For more information, see the USS_KERNEL_PVTSTG_THRESHOLD check
  in IBM Health Checker for z/OS User's Guide
  
  System action: The system continues processing.
  
  Operator response: Report the problem to the system programmer.
  
  System programmer response:  Continue monitoring private
  storage usage in the kernel address space using the DISPLAY
  OMVS,STORAGE command.  If the amount of allocated kernel
  private storage continues to increase consider reducing non-
  critical UNIX workloads to prevent disruption of critical UNIX
  workloads.  If reducing the systems UNIX workload does not
  decrease the amount of allocated kernel private storage contact
  IBM Support.
  
  Use the DISPLAY OMVS,STORAGE command to determine the systems
  current UNIX workload.  Quiescing processes with large number
  of threads will provide the most relief for kernel private
  storage shortages.
  
  Problem determination: Not applicable
  
  Module: BPXHCFL4
  
  Source: z/OS UNIX System Services
  
  Reference Documentation:
  o For information about using the DISPLAY OMVS command, see
    z/OS MVS System Commands.
  
  Automation: Not applicable
  
  Routing Code: Not applicable
  
  Descriptor Code: Not applicable
  

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

  UA77731

Modules/Macros

• BPXHCFL4 BPXSMTSK BPXZOCVT
  

Fix information

• Fixed component name

  OPENMVS SYS SRV

• Fixed component ID

  5695SCPX1

Applicable component levels

• R790 PSY UA77731

     UP15/06/26 P F506

Fix is available

• Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.