IBM Support

PI17462: WMQ Z/OS: SOME MEMBERS OF A QSG MAY NOT REGISTER INTEREST IN TRIGGER-EVERY EVENTS. THOSE WITH INTEREST USE MORE DXWB STORAGE

A fix is available

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • A problem is reported which may have symptoms including:
    
    - CSQY220I reports a higher-than-usual amount of local storage
      being used.
    
    - ABN=878-00000010,C=R3600.710.DB2M,M=CSQ5MFRR
    
    - ABN=5C6-00E20016,U=xxxxxxx ,C=R3600.710.MMC -CSQMTRG2,
      M=CSQGFRCV,LOC=CSQSLD1 .CSQSVSTK+000005DA
    
    - ABN=5C6-00E7014F,LOC=CSQXADPM.CSQXADPM+05268
    
    The ABEND878 and the ABEND5C6-00E20016 are due to a lack of
    private storage in the MSTR address space. The extra storage
    usage is due to a large number of queued requests from the XCF
    exits. These are held in DXWB blocks in SP229 KEY7 storage
    chained off the DXGB control block.
    
    The XCFWRK01 task is processing two notification types from the
    backlog of DXWB blocks.
    
    Type 1 -
    An application which has a shared, triggered application queue
    open for input closes it. If this is the last application with
    that queue open, a CSQE_KAT_Tap_Bridge notification is sent to
    all members of the Queue Sharing Group (QSG).
    See PI17684 for an additional fix relating to this type.
    
    Type 2 -
    When an application puts a message to a shared TRIGTYPE(EVERY)
    application queue, the putting queue manager determines which
    queue managers have an interest in the trigger-every event and
    sends a notification to one of the interested ones. On the
    notified queue manager, the resulting action is to generate a
    trigger message to the initiation queue.
    
    On the failing queue managers, around 80% are Type 1 and 20%
    are Type 2.
    
    For other queue managers in the QSG, all the notifications
    are of Type 1. No Type 2 requests are received. These queue
    managers did not indicate interest in the events due to an
    earlier timing problem:
    
    When two trigger monitor tasks open a shared initiation queue
    at the same time, the queue managers may incorrectly determine
    that neither is the first to open the queue. This results in
    the queue manager not registering an interest in messages being
    put to trigger-every application queues. Other queue managers
    in the QSG that have interest will therefore need to handle a
    higher volume. Depending on the workload volume, those queue
    managers may start to run out of storage.
    
    
    Additional symptom for this problem may include:
    - If none of the other queue managers in the QSG has a trigger
    monitor task with the initiation queue open, this can result in
    no trigger messages being generated when messages are put to
    the TRIGTYPE(EVERY) application queue.
    
    - RESET QSTATS(<shared initq>) shows MSGSIN(0) and HIQDEPTH(0)
    on the queue managers that failed to register interest.
    
    - The ABEND878 dump is partial with
    SDRSN = 80080000 00200000 00000000 00000000
    and does not contain enough of the MSTR address space storage
    to determine why all the storage was being used.  A console
    dump is needed when the CSQY220I message indicates a
    significant increase in the storage being used.
    
    
    Additional keywords:
    ABEND878 ABENDS878 878 S878 S0878 10 00000010 RC10 RC00000010
    ABEND5C6 ABENDS5C6 5C6 S5C6 S05C6 00E20016 00E7014F
    CSQ5MGX0 CSQ5MFRR CSQMTRG2 CSQEKATM
    SUBPOOL 229 KEY 7 K7
    TRIGTYPE TRIGGERTYPE EVERY
    RESET QSTATS MSGIN HIQDEPTH
    performance cpu
    KAT Kiss and Tell
    

Local fix

  • Correct the issue for the QSG members that are not receiving
    the trigger-every events:
    1) Stop all trigger monitor tasks for the shared initq on those
    queue managers
    2) Restart one trigger monitor task and allow time for it to
    open the queue.
    3) Restart the other trigger monitoring tasks one at a time.
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED: All users of WebSphere MQ for z/OS Version 7 *
    *                 Release 1 Modification 0.                    *
    ****************************************************************
    * PROBLEM DESCRIPTION: Various problems occur when multiple    *
    *                      applications open a shared initiation   *
    *                      queue simultaneously on the same queue  *
    *                      manager.                                *
    *                                                              *
    *                      Symptoms of this problem may include:   *
    *                       - If none of the other queue managers  *
    *                         in the QSG has a trigger monitor     *
    *                         task with the initiation queue open  *
    *                         no trigger messages are generated    *
    *                         for shared application queues with   *
    *                         TRIGTYPE(EVERY).                     *
    *                                                              *
    *                       - Increased workload on other queue    *
    *                         managers with the initiation queue   *
    *                         open. This can lead to a build up    *
    *                         of DXWB control blocks and lead to   *
    *                         those queue managers encountering    *
    *                         a short of storage condition and     *
    *                         abending S878.                       *
    ****************************************************************
    * RECOMMENDATION:                                              *
    ****************************************************************
    When two trigger monitor tasks open a shared initiation queue at
    the same time, the queue manager may incorrectly determine that
    neither is the first to open the queue. This results in the
    queue manager not registering an interest in messages being put
    to shared trigger-every application queues.
    If no other queue managers have interest in the application
    queues, no trigger messages will be generated due to messages
    arriving on those queues.
    If other queue managers have successfully registered interest in
    the application queues, the local queue manager will not
    participate in generating trigger messages for those queues,
    resulting in increased workload on any queue managers that do
    have an interest. This additional workload can lead to a build
    up of DXWB control blocks and contribute to storage shortages
    on the queue managers with interest.
    

Problem conclusion

  • Additional serialisation is added to CSQMOPNI to ensure that
    interest in shared application queues is correctly registered
    when multiple applications are opening a shared initiation queue
    simultaneously.
    100Y
    CSQMOPNI
    CSQMOSQ1
    

Temporary fix

  • *********
    * HIPER *
    *********
    

Comments

APAR Information

  • APAR number

    PI17462

  • Reported component name

    WMQ Z/OS V7

  • Reported component ID

    5655R3600

  • Reported release

    010

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    YesHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2014-05-07

  • Closed date

    2014-05-27

  • Last modified date

    2014-07-01

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

    UI18349

Modules/Macros

  • CSQMOPNI CSQMOSQ1
    

Fix information

  • Fixed component name

    WMQ Z/OS V7

  • Fixed component ID

    5655R3600

Applicable component levels

  • R100 PSY UI18349

       UP14/06/27 P F406 ¢

Fix is available

  • Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.

[{"Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"SG19M","label":"APARs - z\/OS environment"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"7.0.1","Edition":"","Line of Business":{"code":"","label":""}}]

Document Information

Modified date:
01 July 2014