PI95380: Queue manager ceases to participate in cluster after REFRESH CLUSTER, mqrc=2189.

Obtain the fix for this APAR.

APAR status

• Closed as program error.

Error description

• In this scenario there were local logger filesystem availability
  problems.  Because of this, there were 1000s of application
  messages on the SYSTEM.CLUSTER.TRANSMIT.QUEUE, and one or more
  channels held indoubt batches of messages.  At this time the
  administrator ran REFRESH CLUSTER on the local queue manager.
  The indoubt batches of messages became rolled back (probably
  automatically due to log space shortage, though the same effect
  could be seen if using RESOLVE CHANNEL to roll back the batch),
  so the messages appeared on the transmission queue again.
  
  After this the local queue manager's repository manager program
  tried to reallocate messages from the rolled-back indoubt
  batches.  The reallocation routine found that the cluster cache
  no longer knew anything about the queues for which they were
  destined.  The repository manager did not break from its
  reallocation routine to ask the full repositories for details of
  the queues, so it suffered repeated
  MQRC_CLUSTER_RESOLUTION_ERROR errors over a period of many
  minutes, which were visible in the MQ trace for the
  rrmReallocMsgs process.
  
  Application calls to MQOPEN for queues not known locally will
  fail with 2189 MQRC_CLUSTER_RESOLUTION_ERROR.
  Other symptoms include:
  -- a failure to recognize other queue managers in the
     cluster including the QMGR hosting the cluster Q.
  -- message buildup on the SCCQ and the SCTQ.
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED: All users of IBM MQ for z/OS Version 8       *
  *                 Release 0 Modification 0.                    *
  ****************************************************************
  * PROBLEM DESCRIPTION: A queue manager experiencing problems   *
  *                      while there are inflight batches of     *
  *                      messages may cease to participate in a  *
  *                      cluster after a REFRESH CLUSTER         *
  *                      command. Multiple MQPUT1 failures with  *
  *                      reason 88D are seen in chinit trace,    *
  *                      RC 2189, MQRC_CLUSTER_RESOLUTION_ERROR. *
  ****************************************************************
  The reallocation routine within the repository manager was
  suffering a MQRC_CLUSTER_RESOLUTION_ERROR condition repeatedly,
  for the same message each time.  There were 1000s of messages on
  the cluster transmission queue, and because of a flaw in the
  logic flow it would re-read the same message 1000s of times.
  It did not break from this loop to send the query to the full
  repository that was necessary to relieve this situation.
  

Problem conclusion

• Logic has been changed such that the reallocation routine
  schedules a re-run of itself in 60 seconds time, and breaks from
  its work to allow the repository manager to request information
  from the full repositories.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

  IT24269

• APAR is sysrouted TO one or more of the following:

  PH28037 UI70970

Modules/Macros

• CSQXRRMF
  

Fix information

• Fixed component name

  IBM MQ Z/OS V8

• Fixed component ID

  5655W9700

Applicable component levels

• R000 PSY UI70970

     UP20/10/13 P F010

Fix is available

• Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.