IXC430E system has stalled XCF group members DFHIR000 and CICS hangs
Your CICS region is hung and you receive message IXC430E SYSTEM xxxx HAS STALLED XCF GROUP MEMBERS. You do not receive message IXC432I indicating the stall is no longer occurring until about 30 minutes later. When you respond 'D XCF,GROUP' to IXC430E it returns message IXC331I and includes the CICS XCF group *DFHIR0000.
Following is the complete IXC331I message:
IXC331I DISPLAY XCF indicates stalls on local system groups
$$TM(2) ARC PLEX0(2) ALTGASSC(1) BBGROUP(1) COFVLFNO(6) *DFHIR000(58)
In this case, CICS was not being dispatched because the region has a lower dispatching priority than DB2. The following explains how this problem can occur:
- Two CICS regions are trying to communicate using the cross-system coupling facility (XCF).
- An XCF message is being routed to another system.
- XCF is not receiving a response.
This is often a CICS dispatching priority problem. It is typically diagnosed the same way a CICS hang is diagnosed.
Diagnosing the problem
A dump of the CICS address space, the XCF address space, and the XCF dataspaces might be needed. To get this dump, you can enter the following MVS dump command at the time of the hang:
where xxxxxx is whatever you would like to call the dump
and yyyyyyyy is the CICS jobname taken from message IEC431I
After you get a dump, you can enter the following IPCS commands to diagnose this problem:
- ST SYS to see the time that your system dump is taken. You will need the local time.
- VERBX DFHPDxxx 'CSA' where xxx is the release of CICS you are using, to determine the time the CICS region was last dispatched. This can be found at offset x'50' in the CSA (CSATODP). For example, 0556121F. If the CSA time is less than the time the dump was taken this indicates that CICS is not being dispatched by MVS.
- VERBX DFHPDxxx 'KE=3' shows the address of QR TCB under the column KE_KTCB in Kernel Domain KE_TASK Summary on the same line with the STATUS of "KTCB QR". You can get the address of the associated MVS TCB (KTCB_ATTACH_TCB_ ADDRESS) from offset x'50' in CICS TS V4.2 and below (or offset x'68' in CICS TS V.51) in the KTCB for the QR TCB.
- SUMM FORMAT ASID(X'aaaa') where aaaa is the ASID of the CICS region. This displays the TCB information for the region.
- Enter F 'TCB: tttttttt' where tttttttt is the MVS TCB address of the CICS QR TCB. If the WLIC that follows the PRB under the TCB is 00020001 and the LINK field is 01xxxxxx, this indicates that the TCB is in a wait. In this case, the OPSW pointed into DFHDSDS3+940 but the offset might be different in newer releases of CICS. This is a normal wait under CICS. CICS is just waiting to be redispatched by MVS.
- SYSTRACE ALL displays the system trace. Look at the DSP entries to determine the ASIDs that are being dispatched by MVS. In this case, there were no trace entries for CICS.
- SELECT ALL displays the jobname associated with the ASIDs you see in the system trace.
- SUMM FORMAT JOBNAME(xyz) where xyz are the jobnames associated with the ASIDs found in the system trace displays the TCB information for the job. Locate the ASCB by entering F ASCB then find the the dispatching priority (DPH) of the address space.
ASID JOBNAME DPH (dispatching priority)
0080 WDB035B 41
0086 WDB135D 41
008F WDB135A 41
0095 DBPADBM1 8C
These ASIDs all had a higher dispatching priority than CICS which had a priority of 3B. Therefore, the dispatching priority for CICS was not high enough so CICS was never being dispatched. In this case, the dispatching priority of the DB2 address space (DBPADBM1) was 8C which is much higher than the dispatching priority of CICS. The dispatching priority for CICS should be at least as high as the dispatching priority of DB2.
Resolving the problem
Review your z/OS MVS workload management policy for the performance goals for the work that is being carried out by the CICS region. In this case, the importance of the performance goals for the CICS work should be made the same as the importance of the performance goals for the DB2 work. For guidance on using MVS workload management, see z/OS MVS Planning: Workload Management in the IBM Knowledge Center.
CICS/TS CICS TS CICS Transaction Server