Determining the problem area

Sysplex services cover a wide range of processes. To narrow down which process is the problem area, you need to gather all the external symptoms. These include messages, dumps, and logrec information. Once you have this information, use Table 1 and the associated notes to help determine which problem areas the symptoms point to.

Table 1. Determining the problem area for sysplex services (XCF and XES)
Problem Area─> Symptoms ║ ║ V	Connection Services IXLCONN IXLDISC IXLEERSP	Rebuild Processing	Mainline Services ¹	Coupling Facility Interface	Couple Dataset Services	CFRM	XCF Signaling	SFM	ARM	XCF Client / Server	XCF Note Pad
	XES				XCF
MESSAGES
From a subsystem	Note ²	Note ³	Note ⁵			X			X
From XES(IXL) to the console	X	X	Note ⁶	Note ⁶		X					X
From XCF(IXC) to the console					X	X	Note ¹⁰	Note ¹¹	X	X	X
About a policy						X			X
About GRS ring disruptions							X
About loss of signaling connectivity							X
In SYSLOG	Note ¹⁴				X	X	X			X	X
About couple data set switch					X	X		Note ¹²	Note ¹⁹
About restarts									Note ²⁰
About CFRM policy not active						X
LOGREC DATA SET
Software record	Note ¹⁵				Note ²²			Note ¹³	Note ²¹
ABEND026	Note ¹⁶	Note ¹⁶	Note ⁷
Hardware record			Note ¹⁷	Note ¹⁷
SYSTEM STATUS
ABEND00C					X	X	X	X	X	X	X
ABEND026	X	X	Note ¹⁸	X
WAIT0A2/9C						X
WAIT0A2/10					X
WAIT0A2/130									X
WAIT0A2/140									X
WAIT0A2/68										X	X
WAIT0A2/6A											X
System hang						X
Poor performance for the CF or system				X
Excessive spin				Note ⁸
Subsystem hang	X	Note ⁴	X			X
Subsystem ABEND			X						X
Performance degradation					Note ⁹

Fast path: Explanatory notes for Table 1.

Mainline services include: IXLLIST IXLCACHE IXLFCOMP IXLVECTR IXLLOCK IXLSYNCH IXLRT IXLUSYNC.
Messages received from a subsystem or application describing a failing connection to the coupling facility.
Messages received from a subsystem or application describing the success or failure of the rebuilding of a coupling facility structure.
Subsystem or application is stalled during rebuild of a coupling facility structure.
Messages from subsystems describing failing coupling facility structure operations.
Messages from XES (prefixed with IXL) indicating either coupling facility failures or coupling facility path failures.

This ABEND is recorded in the LOGREC data set for reason codes:

Reason code	Explanation
x‘0C010101’	Indicates that an error occurred in the user's complete exit. The connector is terminated.
x‘0C150101’	Indicates that an error occurred in the user's contention exit. The connector is terminated.
x‘0C3F0101’	Indicates that an error occurred in the user's notify exit. The connector is terminated.
x‘0C680101’	Indicates that an error occurred in the user's notify exit. The connector is terminated.
x‘0E0A0101’	Indicates that an error occurred in the user's list transition exit. The connector is terminated.

Note: XES does not take a dump if a problem occurs in a user exit.

Excessive spin conditions may indicate that hardware interface problems exist in XES or the coupling facility hardware.
System performance degradation to the coupling facility may indicate that excessive storage usage has occurred due to a backlog of requests to the coupling facility
XCF messages indicating path problems on the console or in the SYSLOG. Additionally, messages will describe the action being taken against the XCF signaling path (that is, starting, stopping, or restarting)
Removal of a system from the sysplex did not occur when it was expected. This may be indicated by the operator prompt for IXC102A when automatic sysplex partitioning was expected from sysplex failure management (SFM). IXC messages might indicate that a failure occurred while attempting to partition a system from the sysplex using SFM.
Sysplex failure management (SFM) couple data set switching occurred unexpectedly - note that SFM does not cause the system to enter a wait state when both SFM couple data sets are lost.
A symptom record is placed in the LOGREC data set when:
- The isolation of a system from the sysplex has failed. Sysplex failure management records information indicating the results of the failure isolation.
- A system is fenced from the sysplex. Sysplex failure management records information about the system that was fenced.
Message IXL012I is written to the SYSLOG only. This message contains the return code and reason code for a failed invocation of the IXLCONN sysplex service macro.
A symptom record is placed in the logrec data set when a failed invocation of the IXLCONN sysplex services macro occurs. The symptom record includes the following data from IXLCONN:
- IXLCONN return code
- IXLCONN reason code
- JOBNAME of the issuer of the IXLCONN sysplex services macro
- ASID of the issuer of the IXLCONN sysplex services macro
- IXLCONN parameter list
- IXLCONN answer area mapped by IXLYCONA
In addition, message IXL012I is found in SYSLOG.

This ABEND is recorded in the LOGREC data set for reason codes:

Reason code	Explanation
x‘0E0D0001’	Indicates that an unexpected return code was received from the user's event exit. The connector is terminated.
x‘0E0D0101’	Indicates that an error occurred in the user's event exit. The connector is terminated.

A hardware failure was encountered while the system was communicating with the coupling facility. A symptom record is placed in the logrec data set.
A dump received from ISSUER=IXLR1DIA with an ABEND026 and a reason code of x'0C1Cxxxx' (where xxxx could be anything) indicates that a mainline operation to the coupling facility failed. An entry is recorded in the LOGREC data set.
Automatic Restart Management couple data set switching occurred unexpectedly; note that automatic restart management does not cause the system to enter a wait state when both of the ARM couple data sets are lost.
Automatic Restart Management issues message IXC804I if an element was de-registered because of a failure in its event exit.
A symptom record is placed in the LOGREC data set when a cross-system restart is initiated by automatic restart management.
The system writes a record to the LOGREC data set when removal of a couple data set encounters unusual conditions.