Mustgather: Read first for problems encountered with Debug Tool for z/OS

	Gathering general information
	Gathering problem specific information
	Submitting information to IBM Support

General hints and tips

Gathering general information

General information is information that is needed for any reported problem, and this information includes the following:

• A complete description of the problem, including the following:
  • When did the problem first occur?
  • Is the problem a one time failure or reoccurring?
  • Was software maintenance applied?
  • A test scenario including what the application was doing at the time of the failure and the debugging commands used.
• Debug Tool for z/OS version and maintenance level. Use CALL %VER to obtain your Debug Tool version. For additional information, refer to How to determine service level.
• Compiler or Assembler version.
• Subsystem (CICS, DB2, IMS, WAS) version, release, and modification. If you do not know the subsystems used in your environment, contact your system administrator.
• Operating system OS/390 or z/OS version, release, and maintenance level if known. This information can be obtained through SMP/E.
• Language Environment version and maintenance level if known. The version is often the same as the operating system version that you are running.
• A valid contact phone number and email address

Click on the problem type or component that best describes your Debug Tool problem. This will provide you with a listing of the documentation that the support team requires to diagnose your problem.

Gather the following documentation for abends before calling IBM support:

• SYSMDUMP. For information on how to request a dump under Language Environment, refer to http://www.ibm.com/servers/eserver/zseries/zos/le/assist/support/10573r7.html.
• If your environment includes a CICS region, see CICS problem.

Gather the following documentation for unexpected results before calling IBM support:

• Debug Tool log containing the following:
  1. The command you used,
  2. The output from the command, and
  3. Any error messages you received.

Gather the following documentation for installation problems before calling IBM support:

• SMP/E output from APPLY problems
• JES output from failed job runs including failing IVP runs

Gather the following documentation for CICS problems before calling IBM support:

• CICS version, release, and maintenance level. Contact your CICS administrator for this information.
• If the problem can be recreated:
  1. Start up JCL for CICS region
  2. JES output for CICS region
  3. Profile creation in DTCN or CADP
  4. Screen shots of entries in DTCN/CADP
  5. Load modules used in recreating the problem
  6. Steps leading up to the failure
• If the problem cannot be recreated gather the following:
  • A SYSMDUMP that includes CICS trace information. In order to create a SYSMDUMP that includes CICS trace information, you must specify the following before creating a SYSMDUMP:
    1. Use CETR to set all domains to ‘1’, except for ‘AP’ which is set to ‘1-2’.
    2. If you are using Remote Debug via TCPIP, set 'SO' to '1-2'
    3. If you encountered a storage problem, for example, a short-on-storage (SOS) error, set the trace component SM to '1-2'
    4. Trace Table Size must be set at a minimum of 10 megabytes.
    5. Use CEMT INQ SYS to verify the dumping parameter is set to SYSDUMP. If it is not, set to SYSDUMP by over typing the current value.
    6. Set up a TRD for the CICS abend code you are seeing. For example, CEMT SET TRD(ASRA) ADD SYSDUMP
  • JES output for CICS region

Gather the following documentation for remote debugger problems before calling IBM support:

• Version and release of the remote debugger you are using. The version is either Distributed Debugger (JSUI) or Compiled Language Debug (WSED, WD4z, WDDz). If you are using JSUI, you can find release information in the DOS window in which JSUI is running. For example, IBM Distributed Debugger (User Interface)
Version 9.2 (2005/06/09) - Licensed Material - Property of IBM. Provide a screen shot of this information. If you are using a remote debugger other than JSUI, you can use the About option in the Help pulldown menu to obtain release information.
• TEST RUNTIME string that was used to invoke the debugger.
• The deamon to which the workstation is listening.
• If possible, provide a testcase with instructions on how to recreate the problem.
• If a testcase cannot be provided, provide an Execution and Program Data Control (EPDC) trace. EPDC defines the actions taking place between the host and the remote debug (workstation) client. The EPDC trace can show what data and actions took place before the error occured. The EPDC trace is created differently depending on the version of the remote debug client that you are using:
  • Distributed Debugger: VisualAge, C/C++ Productivity Tools
  • Compiled Language Debugger:WD4Z (formerly WSED),WDDZ
  For information on how to create an EPDC trace, click here.
• WTO messages issued by Debug Tool for TCPIP errors. These messages provide useful information, such as error type and TCPIP socket function code. You can collect these messages from the job output or console log.
• If your connection was lost, provide a SYSMDUMP. For information on how to request a dump, refer to http://www.ibm.com/servers/eserver/zseries/zos/le/assist/support/10573r7.html.

Gather the following documentation for DTSU problems before calling IBM support:

• EQASTART trace:
  1. In the EQASTART file there is this statement: xrc = trace "O"). Replace the "O" with "?r" as follows: xrc trace "?r") and save this update.
  2. Restart EQASTART. When the trace starts, you will see the current executable statement displayed in the foreground.
  3. Press Enter to advance to the next instruction.
• Perform a screen capture of the following DTSU dialog panels. Enter the panelid command in the ISPF command prompt to display the ISPF panel ids.
  • EQAPFORS - Main DTSU panel
  • EQAPFPRM - Display this panel by entering forward-slash in the "Enter / to modify parameters" field.
  • ISREDDE2 - Generated JCL created by pressing F10-Submit in the EQAPFORS panel
• Setup files
  • hlq.SEQATLIB: EQAZDFLT. EQAZDSYS, EQAZDUSR
  • hlq.EQASTART: EQASTART
  • DTSU
• Use TSO ISRDDN output to show data set concatenations. For information about ISRDDN, refer to the ISPF User's Guide at: http://publibfp.boulder.ibm.com/cgi-bin/bookmgr/BOOKS/ISPZUG50/CCONTENTS?SHELF=ispzpm50&DN=SC34-4822-05&DT=20060621022939

Gather the following documentation for VTAM and TIM problems before calling IBM support:

• JCL output
• VTAMLST member containing the definitions for the EQAnnn minor node
• Terminal LU definitions (EQAWTRML)
• TCPIP configuration file information for VTAM port definitions
• EQAYSESSM job output (for TIM only)

Gather the following documentation for performance problems before calling IBM support:

• Testcase with instructions on how to recreate the problem. Include all necessary files, such as the load module, DLLs run with the load module, etc. (LMODS, SIDEFILES/LISTINGS, DLLs, SOURCE). For CICS testcases, include any maps needed to run the application.
• Output from any performance monitoring tools such as Application Performance Analyzer for z/OS.
• SYSMDUMP. For information on how to request a dump, refer to http://www.ibm.com/servers/eserver/zseries/zos/le/assist/support/10573r7.html.
• If running under CICS, do ONE of the following:
  • Include a SYSMDUMP that includes CICS trace information. See CICS problem.

• Create a CICS auxiliary trace. Specify the following CICS Startup parameters:
  1. Use CETR to set all domains to ‘1’, except for ‘AP’ which is set to ‘1-2’. Trace Table Size must be set at a minimum of 10 megabytes and Auxiliary Trace Status set to started. Please ftp the unformatted trace that is created.
  For additional information about creating a CICS auxiliary trace, refer to the following Debug Tool technote: http://www.ibm.com/support/docview.wss?&uid=swg21212231 or the CICS Problem Determination Guide at http://publibfp.boulder.ibm.com/cgi-bin/bookmgr/BOOKS/DFHS1B00.

Gather the following documentation for storage problems before calling IBM support:

• Testcase with instructions on how to recreate the problem. Include all necessary files, such as the load module, listing, source, EQALANGX, DLLs run with the load module, etc. (LMODS, SIDEFILES/LISTINGS, DLLs, SOURCE). For CICS testcases, include any maps needed to run the application.
• When running under LE, gather an LE storage report if possible. Setting LE runtime option RPTS(ON) creates a storage report when the program runs to completion. If the program only runs to completion without Debug Tool, create the report without Debug Tool.
• Create a SYSMDUMP of the out of storage error. For information on how to request a dump under LE, refer to http://www.ibm.com/servers/eserver/zseries/zos/le/assist/support/10573r7.html.
• If running under CICS, do ONE of the following:
  • Include a SYSMDUMP that includes CICS trace information. See CICS problem.
  OR
  • Create a CICS auxiliary trace. Specify the following CICS Startup parameters:
    Use CETR to set all domains to ‘1’, except for ‘AP’ which is set to ‘1-2’. Trace Table Size must be set at a minimum of 10 megabytes and Auxiliary Trace Status set to started. Please ftp the unformatted trace that is created.
  
  For additional information about creating a CICS auxiliary trace, refer to the following Debug Tool technote: http://www.ibm.com/support/docview.wss?&uid=swg21212231 or the CICS Problem Determination Guide at http://publibfp.boulder.ibm.com/cgi-bin/bookmgr/BOOKS/DFHS1B00.

After a Problem Management Record (PMR) is open, you can submit diagnostic MustGather data to IBM using one of the following methods:

FTP diagnostic data to IBM
If FTP is not possible, email diagnostic data to techsupport@ecurep.ibm.com. You must add PMR xxxxx bbb ccc in the subject line of your email. xxxxx is your PMR number, bbb is your branch office, and ccc is your IBM country code. Click here for more details.

Always update your PMR to indicate that data has been sent. You can update your PMR, or open a new PMR, in one of two ways:

• Online: Go to the Submit and track problems page on the IBM Software Support site or IBMLink to open an Electronic Technical Response (ETR). Enter your information into the appropriate problem submission tool.
• By phone: Call 1-800-IBMSERV (1-800-426-7378) in the United States or, from other countries, go to the contacts page of the IBM Software Support Handbook on the Web (techsupport.services.ibm.com/guides/contacts.html) and click the name of your geographic region.

• Search for known problems on the Debug Tool Web site.
• Search the Debug Tool for z/OS documentation in the IBM System z Enterprise Development Tools and Compilers information center.
• Search Lookat Messages for detailed message information.
• Obtain the latest Debug Tool maintenance.
• Obtain the latest Compiler and LE maintenance for Debug Tool for z/OS..
• Review Preventive Service Planning buckets if you are installing or migrating Debug Tool for z/OS.
• Learn about the Electronic Service Request (ESR) tool for submitting problems electronically.

MustGather documents aid in problem determination and save time resolving Problem Management Records (PMRs). These documents are located on the product support sites and contain specific instructions about what documentation to gather for specific problems. You can find MustGather documents by searching on the word MustGather on the eSupport Web page:

http://www.ibm.com/software/awdtools/deployment/support/

Collecting MustGather data early, even before opening the PMR, helps IBM® Support quickly determine if:

1. Symptoms match known problems (rediscovery).


2. There is a non-defect problem that can be identified and resolved.


3. There is a defect that identifies a workaround to reduce severity.


4. Locating root cause can speed development of a code fix.