Troubleshooting
Problem
TRCCNN collects all of the information that a communications trace collects. Unlike the communications trace, TRCCNN does not drop frames on high-speed adapters. In addition, TRCCNN can be used without entering SST.
Resolving The Problem
IMPORTANT: Before you start a TRCCNN or TRCINT command, ensure that you have the corrective PTFs for APAR MA48477 applied to your system.
The corrective PTFs are required only when the defective PTF listed is applied.
The corrective PTFs are required only when the defective PTF listed is applied.
| Release | Corrective PTF | Defective PTF | Included with cumulative PTF number |
|---|---|---|---|
| R720 | MF67576 | MF67510 | 0296 |
| R730 | MF67575 | MF67511 | 0310 |
| R740 | MF67574 | MF67498 | 0303 |
When to Use TRCCNN
Use of the TRCCNN command is preferable to using a communications trace. It collects all of the valid frames of information that a communications trace collects. Unlike the communications trace, TRCCNN does not drop frames on high-speed adapters and it does not log malformed frames of information. This functionality can make TRCCNN a poor diagnostic tool choice if there is a problem with corrupted or damaged frames. TRCCNN can be used without entering SST (if a user does not know the SST user ID and password); however, the user must still have all of the special authorities required to take traces.
How to Start TRCCNN
Examples:
Use one of the following commands:
Note: TRCCNN SET(*ON) might take a long time to run, this time is the time taken to create the LIC trace table. It might keep the emulation session input inhibited for several seconds.
Example 1:
MyTraceCNN is the name given to the trace table, the LIC table that the trace data is saved in. The trace is filtered for the TCP/IP address of a PC at 9.10.45.13. This example traces all line descriptions and all interfaces on the iSeries system.
Notes:
Use of the TRCCNN command is preferable to using a communications trace. It collects all of the valid frames of information that a communications trace collects. Unlike the communications trace, TRCCNN does not drop frames on high-speed adapters and it does not log malformed frames of information. This functionality can make TRCCNN a poor diagnostic tool choice if there is a problem with corrupted or damaged frames. TRCCNN can be used without entering SST (if a user does not know the SST user ID and password); however, the user must still have all of the special authorities required to take traces.
How to Start TRCCNN
Examples:
Use one of the following commands:
Note: TRCCNN SET(*ON) might take a long time to run, this time is the time taken to create the LIC trace table. It might keep the emulation session input inhibited for several seconds.
Example 1:
MyTraceCNN is the name given to the trace table, the LIC table that the trace data is saved in. The trace is filtered for the TCP/IP address of a PC at 9.10.45.13. This example traces all line descriptions and all interfaces on the iSeries system.
TRCCNN SET(*ON) TRCTYPE(*IP) TRCTBL(MyTraceCNN) SIZE(1000 *MB) TCPDTA(*TCP () () *N '9.10.45.13')Notes:
| 1. | The amount of storage specified by SIZE parameter is immediately allocated from the system auxiliary storage pool (ASP 1). This storage space is not dynamically allocated as it is needed. This storage space is not available for use by the system except to record trace-related information. Because the maximum trace size is 258048 megabytes, the *MAX setting must be used with care (or better yet, not used at all). When you specify *MAX or any other large value on this parameter, the amount of free space in the system ASP must be checked. Use the Work with System Status (WRKSYSSTS) command to determine the amount of available free space in the system ASP. System performance degradation can result when the size of the free space in the system ASP is significantly reduced as a result of the value specified. After the trace is complete and the user that started the trace logs off the emulation session, the trace table is removed. However, it is unknown whether all of the allocated space is freed up or not. If storage is not freed up, it is possible that the trace table was not deleted. SST, Option 1 (Start a service tool), and Option 2 (Trace Licensed Internal Code) can be used to manually delete the trace table; freeing up at least some of the allocated storage. It can take several minutes to delete the trace, and users appear to be unable to access the WRKSYSSTS screen while the trace is being deleted. |
| 2. | 1 Gig is sufficient in nearly every situation. |
Example 2:
The second example is similar to Example 1 but shows the trace size set to 500 Meg and the trace filtered for only the ETHLINE line description.
TRCCNN SET(*ON) TRCTYPE(*IP) TRCTBL(MyTraceCNN) SIZE(500 *MB) TCPDTA(*TCP () () *N '9.10.45.13' ETHLINE)NOTE: Specifying a line description can cause the resulting trace to show only half of the connection. If the system has multiple network adapters that are connected to the same network, the system typically sends on one adapter and receives on the other. If you specify a line description in the trace on such a system, the trace captures sent or received data only, not both, rendering the trace useless.
Example 3:
This example traces all activity over all TCP/IP addresses. It traces all active line descriptions and interfaces on the system.
TRCCNN SET(*ON) TRCTYPE(*IP) TRCTBL(MYTRACECNN) SIZE(1000 *MB) TCPDTA(*TCP () ())Example 4:
This example shows how to trace TCP activity over loopback. It is useful for tracing IBM® Toolbox for Java™ activity running on OS/400 or i5/OS.
TRCCNN SET(*ON) TRCTYPE(*IP) TRCTBL(MyTraceCNN) TCPDTA(*N () () '127.0.0.1' '127.0.0.1')Example 5:
This example shows how to trace TCP for one IP address and have the trace stop and print when a predefined message is logged to the specified message queue. In this example, the trace stops as soon as the message SQL7008 is logged in any QZDASOINIT job. Note if the message SQL7008 appears in ANY QZDASOINIT job, this trace stops and prints.
TRCCNN SET(*ON) TRCTYPE(*IP) TRCTBL(COMMTRACE1) SIZE(160000) TCPDTA(*N () () *N '1.2.3.4') WCHMSG((SQL7008)) WCHMSGQ((*JOBLOG)) WCHJOB((*ALL/QUSER/QZDASOINIT))When it is necessary to have the trace stop only after the message occurs in a job that is servicing a connection from a specific IP address, that can be done but there is extra setup. First, a custom subsystem needs to be created and the system configured so that connections from our target IP address are serviced by jobs in the custom subsystem rather than the default. To follow this example with the database host server, the QZDASOINIT prestart job entry in that custom subsystem must be changed so that the job name is something other than QZDASOINIT. For example, I created a copy of QUSRWRK and called it ODBCWRK. I then modified the prestart job entry so that the job name is ODBCSOINIT: CHGPJE SBSD(ODBCWRK) PGM(QZDASOINIT) JOB(ODBCSOINIT). Then, any user that is routed to that subsystem has their database host server jobs serviced by ODBCSOINIT jobs. For more information about host to set up an environment where you route connections from one IP address to their own subsystem, see technote N1014359. The previous trace command is modified so that the job name is now ODBCSOINIT instead of QZDASOINIT:
TRCCNN SET(*ON) TRCTYPE(*IP) TRCTBL(COMMTRACE1) SIZE(160000) TCPDTA(*N () () *N '1.2.3.4') WCHMSG((SQL7008)) WCHMSGQ((*JOBLOG)) WCHJOB((*ALL/QUSER/ODBCSOINIT))Example 6:
This example shows how to trace secure connections that use TLS (SSL) encryption. This traces sockets traffic before and after the data is passed to or from the encryption algorithm. Since the pre-encryption layer does not contain TCP/IP information, this trace type cannot also contain TCP filters such as IP address or port #. Omitting these filters results in larger trace output sizes so it might be necessary to collect the traces when other SSL/TLS traffic on the system is low. Example trace command:
TRCCNN SET(*ON) TRCTYPE(*SSL) TRCTBL(MyTraceCNN) SIZE(1000 *MB)*Note*: A concurrent TRCCNN trace of type *IP, with TCP/IP filters, is generally recommended so that the unencrypted SSL trace data can be matched up with the encrypted trace containing the SSL handshake and TCP/IP data.
Ending TRCCNN
Ending TRCCNN
The preferred output from the trace is a QPCSMPRT spooled file. This output contains all of the trace information as well as job and TDE information that is useful to IBM support. To stop the trace and produce spooled file output, run the command:
TRCCNN SET(*OFF) TRCTBL(MyTraceCNN)For large traces, this command can take some time to complete. In that case, submit the job:
SBMJOB CMD(TRCCNN SET(*OFF) TRCTBL(MYTRACECNN))Starting in 7.1, the TRCCNN command can create output in a stream file packet capture format that can be read by Wireshark and similar tools. Be certain that PTFs MF59955 for 7.1 or MF59962 for 7.2 are applied to the system before you end the trace this way or records might be missing from the trace. This output is smaller, more portable and is the preferred method for any system that is not running 6.1 or earlier. Here is how to stop the trace and create the packet capture (PCAP) output:
This command creates a file in the root of the IFS named trccnn.pcap and the *YES option causes the file to be overwritten if it existed. This stream file can be copied to a PC by using a mapped network drive, Access Client Solutions, or FTP.
Creating a Packet Capture Stream File From Printed TRCCNN Output
TRCCNN SET(*OFF) TRCTBL(MYTRACECNN) OUTPUT(*STMF) TOSTMF('/trccnn.pcap' *YES)This command creates a file in the root of the IFS named trccnn.pcap and the *YES option causes the file to be overwritten if it existed. This stream file can be copied to a PC by using a mapped network drive, Access Client Solutions, or FTP.
Creating a Packet Capture Stream File From Printed TRCCNN Output
If you have a spooled file as the output of the TRCCNN command, the QSYSPRT spooled file can be used to create a packet capture format stream file by using the CVTTRCCNN command. The CVTTRCCNN command is part of the IBM i support library, which can be obtained as a save file from here: http://public.dhe.ibm.com/services/us/igsc/qsptlib/
To use that tool, add QSPTLIB to your library list:
ADDLIBLE QSPTLIBEnter the command CVTTRCCNN and prompt it using the F4 key to provide the details. The simplest case where you started and stopped the trace in the same job and have only one QSYSPRT spooled file is:
CVTTRCCNN SPLF(QSYSPRT) OUTF('/trccnn.pcap')Determining whether a TRCCNN is active on the system
The easiest way to determine whether a trace is active is to use the command
WRKTRC *LIC; however, that command is new in V5R4. On older releases, it is necessary to go into SST and check for the LIC trace. Do the following:
| 1. | On the operating system command line, type the following: STRSST Press the Enter key. |
| 2. | Type the SST user ID and password at the prompt. |
| 3. | Select Option 1, Start a Service Tool, and press the Enter key. |
| 4. | Select Option 2, Trace Licensed Internal Code, and press the Enter key. |
| 5. | Look for a trace by the same name as the TRCTBL setting that was used when you ran the TRCCNN *ON (in the previous example, TRCTBL(MyTraceCNN). If the status of the LIC trace entitled MyTraceCNN is ACTIVE, the TRCCNN is still running. If the MyTraceCNN Trace is in some status other than ACTIVE, it is still being started or of being stopped. Once TRCCNN *OFF completes, the MyTraceCNN Trace disappears from the list of LIC Traces. |
To trace multiple clients with TRCCNN you need to run multiple traces, you cannot filter the trace for more than one remote IP address.
Example:
TRCCNN SET(*ON) TRCTBL(TRC88) TRCTYPE(*IP) TCPDTA(*N () () *N '9.10.46.88')
TRCCNN SET(*ON) TRCTBL(TRC22) TRCTYPE(*IP) TCPDTA(*N () () *N '9.10.46.22')
TRCCNN SET(*ON) TRCTBL(TRC19) TRCTYPE(*IP) TCPDTA(*N () () *N '9.10.46.19')A unique trace table is generated for each trace.in SST for each session.
Delivering the Trace to IBM
If you created a stream file from the TRCCNN, copy it to a PC by using a mapped drive, Navigator for i, Access Client Solutions, or FTP. Then, upload it to the Enhanced Customer Data Repository (ECUREP) by using the web tool at http://www.ecurep.ibm.com
The spooled files can be transferred to the PC as text files (do not convert them to PDF format) by using IBM i Access Client Solutions:
- Open the main IBM i Access Client Solutions program, select your system, then click 'Printer Output'.
- If you do not see the QPCSMPRT and QSYSPRT spool files, press Ctrl+F to apply a different filter. Specify the user that was used to stop the trace, all output queues, all user data, and all dates and press 'OK' to apply the filter.
- Clicking the 'Date created' column heading two times brings the most current spool files to the top of the list.
- Select 'Preferences' from the 'Edit' menu and make sure that the option 'Use PDF format...' is not checked (we need the traces in a text file format) and click 'OK'.
- Hold the Shift key down and click the QSYSPRT and QPCSMPRT spool files created by the trace, right-click them and select 'Download...'.
- Choose a location for the files to be created on your computer and press OK.
Or you can copy the output queue to your PC in a save file format by using the information in the following document:
N1015044: Saving an Output Queue with SPLFDTA(*ALL) to Send Spooled Files to Software Support
Once you have the data on a PC, you can upload it to the Enhanced Customer Data Repository (ECUREP) by using the web tool at http://www.ecurep.ibm.com
[{"Product":{"code":"SWG60","label":"IBM i"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Component":"Host Servers","Platform":[{"code":"PF012","label":"IBM i"}],"Version":"Version Independent","Edition":"","Line of Business":{"code":"LOB57","label":"Power"}}]
Historical Number
31371244
Was this topic helpful?
Document Information
Modified date:
21 July 2021
UID
nas8N1019273