Using Cross Component Trace to troubleshoot applications

You can use Cross Component Trace (XCT) to help diagnose problems in WebSphere® Application Server.

About this task

Administrators using WebSphere Application Server need to use log and trace files to determine whether their applications and the server are running correctly.

Depending on the nature of your applications, multiple threads within an application server may be used to handle requests, such as HTTP requests or JMS requests. Some requests may be handled by more than one application server, such as when one application server makes a request to another application server for a web services request.

You can use XCT to augment your log and trace files with correlation information. This correlation information clarifies which threads and which application server processes participated in the handling of each request.

Procedure

Enable XCT if you have not done so already.
1. Log on to the administrative console.
2. If using an admin agent topology, select a node that you want to manage and navigate to it.
3. From the navigation section in the console, choose Troubleshooting > Logs and trace.
4. Select the server that you want to enable XCT for.
5. Choose Change log detail levels.
6. Select the Configuration tab if you want to make static configuration changes, or the Runtime tab if you want to make changes to the runtime state of the server.
7. Check Enable log and trace correlation checkbox.
8. Select Include request IDs in log and trace records, Include request IDs in log and trace records and correlation log records, or Include request IDs in log and trace records, create correlation log records, and capture data snapshots as appropriate.
9. Click OK.
10. If you made your changes on the Configuration tab, save them and restart the application server.
  Best practices:
  - Enable XCT to include request IDs in log and trace files when you want to see which log and trace entries, in all threads and application server processes, are related to the same request. Request IDs are only recorded when using HPEL log and trace mode and can be seen or used for filtering using the logViewer command.
  - Enable XCT to create correlation log records when you want to log how requests branch between threads and processes, and see extra information about each request. Enabling XCT to create correlation log records might have a significant performance impact on your system, so is best suited to test and development environments.
  - Enable XCT to capture data snapshots when you want to store entire request and response bodies to the file system. Enabling XCT to capture data snapshots might have a significant performance impact on your system, so is best suited to test and development environments. XCT captures data snapshots for message requests and responses handled by the SIBus.
  - The $WSXCTCONTEXTID private header contains the XCT context ID and is used to associate requests across components. To associate outbound HTTP requests from an application server with an XCT context, set this header to the value of the XCT context ID on the inbound request to the application server.
  Avoid trouble: Data snapshots are captured and written to the $SERVER_LOG_ROOT/snapdata directory. The application server does not automatically clean up files from this directory. You will need to delete the files from this directory periodically when data snapshot capturing is enabled. Data snapshots store entire request and response contents and may include sensitive information. This option might not be appropriate for use in production environments.
Use XCT request ID information to track requests.
1. Ensure you are using High Performance Extensible Logging (HPEL) log and trace mode, since basic mode log and trace does not store request IDs. Read the Changing from basic mode to HPEL logging and tracing topic for more information.
2. Enable XCT if you have not done so already.
3. Filter your logs to look for important information, such as errors or warnings using the HPEL LogViewer command-line tool. Output your logs using advanced format so that you can see the request ID information in the logs.
  For example, use the command, as follows:
```
logViewer -minLevel WARNING -format advanced
```
```
 logViewer.sh -minLevel WARNING -format advanced
```
```
logViewer.sh -minLevel WARNING -format advanced -instance <instanceid>
```
4. When you have found log entries that are of interest to you, note the request ID associated with those entries.
5. Filter your logs by request ID using the HPEL LogViewer command-line tool and using the request IDs you noted in the previous step, as follows:
```
logViewer -includeExtensions requestID=<requestID>
```
```
logViewer.sh -includeExtensions requestID=<requestID>
```
```
logViewer.sh -includeExtensions requestID=<requestID> -instance  <instanceid>
```
  Read the LogViewer command-line tool topic for more information.
Use XCT correlation log records to determine the hierarchy of call chains.
1. Enable XCT if you have not done so already. Select Include request IDs in log and trace records and correlation log records to ensure that XCT creates correlation log records.
2. If you are using HPEL, convert your logs to text using the LogViewer command-line tool.
  For example, use the command, as follows:
```
logViewer -outLog myLog.txt
```
```
logViewer.sh -outLog myLog.txt
```
```
logViewer.sh -instance <instanceid> -outLog myLog.txt
```
  Read the LogViewer command-line tool topic for more information.
3. Use the IBM® WebSphere Cross Component Trace Logviewer tool available with the IBM Support Assistant to load log and trace files from all application servers involved in handling your requests and see the hierarchy of request call chains. Read the Using IBM Support Assistant topic for more information.

Results

The server is now configured to use XCT.