IBM Support

MustGather: Performance, hang, or high CPU issues with WebSphere Application Server on AIX



If you are experiencing performance, hang, or high CPU issues with WebSphere Application Server on AIX, this MustGather will assist you in collecting the data necessary to diagnose and resolve the issue.

Diagnosing The Problem

Complete the following three steps:

(1) Collecting the required data:

If you have not already done so, enable verboseGC and restart the server.

At the time of the problem, run one of the attached scripts with the appropriate command:

These scripts will create a file named aixperf_RESULTS.tar.gz and ten to twelve javacore text files. These scripts should be executed as the root user. As with any script, you may need to add execute permissions before executing the script (chmod).

In the above commands, [PID] is the Process ID of the problematic JVM(s). If specifying multiple Process IDs they should each be separated by a space.

(2) Collecting log files:

Collect the server logs (SystemOut.log, SystemErr.log, native_stderr.log, native_stdout.log) from the problematic server(s):


(3) Submitting required data:

Zip/Tar all the files gathered:
  • aixperf_RESULTS.tar.gz
  • all javacores (10 to 12) generated in your profileName folder
  • server logs (SystemOut.log, SystemErr.log, native_stderr.log, native_stdout.log)

Send the results to IBM Support: "Exchanging information with IBM Support"

Frequently Asked Questions:
  • What is the impact of enabling verboseGC?
    VerboseGC data is critical to diagnosing these issues. This can be enabled on production systems because it has a negligible impact on performance (< 2%).

    What is the aixperf_RESULTS.tar.gz file and where can I find it?
    The aixperf_RESULTS.tar.gz file is created while running the script and contains output from the commands called by the script. It will be created in the directory from which you execute the script.

    What are 'javacores' and where do I find them?
    Javacores are snapshots of the JVM activity and are essential to troubleshooting these issues. These files will usually be found in the <profile_root>. If you do not find the files here, you can search your entire system for them using the following command:

    find / -name "*javacore*"

If asked to do so:
The preceding data is used to troubleshoot most of these type of issues; however, in certain situations Support may require additional data. Only collect the following data if asked to do so by IBM Support.
  • System core
    Monitor process sizes and paging usage
    • The script will collect data every 5 minutes until it is stopped manually. Run the following command before the issue occurs to start the script:


      This will create two files: ps_mon.out and vmstat_mon.out.

Related information

Recording your screen to share with IBM Support
MustGather: Read first for WebSphere Application Server
How to enable verbosegc in WebSphere
IBM Thread and Monitor Dump Analyzer
Submitting information to IBM support
Steps for getting support for WebSphere Application Server
Troubleshooting guide for WebSphere Application Server
Checking if the dbx utility is installed on AIX

Cross reference information
Product Component Platform Version Edition
WebSphere Application Server - Express Hangs/performance degradation AIX 7.0, 6.1, 6.0.2, 6.0.1, 6.0, 5.1
Runtimes for Java Technology Java SDK AIX All Versions

Document information

More support for: WebSphere Application Server

Component: Hangs/Performance Degradation

Software version: 6.0, 6.1, 7.0, 8.0, 8.5,

Operating system(s): AIX

Software edition: Base, Express, Network Deployment

Reference #: 1052641

Modified date: 17 April 2019