IBM Support

Using TPROF FOR WINDOWS Tool to aid in the debugging of high CPU usage issues on Microsoft Windows

Question & Answer


Question

How do I use the TPROF FOR Windows® TOOL in high CPU usage problems on Microsoft® Windows?

Answer

To aid in the collection of high CPU data, the TPROF FOR WINDOWS tool collects additional information about the processes and threads consuming high CPU. This document is designed to be used in conjunction with the MustGather: Performance, hang, or high CPU issues on Windows.

Description about TPROF FOR WINDOWS Tool

TPROF is a system profiling tool used for system performance analysis. TPROF is based on the sampling technique which encompasses the following steps:

* Interrupt the system periodically (TPROF tick).
* Record address, process id and thread id of interrupted code.
* Return to the interrupted code

The CPU data generated by this tool is pretty similar to tprof on AIX data. For more information about this tool please visit : http://perfinsp.sourceforge.net/tprof.html

Download, Install And Run The Tool

1> Download and install the tool following the instructions(steps 1 through 6) given in the link below:
http://perfinsp.sourceforge.net/install_windows.html.

NOTE: Administrator privileges are needed to install the tool. Read the licensing information before downloading the tool.

2> Once downloaded and installed wait for the high CPU to occur. At the time of high CPU start the tool as :
    a. Open a command prompt and run the following command

    > cd /d c:\ibmperf\bin
    > setrunenv

    b. Enter the following command to run tprof

    > run.tprof


    As soon as you see the following message hit ENTER to start the trace :

    ***** Press ENTER when ready to begin tracing or Ctrl-C to quit ...


    Once started you will see the following message :

    Trace started at 11:52:50.28 ...

    C:\ibmperf\bin>SWTRACE ON
    Command completed successfully. Tracing started.

    *****
    *****
    ***** Tracing. Press ENTER when ready to stop ...


3> Collect the javacores following the MustGather: Performance, hang, or high CPU issues on Windows


4> Stop the tprof profiling by pressing ENTER. It will take couple of seconds to generate the report. The tprof report will be generated once you see the following message :

*****************************************************************
TProf report "C:\ibmperf\bin\tprof.out" generated.
*****************************************************************

To engage IBM Support send the generated tprof.out and other logs as collected in MustGather: Performance, hang, or high CPU issues on Windows
Send the results to IBM Support.

Analyzing The TPROF report

The TPROF report in tprof.out has following sections :

)) ProcessGives the CPU usage of each process.
)) Process_ModuleGives the CPU usage of each module(library) within each process.
)) Process_Module_SymbolGives the CPU usage of functions within each module of each process.
)) Process_ThreadGives the CPU usage of each thread within each process.
)) Process_Thread_ModuleGives the CPU usage of each module within each thread within each process.
)) Process_Thread_Module_SymbolGives the CPU usage of functions within each module of each thread within each process.
)) ModuleGives the CPU usage of modules
)) Module_SymbolGives the CPU usage of function within each modules

1> First see the process which took high CPU from the ")) Process" section.

================================
)) Process
================================

LAB TKS %%% NAMES

PID 78290 91.86 java.exe_0f10
PID 2409 2.83 IdleProcess_0000
PID 529 0.62 svchost.exe_0440

In the above sample output its the java.exe process which took 91.86% of CPU. The number in hex appended to each process after '_' is the process ID. So here its "0f10" which is 3856. Thus the java.exe process with process ID 3856 was the one which took 91.86% of CPU during the run of TPROF.

2> Once the process taking high CPU is identified, to find out the thread(s) taking high CPU see the ")) Process_Thread" section.

================================
)) Process_Thread
================================

LAB TKS %%% NAMES

PID 78290 91.86 java.exe_0f10
TID 35627 41.80 tid_155c
TID 32131 37.70 tid_00fc
TID 953 1.12 tid_14e0

In the above sample output its the thread with native id 0x155c which took maximum amount of CPU. The number in hex appended to each thread after '_' is the thread ID.

To find the thread in the corresponding javacore(s) search for the string '155c', here is the output :

3XMTHREADINFO "WebContainer : 21" (TID:0x000000001B604300, sys_thread_t:0x000000000C51E6F8, state:CW, native ID:0x000000000000155C) prio=5
4XESTACKTRACE at org/apache/xml/dtm/ref/DTMDefaultBaseTraversers$DescendantTraverser.next(DTMDefaultBaseTraversers.java:712)
4XESTACKTRACE at org/apache/xpath/axes/AxesWalker.getNextNode(AxesWalker.java:333(Compiled Code))
4XESTACKTRACE at org/apache/xpath/axes/AxesWalker.nextNode(AxesWalker.java:361(Compiled Code))

Note that '155c' is there in the native ID.


3> To find out the module taking high CPU see the ")) Process_Module" section.

================================
)) Process_Module
================================

LAB TKS %%% NAMES

PID 78290 91.86 java.exe_0f10
MOD 49822 58.46 C:\WINDOWS\system32\drivers\mfetdik.sys
MOD 18779 22.03 E:\apps\IBM\WebSphere\AppServer\java\jre\bin\j9jit23.dll
MOD 4179 4.90 C:\WINDOWS\system32\ntoskrnl.exe

In the above sample output its the module mfetdik.sys taking maximum amount of CPU. After that its the j9jit23.dll which is taking CPU.


4> To find out the functions within each module which are taking high CPU see the ")) Process_Module_Symbol" section. Please note that this section displays the exported symbols of the modules.

================================
)) Process_Module_Symbol
================================

LAB TKS %%% NAMES

PID 78290 91.86 java.exe_0f10
MOD 49822 58.46 C:\WINDOWS\system32\drivers\mfetdik.sys
SYM 49806 58.44 NoSymbols
SYM 16 0.02 <<.text>>

MOD 18779 22.03 E:\apps\IBM\WebSphere\AppServer\java\jre\bin\j9jit23.dll
SYM 13717 16.09 TR_IProfiler::searchForSample
SYM 974 1.14 parseBuffer

In the above sample output there is no symbol information for the function taking maximum CPU within mfetdik.sys. For j9jit23.dll its the TR_IProfiler::searchForSample taking maximum amount of CPU.

The above information will give an idea where the CPU is spinning. If required see the Process_Thread_Module, Process_Thread_Module_Symbol, Module, Module_Symbol sections for more granular information.

[{"Product":{"code":"SSEQTP","label":"WebSphere Application Server"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Component":"100% CPU Usage","Platform":[{"code":"PF033","label":"Windows"}],"Version":"9.0;8.5.5;8.5;8.0;7.0","Edition":"","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
23 June 2018

UID

swg21403450