Troubleshooting
Problem
Just-In-Time Compiling, or JIT, is a component of the IBM SDK designed to compile Java™ bytecode into native code. A JIT failure can occur while compiling Java bytecode into native code, while optimizing compiled code, or while executing the code that has been compiled.
Cause
A JIT failure can occur during compiling or when the compiled code is run.
Symptoms
Stack overflow, looping, crash, incorrect output, High CPU
Resolving The Problem
This document is applicable for the IBM SDK 5.0, SDK 6.0.X, SDK 7.0, SDK 7.1, and SDK 8.0 that are bundled with various versions of WebSphere Application Server 6.1 or later (or WebSphere Liberty Profile that is using a compatible IBM SDK).
If determined that the Java Virtual Machine (JVM) failure is occurring in the JIT library, follow in order this 3 step process:
- Identifying the failing method
- Exclude the method
- Adjusting JIT optimization levels (if asked to do so)
1. Identifying the Failing Method
- For a crash during compilation, look for the line in the javacore that contains the string “Compiling method”. This is the method you will exclude.
1XHEXCPMODULE Compiling Method: java/lang/Math.max(II)I;
- If the javacore from the crash does not contain the line above, it may have crashed in code already compiled by the JIT. Look for the current thread that was executing at the time of the crash. The method executing in the current thread will contain the string "Compiled Code".
4XESTACKTRACE at java/lang/Math.max(Math.max:314(Compiled Code))
- For scenarios that do not have the above output, such as high CPU, enable JIT tracing.
-Xjit:verbose={compileStart|compileEnd},vlog=/tmp/jit.log
If you omit vlog, the output will go into the native_stderr.log.
Sample output
(warm) Compiling java/lang/Math.max(II)I t=122140
+ (hot) java/lang/Math.max(II)I @ 0x10C11DA4-0x10C11DDD
The last entries listed in the trace are the methods to exclude.
2. Exclude the Method
Excluding individual classes and methods will prevent the JIT from compiling code suspected of causing a JIT failure.
-Xjit:exclude={package/class.method}
Appending multiple classes and methods is allowed:
-Xjit:exclude={package1/class.method|package2/class.method}
Also, multiple exclude options can be used by using the comma separator
-Xjit:exclude={package1/class.method},exclude={package2/class.method}
- Setting the JVM Arguments
- WebSphere Application Server 7.0, 8.0, 8.5.X, or 9.0
In order to debug JIT failures and determine a temporary work around, edit the Generic JVM Arguments and add the -Xjit argument:
To set the following Environment Entries from the administrative console:
Select Administrative Agent > Java Process Definition > Java Virtual Machine > and append to the Generic JVM Arguments
Alternatively from a Deployment Manager or Application Server administrative console:
Select Servers > expand Server Types > WebSphere application servers > server_name> expand Java and Process Management > Process Definition > Java Virtual Machine > and append to the Generic JVM Arguments
WebSphere Application Server 8.5.5 Screenshot of "Admin Console"
WebSphere Application Server 7.0 Screenshot of "Admin Console"
- WebSphere Application Server 6.1
To set the following Environment Entries from the administrative console:
Select Servers > Application Servers > server_name > expand Java and Process Management > Process Definition > Java Virtual Machine > and append to the Generic JVM Arguments
Example
The failing class and method is identified as java/lang/Math.max(II)I. To skip this method, add the following JVM argument:
-Xjit:exclude={java/lang/Math.max(II)I}
You can also use wildcards if the full method string is not known, or you want to target multiple methods:
-Xjit:exclude={java/lang/Math.max*}
Frequently Asked Questions (FAQs)
What is JIT?
JIT stands for Just-In-Time compiler. The purpose of JIT is to compile code that is most frequently used to speed up subsequent calls to those methods. Long running applications will have the most benefit using JIT.
When does code get compiled?
Classes or methods that are frequently called will be compiled with the JIT automatically. The more frequently a method is called, the more aggressive the optimization will be. Thread stacks in a javacore that list “(Compiled Code)” are already compiled.
Is there a way to disable the Just-In-Time compiler?
Yes. Append one of the following to the generic JVM arguments.
-Xint
-Djava.compiler=NONE
In addition, the deprecated environment variable is still valid in IBM SDK 5.0 and 6.0, and can also be used to disable the JIT compiler:
JAVA_COMPILER=NONE
Can multiple options be combined in the same argument?
Yes. Here is the general syntax:
-Xjit:param,param=value,...
-Xjit:exclude={package/class.method}(param,param=value)...
-Xjit:{package/class.method}(param,param=value),...
Please note that using multiple -Xjit arguments are not recommended and will not be supported. Instead combine all into one -Xjit argument, separating each value using commas.
How do I use wildcards?
The wildcard character * can be used in either the package/class name or in the method name.
To skip all methods in the Math class:
-Xjit:exclude={java/lang/Math.*}
To skip all classes/methods in the java/lang package:
-Xjit:exclude={java/lang/*}
To skip all methods starting with “writeTo”:
-Xjit:exclude={com/ibm/mq/MQMsg2.writeTo*}
How do I stop JIT from compiling more aggressive optimizations if larger bytecode is produced?
Add the parameter disableInlining to the JIT parameters.
-Xjit:disableInlining
You can even use disableInlining, or any JIT parameter, on specific classes and/or methods.
-Xjit:{java/lang/Math.max(II)I}(disableInlining)
Can these options be used on Solaris or HP-UX?
No. The -Xjit argument is specific to the IBM SDK. Solaris and HP-UX versions of WebSphere Application Server are bundled with Sun’s implementation of the Java SDK. They do have a real-time compiler called HotSpot and it has its own configuration options.
Other Options
3. Adjusting JIT Optimization Levels
To narrow down the cause of the JIT failure, adjusting the compiling optimization can help. Setting noOpt disables all optimizations but still allows the JIT to compile the code.
-Xjit:optLevel=noOpt
-Xjit:{java/lang/Math.max*}(optLevel=noOpt)
The optimization list below is ordered from the most aggressive to the least expensive.
1. scorching
2. veryHot
3. hot
4. warm <-- Initial compiling usually occurs at this level
5. cold
6. noOpt
Adjusting the JIT threshold
Previously referred to as the MMI threshold, this determines when the JIT will perform compilation. Once the counter reaches 0, compilation will be executed and the counter will reset.
-Xjit:count=VALUE
By default, VALUE is set to 1000.
Setting VALUE to 0 means every method is compiled, and thread execution will wait until compilation is finished. This is called synchronous compilation mode.
-Xjit:count=0
-Xjit:exclude={java/lang/Math.max*}(count=0,optLevel=hot)
Related Information
[{"Product":{"code":"SSEQTP","label":"WebSphere Application Server"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Component":"Java SDK","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF016","label":"Linux"},{"code":"PF033","label":"Windows"}],"Version":"9.0;8.5.5;8.5;8.0;7.0","Edition":"Base;Express;Liberty;Network Deployment","Line of Business":{"code":"LOB36","label":"IBM Automation"}},{"Product":{"code":"SS7JFU","label":"WebSphere Application Server - Express"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Component":"Java development","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF016","label":"Linux"},{"code":"PF033","label":"Windows"}],"Version":"8.0;7.0;6.1","Edition":"","Line of Business":{"code":"LOB45","label":"Automation"}}]
Was this topic helpful?
Document Information
Modified date:
15 June 2018
UID
swg21294023