IBM Support

Troubleshooting native memory issues

Troubleshooting


Problem

There are two types of memory problems which will throw an OutOfMemoryError: If a Java™ object cannot be allocated or if there is no more memory available for native code allocations (c malloc) or threads. The first case is called Java heap exhaustion and the second, a native memory issue. Finding the users of native memory can be a lengthy process, as there is no way of knowing which code allocated the native memory. There are no comparable heapdumps for native memory, so we have to use trial and error, going through each of the potential native memory users until we find the one that is causing the problem. Following are the most common causes of native memory issues in WebSphere Application Server, so you should try them first. They are all trying to reduce the native memory used (the native memory footprint) so both the java heap memory and native memory can fit in the address space for the application server process. If these steps do not solve your native memory issue, IBM support will guide you through the rest.

Symptom

The process size for the application server will grow quite large. You may also see OutOfMemoryError’s thrown, malloc errors in SystemOut.log or errors creating threads.
JVMDBG001: malloc failed to allocate …
“Failed to fork OS thread”, or “cannot create any more threads”.

Resolving The Problem


1. Check the maximum heap size. This is the most common cause for native memory problems in 32bit WebSphere Application Server installs. If you have a 64bit install, please bypass this step, the heap size will not be causing the problem.

Since the native memory area is the memory for the process left over from the Java heap, the larger the heap, the smaller the native area. If the maximum heap size is too big, the address space reserved for it will encroach on the native memory area, causing there to not be enough native memory for the app server. If your maximum heap size is 1536m or higher, we will want to lower it to 1280m or even 1024m.


Each 32bit process has an address space of 2 ** 32, (for example., 4,294,967,296) bytes or 4GB. In each process, the operating system reserves just under 2GB of the address space. This is used for operating system libraries, etc. This leaves just over 2GB of address space for a process's code to use (in this case Java). This address space is shared by theJava heap and the native area. When you increase the Java heap size you are decreasing the native area, as it only has what is left over from the heap. It is the max heap setting here, not just how large the heap has expanded to. The heap may have only expanded to 256m, but if the max heap is set to 1792m, then 1792m of address space is reserved and there will only be just over 256m available for native memory use.

NOTE: zSeries machines have 31bit and 64bit versions. When it is not 64bit WebSphere Application Server, it will be 31bit. The Java Virtual Machine (JVM) version shows 32bit, but this is incorrect. With only 31 bits, the address space is half that of 32bit addressing (2**31 == 2GB). This makes the available space for the Java heap and native much less. In fact you have to make special operating system settings if you want a heap size greater than 800m (mapped_base). If you are seeing a native OutOfMemoryError on Linux on zSeries, you will want to reduce the heap size if it is more than 600m.
2. Reflection delegating classloaders. When using Java reflection, if the JVM uses a Java bytecode accessor on the class being reflected, it will use native memory. If there is frequent use of Java reflection, there can be a significant amount of native memory use.  You will see a large number of classloader/class pairs in the javacores like:
2CLTEXTCLLOAD         Loader sun/reflect/DelegatingClassLoader(0x0000000700B8D9C0)
3CLTEXTCLASS             sun/reflect/GeneratedMethodAccessor5(0x0000000030331100)
They can add a Generic JVM argument to control this: -Dsun.reflect.inflationThreshold=0

See the following page for more detail and instructions on how to limit the native memory use of the Java reflection accessors:
Potential native memory use in reflection delegating classloaders
---- Side note on similar looking pairs:
The setting does NOT control these pairs however:
2CLTEXTCLLOAD          Loader sun/reflect/DelegatingClassLoader(0x00000002D64A47B0)
3CLTEXTCLASS               sun/reflect/GeneratedSerializationConstructorAccessor28805(0x000000000C8DC500)
** From past history from working with Java Level 3, this is an application issue that the client's application team must address.  
=== Begin Java Level 3 feedback ===
The sun.reflect.DelegatingClassLoader instances have all been created to load GeneratedSerializationConstructorAccessor classes. These are not created by reflection operations, but for serialization when a java.io.ObjectStreamClass is created for a specific class (via ObjectStreamClass.lookup()). Importantly, this code path always creates a Java bytecode accessor, never a JNI accessor - i.e. the inflation threshold does not apply here.
I don't think there's anything that can be done from a JDK standpoint. The large number of these accessors is a consequence of the extremely large number of different classes being serialized in this JVM process. It's not clear whether something can be done at the WAS or application level to reduce the number amount of serialization.
We could try to get further insight into exactly what is driving the serialization by using an Xtrace option to record a stack trace whenever one of the GeneratedSerializationConstructorAccessor classes is created:
-Xtrace:none,maximal=mt,trigger=method{sun/reflect/GeneratedSerializationConstructorAccessor*.<init>,jstacktrace},output={/tmp/trace_%p.bin,500m}
The binary trace will be written to a file called "/tmp/trace_[PID].bin", up to a maximum size of 500MB. If the customer gathers this trace and uploads it, I can format it and try to aggregate the stacks.
A system dump taken when the number of GeneratedSerializationConstructorAccessors is high might also help us to provide further insight. For example, we should be able to determine which classes these Accessors were generated for.
=== End Java Level 3 feedback ===
3. JAXB classloaders - The javacores for show a large number of the same classloader/class pairs like:
2CLTEXTCLLOAD         Loader com/ibm/xml/xlxp2/jaxb/codegen/AbstractGeneratedStubFactory$RootStubClassLoader(0x000000070857FBC8)
3CLTEXTCLASS             com/ibm/xml/xlxp2/jaxb/unmarshal/JAXB_Deserialization_Root_Stub(0x00000000348F1C00)
This can be controlled by adding a JVM Custom Property of:
NAME: com.ibm.xml.xlxp.jaxb.opti.level
VALUE: 0
For more detail see the section "com.ibm.xml.xlxp.jaxb.opti.level" on "Java virtual machine custom properties"
4. LINUX ONLY - Malloc Arenas - If there is native memory growth on Linux, get linperf.sh output and the javacores.  If there is growth seen in top.out in the RES column for a pid that is not reflected in the NATIVEMEMINFO section of the javacore, then it could be malloc arena's.  The first thing Java Level 3 would ask the client to do is to set an Environment entry on the JVM getting the native growth of:
NAME: MALLOC_ARENA_MAX
Value: 1
** Uppercase the name
** For navigation see "Setting environment entries in WebSphere Application Server"
https://www.ibm.com/support/pages/node/88901
This resolves many of the issues.  If not, then move on to the next steps.


5. WebContainer DirectByteBuffer use. The WebContainer may store a large number of buffers in native memory when sending large responses back to the requesters asynchronously. We can reduce the native memory footprint by setting a WebContainer property to send responses in synchronous mode. The downside of synchronous mode is that it will use a thread exclusively during the response..
See the following page for more detail and instructions:
Setting the WebSphere Application Server WebContainer to synchronous mode
** NOTE: If verbosegc is enabled, one indicator that they have DBB's allocated but not getting all references cleaned up is to search for the word phantom.  If you see many phantom references that have 0 or a low number cleared, this can indicate that changing channelwritetype to sync will resolve this growth.


6. AIO Native Transport. (WebSphere Application Server 6.1 and higher).
In general, running the WebSphere Application Server TCP Channel in AIO mode gives better performance and scalability than using the TCP Channel in NIO mode. But this faster performance, different timing, and slightly different threading design can increase the native memory footprint. If the application server is short on native memory, sometimes disabling the AIO native library can increase the available address space for native memory.
See the following page for more detail and instructions:
Disabling AIO (Asynchronous Input/Output) native transport in WebSphere Application Server


7. Thread pool issue. This native memory footprint issue can sometimes occur when data is attached to thread pool threads, via ThreadLocal java objects, and these threads are frequently created and destroyed. When a thread is released the memory it was holding onto is dereferenced for garbage collection. This includes heap and native memory. If there are any garbage collection problems or another object is referencing the ThreadLocal objects, then they cannot get garbage collected and therefore the native memory does not get freed either. To fix the problem, use a fixed size thread pool.
See the following page for more detail and instructions to set your thread pool size:
Potential native memory use in WebSphere Application Server Thread Pools
If none of the above resolves the issue, then we'll need to get the proper detailed data depending on the OS they're running on (Linux Native Memory Tracker for Linux, UMDH for Windows, etc) but we should get this at Java L3's request.
 
Other considerations that can manifest in native OutOfMemory errors:
1) A system may not have enough physical memory (RAM) to run all of the JVMs:
"Considerations for capacity planning to avoid OutOfMemory (OOM) errors"
https://www.ibm.com/support/pages/node/531399
2) A JVM running on Linux may get an error:
""systhrow" (00040000) Detail "java/lang/OutOfMemoryError" "Failed to create a thread: retVal -1073741830, errno 11" received"
See the technote "Insufficient ulimit -u (NPROC) Value Causes Native OutOfMemory warning message to be issued"
https://www.ibm.com/support/pages/node/232509
3) A JVM may throw a native OutOfMemory error if the heap is starting in the lower 4Gb address space and causes issues with things that must go in the lower 4Gb address space when compressed references is used.  For more detail see "IBM Using -Xgc:preferredHeapBase with -Xcompressedrefs"
https://www.ibm.com/support/pages/node/507367
4) Direct Byte Buffers
If you get a message in the SystemOut.log containing:
java.lang.OutOfMemoryError: Direct buffer memory::Please use appropriate '<size>' via -XX:MaxDirectMemorySize=<size>
You have "-XX:MaxDirectMemorySize=<value>" set and you are using Direct Byte Buffers at a higher rate than you have it set for.  You could have the value set too low or you could have an excess of Direct Byte Buffers used and referenced at one time.  For more details see the section in the "IBM WebSphere Application Server Performance Cookbook"

[{"Product":{"code":"SSEQTP","label":"WebSphere Application Server"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Component":"Out of Memory","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF010","label":"HP-UX"},{"code":"PF016","label":"Linux"},{"code":"PF027","label":"Solaris"},{"code":"PF033","label":"Windows"}],"Version":"9.0;8.5.5;8.5;8.0;7.0","Edition":"","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
01 July 2023

UID

swg21373312