IBM Support

java.lang.OutOfMemoryError: Java heap space - Socket Listener error shows in noapp.log -GIS 4.3

Technote (troubleshooting)


Problem(Abstract)

java.lang.OutOfMemoryError: Java heap space - Socket Listener error shows in noapp.log -GIS 4.3

Symptom

Sporadically OutOfMemory errors on node 2 of a 2 node GIS cluster. Currently running GIS 4.3 on a windows platform using MS SQL 2005. No other infromation available as there are no heapdumps or javacores available.


Error Message

[2010-09-17 11:05:49.174] ALL 000000000000 GLOBAL_SCOPE Exception in thread "SocketListener1-1"
[2010-09-17 11:05:49.174] ALL 000000000000 GLOBAL_SCOPE java.lang.OutOfMemoryError: Java heap space
[2010-09-17 11:12:32.69] ALL 000000000000 GLOBAL_SCOPE
[2010-09-17 11:12:32.705] ALL 000000000000 GLOBAL_SCOPE Exception in thread "Thread-1573"
[2010-09-17 11:12:32.721] ALL 000000000000 GLOBAL_SCOPE Exception in thread "Thread-23"
[2010-09-17 11:12:32.736] ALL 000000000000 GLOBAL_SCOPE Exception in thread "Thread-26"
[2010-09-17 11:12:32.736] ALL 000000000000 GLOBAL_SCOPE Exception in thread "RMI RenewClean-[aaa.bbb.ccc.ddd:xxxx]"
[2010-09-17 11:12:32.752] ALL 000000000000 GLOBAL_SCOPE Exception in thread "JNP Server"
[2010-09-17 11:12:32.768] ALL 000000000000 GLOBAL_SCOPE Exception in thread "RMI RenewClean-[aaa.bbb.ccc.ddd:xxxx,com.sterlingcommerce.woodstock.util.frame.jndi. AddrClientFactory@139746e]"
[2010-09-17 11:12:34.173] ALL 000000000000 GLOBAL_SCOPE Exception in thread "RMI RenewClean-[aaa.bbb.ccc.ddd:xxxx,com.sterlingcommerce.woodstock.util.frame.jndi. AddrClientFactory@104e432]"
[2010-09-17 11:12:34.173] ALL 000000000000 GLOBAL_SCOPE Exception in thread "RMI RenewClean-[aaa.bbb.ccc.ddd:xxxx,com.sterlingcommerce.woodstock.util.frame.jndi. AddrClientFactory@17d97e5]"
[2010-09-17 11:12:34.188] ALL 000000000000 GLOBAL_SCOPE Exception in thread "RMI RenewClean-[aaa.bbb.ccc.ddd:xxxx,com.sterlingcommerce.woodstock.util.frame.jndi. AddrClientFactory@6d3947]"
[2010-09-17 11:12:34.188] ALL 000000000000 GLOBAL_SCOPE java.lang.OutOfMemoryError: Java heap space
[2010-09-17 11:12:34.204] ALL 000000000000 GLOBAL_SCOPE

[2010-09-17 11:12:34.204] ALL 000000000000 GLOBAL_SCOPE java.lang.OutOfMemoryError: Java heap space
[2010-09-17 11:12:35.672] ALL 000000000000 GLOBAL_SCOPE

[2010-09-17 11:12:35.687] ALL 000000000000 GLOBAL_SCOPE java.lang.OutOfMemoryError: Java heap space
[2010-09-17 11:12:35.687] ALL 000000000000 GLOBAL_SCOPE

[2010-09-17 11:12:39.091] ALL 000000000000 GLOBAL_SCOPE java.lang.OutOfMemoryError: Java heap space
[2010-09-17 11:12:39.091] ALL 000000000000 GLOBAL_SCOPE

[2010-09-17 11:12:39.122] ALL 000000000000 GLOBAL_SCOPE java.lang.OutOfMemoryError: Java heap space
[2010-09-17 11:12:39.122] ALL 000000000000 GLOBAL_SCOPE

[2010-09-17 11:12:39.138] ALL 000000000000 GLOBAL_SCOPE java.lang.OutOfMemoryError: Java heap space
[2010-09-17 11:12:39.138] ALL 000000000000 GLOBAL_SCOPE

[2010-09-17 11:12:39.153] ALL 000000000000 GLOBAL_SCOPE java.lang.OutOfMemoryError: Java heap space
[2010-09-17 11:12:39.169] ALL 000000000000 GLOBAL_SCOPE

[2010-09-17 11:12:39.169] ALL 000000000000 GLOBAL_SCOPE java.lang.OutOfMemoryError: Java heap space
[2010-09-17 11:12:39.184] ALL 000000000000 GLOBAL_SCOPE

[2010-09-17 11:12:39.184] ALL 000000000000 GLOBAL_SCOPE java.lang.OutOfMemoryError: Java heap space
[2010-09-17 11:12:39.2] ALL 000000000000 GLOBAL_SCOPE

[2010-09-17 11:12:39.2] ALL 000000000000 GLOBAL_SCOPE java.lang.OutOfMemoryError: Java heap space
[2010-09-17 11:12:39.216] ALL 000000000000 GLOBAL_SCOPE

[2010-09-17 11:13:10.768] ALL 000000000000 GLOBAL_SCOPE MIC value MIC Value Length: 20
00000001 6b 68 6a ce 67 ad 13 db 19 53 fb f6 19 6b 75 54 khj?g?.?.S??.kuT
00000011 4b 76 06 ff Kv.?


[2010-09-17 11:13:10.768] ALL 000000000000 GLOBAL_SCOPE Attempting to use signature provider Certicom
[2010-09-17 11:13:10.768] ALL 000000000000 GLOBAL_SCOPE Calling initSign...
[2010-09-17 11:13:10.768] ALL 000000000000 GLOBAL_SCOPE Adding signature...
[2010-09-17 11:13:13.578] ALL 000000000000 GLOBAL_SCOPE java.lang.OutOfMemoryError: Java heap space
[2010-09-17 11:13:13.578] ALL 000000000000 GLOBAL_SCOPE


Resolving the problem

The main resource to analyze is verbose garbage collection logs. It was noted that when GIS got OutOfMemory errors, there were several large spikes of memory with FULL GCs running back to back. It is important to note the time that the large spikes of memory occurred to go back to all the BPs running at that time to investigate the problem.

After researching this issue, it was noted that there was a BP that was generating a really large process data. In addition that BP had around 30 or more child BPs and all of those BPs would inherit the process data from the parent. Process data is maintained in memory. Once the customer reworked the BP so that process data would not get so large, then the issue stopped occurring.

Historical Number

NFX9968

Product Alias/Synonym

Escalation ID


00224437

Severity


Normal

Type


NormalFix

Document information

More support for: Sterling B2B Integrator

Software version: All

Operating system(s): Platform Independent

Reference #: 1553636

Modified date: 03 November 2010


Translate this page: