CGI script causes memory issue

Writing data to a HttpServletResponse OutputStream causes an OutOfMemory error

The default response data transfer mode of the WebContainer is asynchronous. It must acquire enough allocation of native memory buffers to hold the entire response data through the class java.nio.DirectByteBuffers. In case the application response is slow and compounded by a higher user load, it would occupy and hold more native memory usage until the transfer is complete.

The property "com.ibm.ws.webcontainer.channelwritetype" allows the WebContainer to send the response data in synchronous mode. The response data is written synchronously to the response buffer in chunks and flushes the data to the client when it is fully filled up. This will reduce the native memory used to store the response data, but in some cases this can also reduce overall response performance.

For more information, please refer to link in the 'Related information' section below.

The expected exception should be as follows:

0000002e ServletWrappe E SRVE0068E: Uncaught exception thrown in one of the service methods of the servlet: CGI. Exception thrown : java.lang.OutOfMemoryError: Unable to allocate 8192 bytes of direct memory after 5 retries
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:197)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:303)
at com.ibm.ws.buffermgmt.impl.WsByteBufferPoolManagerImpl.allocateBufferDirect(WsByteBufferPoolManagerImpl.java:654)
at com.ibm.ws.buffermgmt.impl.WsByteBufferPoolManagerImpl.allocateCommon(WsByteBufferPoolManagerImpl.java:568)
at com.ibm.ws.buffermgmt.impl.WsByteBufferPoolManagerImpl.allocateDirect(WsByteBufferPoolManagerImpl.java:504)
at com.ibm.ws.webcontainer.channel.WCCByteBufferOutputStream.getNewByteBuffer(WCCByteBufferOutputStream.java:434)
at com.ibm.ws.webcontainer.channel.WCCByteBufferOutputStream.checkWriteArray(WCCByteBufferOutputStream.java:344)
at com.ibm.ws.webcontainer.channel.WCCByteBufferOutputStream.write(WCCByteBufferOutputStream.java:119)
at com.ibm.ws.webcontainer.srt.SRTOutputStream.write(SRTOutputStream.java:96)
at sun.nio.cs.StreamEncoder$ConverterSE.implFlushBuffer(StreamEncoder.java:285)
at sun.nio.cs.StreamEncoder$ConverterSE.implFlush(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:202)
at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:236)
at com.ibm.wsspi.webcontainer.util.BufferedWriter.writeOut(BufferedWriter.java:486)
at com.ibm.wsspi.webcontainer.util.BufferedWriter.flushChars(BufferedWriter.java:372)
at com.ibm.wsspi.webcontainer.util.BufferedWriter.flush(BufferedWriter.java:347)
at java.io.PrintWriter.flush(PrintWriter.java:291)
at com.micromuse.wave.servlets.CGIServlet.processNonBinaryData(CGIServlet.java:362)
at com.micromuse.wave.servlets.CGIServlet.runCgi(CGIServlet.java:252)
at com.micromuse.wave.servlets.CGIServlet.service(CGIServlet.java:166)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1096)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1037)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:145)
at com.ibm.tivoli.nam.security.tip.TIPRoleManagerFilter.doFilter(TIPRoleManagerFilter.java:53)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:190)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:130)
at com.micromuse.wave.servlets.SessionPrepFilter.doFilter(SessionPrepFilter.java:150)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:190)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:130)
at com.micromuse.wave.servlets.FeatureCheckFilter.doFilter(FeatureCheckFilter.java:109)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:190)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:130)
at com.ibm.tivoli.ncw.servlet.filter.LocalePrepFilter.doFilter(LocalePrepFilter.java:67)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:190)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:130)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain._doFilter(WebAppFilterChain.java:87)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:832)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:679)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:566)
at com.ibm.ws.wswebcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:478)
at com.ibm.ws.webcontainer.webapp.WebApp.handleRequest(WebApp.java:3444)
at com.ibm.ws.webcontainer.webapp.WebGroup.handleRequest(WebGroup.java:267)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:815)
at com.ibm.ws.wswebcontainer.WebContainer.handleRequest(WebContainer.java:1466)
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:119)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:458)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewInformation(HttpInboundLink.java:387)
at com.ibm.ws.http.channel.inbound.impl.HttpICLReadCallback.complete(HttpICLReadCallback.java:102)
at com.ibm.ws.ssl.channel.impl.SSLReadServiceContext$SSLReadCompletedCallback.complete(SSLReadServiceContext.java:1818)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:165)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:136)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:195)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:743)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:873)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1473)
Caused by: java.lang.OutOfMemoryError
at sun.misc.Unsafe.allocateMemory(Native Method)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:184)
... 57 more

1) Apply the following jacl script to set the com.ibm.ws.webcontainer.channelwritetype = sync

setChannelWriteType.jacl


2) Execute the command below:

./wsadmin.sh -username tipadmin -password netcool -lang jacl -f ~/setChannelWriteType.jacl

You may be seeing the following message after the execution above is completed:-

WASX7209I: Connected to process "server1" on node TIPNode using SOAP connector; The type of process is: UnManagedProcess


3) Verify that the following file located at $TIPHOME/profiles/TIPProfile/config/cells/TIPCell/nodes/TIPNode/servers/server1/server.xml should contain the following new change:-
<properties xmi:id="Property_1315899081013" name="com.ibm.ws.webcontainer.channelwritetype" value="sync"/>

OR

$TIPHOME/profiles/TIPProfile/logs/wsadmin.traceout with the following line:-
[9/5/11 16:03:47:912 GMT+05:30] 0000000a AbstractShell A WASX7091I: Executing script: "/tmp/setChannelWriteType.jacl"


Note
To optimize client side CGI scripts, it is advisable to avoid transfer large response data such as PDF, large image, large file, etc.