IBM Support

PM98613: PROXY SERVER MAY NOT RELEASE RESOURCES WHEN AN ENDPOINT IS UNAVAILABLE

Fixes are available

8.0.0.8: WebSphere Application Server V8.0 Fix Pack 8
8.5.5.2: WebSphere Application Server V8.5.5 Fix Pack 2
8.0.0.9: WebSphere Application Server V8.0 Fix Pack 9
8.5.5.3: WebSphere Application Server V8.5.5 Fix Pack 3
8.5.5.4: WebSphere Application Server V8.5.5 Fix Pack 4
8.0.0.10: WebSphere Application Server V8.0 Fix Pack 10
8.5.5.5: WebSphere Application Server V8.5.5 Fix Pack 5
8.5.5.6: WebSphere Application Server V8.5.5 Fix Pack 6
8.0.0.11: WebSphere Application Server V8.0 Fix Pack 11
8.5.5.7: WebSphere Application Server V8.5.5 Fix Pack 7
7.0.0.39: WebSphere Application Server V7.0 Fix Pack 39
8.5.5.8: WebSphere Application Server V8.5.5 Fix Pack 8
8.0.0.12: WebSphere Application Server V8.0 Fix Pack 12
8.5.5.9: WebSphere Application Server V8.5.5 Fix Pack 9
7.0.0.41: WebSphere Application Server V7.0 Fix Pack 41
8.5.5.10: WebSphere Application Server V8.5.5 Fix Pack 10
8.5.5.11: WebSphere Application Server V8.5.5 Fix Pack 11
8.0.0.13: WebSphere Application Server V8.0 Fix Pack 13
7.0.0.43: WebSphere Application Server V7.0 Fix Pack 43
8.5.5.12: WebSphere Application Server V8.5.5 Fix Pack 12
8.0.0.14: WebSphere Application Server V8.0 Fix Pack 14
8.5.5.13: WebSphere Application Server V8.5.5 Fix Pack 13
7.0.0.45: WebSphere Application Server V7.0 Fix Pack 45
8.0.0.15: WebSphere Application Server V8.0 Fix Pack 15
7.0.0.45: Java SDK 1.6 SR16 FP60 Cumulative Fix for WebSphere Application Server
7.0.0.39: Java SDK 1.6 SR16 FP7 Cumulative Fix for WebSphere Application Server
7.0.0.41: Java SDK 1.6 SR16 FP20 Cumulative Fix for WebSphere Application Server
7.0.0.43: Java SDK 1.6 SR16 FP41 Cumulative Fix for WebSphere Application Server
8.5.5.14: WebSphere Application Server V8.5.5 Fix Pack 14

Subscribe

You can track all active APARs for this component.

APAR status

  • Closed as program error.

Error description

  • When an endpoint is unavailable, workload management (WLM)
    tells the proxy server to suspend  the request and it
    schedules a thread to resume the request when endpoint data is
    available.
    .
    WLM does not check the return code to see if the thread
    was scheduled successfully.  If the thread is not scheduled
    because the thread pool is full, the request won't be
    resumed which could lead to a loss of resources, such
    as the incoming socket.
    .
    The following message appears in the log.
    .
    WSVR0629I: The request buffer for thread pool "Default" has
    reached its capacity.
    .
    The following FFDC appears.
    .
    com.ibm.ws.util.ThreadPool$ThreadPoolQueueIsFullException
    at com.ibm.ws.util.ThreadPool.execute(ThreadPool.java:1386)
    at com.ibm.ws.util.ThreadPool.execute(ThreadPool.java:1171)
    at
    com.ibm.ws.runtime.WSThreadPool.execute(WSThreadPool.java:93)
    at
    com.ibm.ws.proxy.filter.ucf.UCFHttpRouterFilter.preInvoke(UCFHtt
    pRouterFilter.java:819)
    at
    com.ibm.ws.proxy.filter.ucf.UCFHttpRouterFilter.doFilter(UCFHttp
    RouterFilter.java:536)
    at
    com.ibm.ws.proxy.filter.http.HttpFilterImpl.doFilter(HttpFilterI
    mpl.java:98)
    at
    com.ibm.ws.proxy.filter.http.HttpFilterChain.doRequestFilterChai
    n(HttpFilterChain.java:343)
    at
    com.ibm.ws.proxy.filter.http.HttpFilterChain.doRequestFilterChai
    nWrapper(HttpFilterChain.java:263)
    at
    com.ibm.ws.proxy.filter.http.HttpFilterChain.doRequestFilterChai
    n(HttpFilterChain.java:242)
    at
    com.ibm.ws.proxy.channel.http.HttpProxyConnectionLink.processReq
    uestWork(HttpProxyConnectionLink.java:567)
    at
    com.ibm.ws.proxy.channel.http.HttpProxyConnectionLink.ready(Http
    ProxyConnectionLink.java:330)
    at
    com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscr
    imination(HttpInboundLink.java:453)
    at
    ..
    ..
    

Local fix

  • na
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED:  All users of IBM WebSphere Application      *
    *                  Server Proxy Server                         *
    ****************************************************************
    * PROBLEM DESCRIPTION: WLM running in a proxy server may       *
    *                      encounter ThreadPoolFullException       *
    *                      when an endpoint is unavailable,        *
    *                      however WLM does not handle the         *
    *                      ThreadPoolFullException properly        *
    *                      which may result in the loss of         *
    *                      resources, such as an incoming socket.  *
    ****************************************************************
    * RECOMMENDATION:                                              *
    ****************************************************************
    When an endpoint is unavailable, workload management (WLM)
    tells the proxy server to suspend the request and it
    schedules a thread to resume the request when endpoint data is
    available. However, WLM does not check the return code to see
    if the thread was scheduled successfully. If the thread is not
    scheduled because the thread pool is full, the request won't
    be resumed which could lead to a loss of resources, such as
    the incoming socket.
    The following message appears in the log.
    WSVR0629I: The request buffer for thread pool "Default" has
    reached its capacity.
    The following FFDC appears.
    com.ibm.ws.util.ThreadPool$ThreadPoolQueueIsFullException
    at com.ibm.ws.util.ThreadPool.execute(ThreadPool.java:1386)
    at com.ibm.ws.util.ThreadPool.execute(ThreadPool.java:1171)
    at
    com.ibm.ws.runtime.WSThreadPool.execute(WSThreadPool.java:93)
    at
    com.ibm.ws.proxy.filter.ucf.UCFHttpRouterFilter.preInvoke(UCFHtt
    pRouterFilter.java:819)
    at
    com.ibm.ws.proxy.filter.ucf.UCFHttpRouterFilter.doFilter(UCFHttp
    RouterFilter.java:536)
    at
    com.ibm.ws.proxy.filter.http.HttpFilterImpl.doFilter(HttpFilterI
    mpl.java:98)
    at
    com.ibm.ws.proxy.filter.http.HttpFilterChain.doRequestFilterChai
    n(HttpFilterChain.java:343)
    at
    com.ibm.ws.proxy.filter.http.HttpFilterChain.doRequestFilterChai
    nWrapper(HttpFilterChain.java:263)
    at
    com.ibm.ws.proxy.filter.http.HttpFilterChain.doRequestFilterChai
    n(HttpFilterChain.java:242)
    at
    com.ibm.ws.proxy.channel.http.HttpProxyConnectionLink.processReq
    uestWork(HttpProxyConnectionLink.java:567)
    at
    com.ibm.ws.proxy.channel.http.HttpProxyConnectionLink.ready(Http
    ProxyConnectionLink.java:330)
    at
    com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscr
    imination(HttpInboundLink.java:453)
    at
    com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRe
    quest(HttpInboundLink.java:515)
    at
    com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequ
    est(HttpInboundLink.java:306)
    at
    com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpI
    nboundLink.java:277)
    at
    com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.sen
    dToDiscriminators(NewConnectionInitialReadCallback.java:214)
    at
    com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.com
    plete(NewConnectionInitialReadCallback.java:113)
    at
    com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureComp
    leted(AioReadCompletionListener.java:175)
    at
    com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyn
    cFuture.java:217)
    at
    com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncC
    hannelFuture.java:161)
    at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
    at
    com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
    at
    com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHand
    ler.java:816)
    at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
    at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1700)
    

Problem conclusion

  • WLM has been corrected to properly handle the
    ThreadPoolFullException. If a thread cannot be scheduled
    because the thread pool is full, WLM will return an http 503
    to the proxy server. This is considered normal and expected
    behavior in this case. You should tune thread pool and
    other resources as needed.
    
    The fix for this APAR is currently targeted for inclusion in
    fix packs 7.0.0.39, 8.0.0.8 and 8.5.5.2.  Please refer to the
    Recommended Updates page for delivery information:
    http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980
    

Temporary fix

Comments

APAR Information

  • APAR number

    PM98613

  • Reported component name

    WEBS APP SERV N

  • Reported component ID

    5724H8800

  • Reported release

    800

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2013-10-07

  • Closed date

    2013-10-24

  • Last modified date

    2015-11-04

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    WEBS APP SERV N

  • Fixed component ID

    5724H8800

Applicable component levels

  • R800 PSY

       UP

  • R850 PSY

       UP



Document information

More support for: WebSphere Application Server
General

Software version: 8.0

Reference #: PM98613

Modified date: 04 November 2015