IBM Support

PM43581: SLOW MEMORY LEAK OCCURS WHEN USING PMI REQUEST METRICS CONFIGURED IN "DEBUG" MODE

Fixes are available

7.0.0.21: WebSphere Application Server V7.0 Fix Pack 21
8.0.0.2: WebSphere Application Server V8.0 Fix Pack 2
8.0.0.3: WebSphere Application Server V8.0 Fix Pack 3
7.0.0.23: WebSphere Application Server V7.0 Fix Pack 23
8.0.0.4: WebSphere Application Server V8.0 Fix Pack 4
7.0.0.25: WebSphere Application Server V7.0 Fix Pack 25
8.0.0.5: WebSphere Application Server V8.0 Fix Pack 5
7.0.0.27: WebSphere Application Server V7.0 Fix Pack 27
8.0.0.6: WebSphere Application Server V8.0 Fix Pack 6
7.0.0.29: WebSphere Application Server V7.0 Fix Pack 29
8.0.0.7: WebSphere Application Server V8.0 Fix Pack 7
6.1.0.47: WebSphere Application Server V6.1 Fix Pack 47
8.0.0.8: WebSphere Application Server V8.0 Fix Pack 8
7.0.0.31: WebSphere Application Server V7.0 Fix Pack 31
7.0.0.27: Java SDK 1.6 SR13 FP2 Cumulative Fix for WebSphere Application Server
7.0.0.33: WebSphere Application Server V7.0 Fix Pack 33
8.0.0.9: WebSphere Application Server V8.0 Fix Pack 9
7.0.0.35: WebSphere Application Server V7.0 Fix Pack 35
8.0.0.10: WebSphere Application Server V8.0 Fix Pack 10
7.0.0.37: WebSphere Application Server V7.0 Fix Pack 37
8.0.0.11: WebSphere Application Server V8.0 Fix Pack 11
7.0.0.39: WebSphere Application Server V7.0 Fix Pack 39
8.0.0.12: WebSphere Application Server V8.0 Fix Pack 12
7.0.0.41: WebSphere Application Server V7.0 Fix Pack 41
8.0.0.13: WebSphere Application Server V8.0 Fix Pack 13
7.0.0.43: WebSphere Application Server V7.0 Fix Pack 43
8.0.0.14: WebSphere Application Server V8.0 Fix Pack 14
7.0.0.45: WebSphere Application Server V7.0 Fix Pack 45
8.0.0.15: WebSphere Application Server V8.0 Fix Pack 15
6.1.0.41: Java SDK 1.5 SR12 FP5 Cumulative Fix for WebSphere Application Server
6.1.0.43: Java SDK 1.5 SR13 Cumulative Fix for WebSphere Application Server
6.1.0.45: Java SDK 1.5 SR14 Cumulative Fix for WebSphere Application Server
6.1.0.47: Java SDK 1.5 SR16 Cumulative Fix for WebSphere Application Server
7.0.0.21: Java SDK 1.6 SR9 FP2 Cumulative Fix for WebSphere
7.0.0.23: Java SDK 1.6 SR10 FP1 Cumulative Fix for WebSphere
7.0.0.25: Java SDK 1.6 SR11 Cumulative Fix for WebSphere Application Server
7.0.0.27: Java SDK 1.6 SR12 Cumulative Fix for WebSphere Application Server
7.0.0.29: Java SDK 1.6 SR13 FP2 Cumulative Fix for WebSphere Application Server
7.0.0.45: Java SDK 1.6 SR16 FP60 Cumulative Fix for WebSphere Application Server
7.0.0.31: Java SDK 1.6 SR15 Cumulative Fix for WebSphere Application Server
7.0.0.35: Java SDK 1.6 SR16 FP1 Cumulative Fix for WebSphere Application Server
7.0.0.37: Java SDK 1.6 SR16 FP3 Cumulative Fix for WebSphere Application Server
7.0.0.39: Java SDK 1.6 SR16 FP7 Cumulative Fix for WebSphere Application Server
7.0.0.41: Java SDK 1.6 SR16 FP20 Cumulative Fix for WebSphere Application Server
7.0.0.43: Java SDK 1.6 SR16 FP41 Cumulative Fix for WebSphere Application Server

Subscribe

You can track all active APARs for this component.

APAR status

  • Closed as program error.

Error description

  • When WebSphere Application Server is configured in "DEBUG"
    mode for Performance Monitoring Infrastructure (PMI) Request
    Metrics and using Application Response Measurement (ARM) agents
    causes a slow memory leak occurs.
    
    Heapdump may show the following as leak suspect
    
    394,031,304 (71%) [144] 12 array of java/util/HashMap$Entry
    0x3fe20708
    393,843,192 (71%) [32] 2 java/util/HashMap$Entry 0x3fe206e8
    393,842,976 (71%) [72] 7
    com/ibm/ws/runtime/component/ThreadPoolMgrImpl$ThreadPoolProxy
    0x3fe206a0
    393,841,992 (71%) [104] 5 com/ibm/ws/util/ThreadPool
    0x3fe207a8
    393,841,704 (71%) [48] 2 java/util/HashMap 0x3fe20810
    393,841,640 (71%) [80] 1 array of java/util/HashMap$Entry
    0x3fe20840
    393,841,560 (71%) [32] 2 java/util/HashMap$Entry
    0x3f03b040
    393,841,528 (71%) [136] 11
    com/ibm/ws/util/ThreadPool$Worker
    0x3f03ade0
    239,092,776 (43%) [72] 6 java/lang/ThreadGroup
    0x392148a0
    154,676,168 (28%) [232] 14 array of java/lang/Object
    0x3f03b1a8
    154,658,640 (28%) [136] 7
    com/ibm/ws/pmi/reqmetrics/PmiRmThreadCtx 0x3f03b630
    154,640,016 (28%) [32] 1
    com/ibm/ws/pmi/reqmetrics/PmiRmCache
    0x3f03b7f8
    154,639,984 (28%) [24] 1 java/util/ArrayList
    0x3f03b818
    154,639,960 (28%) [1,191,488] 262,754 array of
    java/lang/Object 0x4848c590      <==== Leak 154M (28%)
    
    189,030,200 (34%) [32] 113 class
    com/ibm/ws/pmi/reqmetrics/PmiReqMetricsImpl 0x31b2a9c0
    189,025,384 (34%) [32] 44 class
    com/ibm/ws/pmi/reqmetrics/WebAppReqMetricsListener 0x31b4f0e0
    189,020,664 (34%) [48] 1 java/util/HashMap 0x3cb0e650
    189,020,616 (34%) [272] 20 array of java/util/HashMap$Entry
    0x3e5d39d0
    156,038,696 (28%) [32] 2 java/util/HashMap$Entry 0x3e5d39b0
    156,038,448 (28%) [248] 44
    com/ibm/ws/wswebcontainer/webapp/WebApp
    0x3e1e6268
    143,289,560 (26%) [24] 1 java/util/ArrayList 0x3e1e6c80
    143,289,536 (26%) [56] 1 array of java/lang/Object
    0x3e1e6c98
    143,289,480 (26%) [24] 2
    org/apache/myfaces/orchestra/conversation/servlet/ConversationMa
    nagerSes
    sionListener 0x3e9a3c68
    143,289,456 (26%) [112] 8
    org/apache/myfaces/orchestra/conversation/ConversationWiperThrea
    d
    0x386b1678
    143,289,120 (26%) [16] 1 java/util/HashSet 0x386b3960
    143,289,104 (26%) [48] 2 java/util/HashMap 0x386b3970
    143,289,040 (26%) [262,160] 4,096 array of
    java/util/HashMap$Entry 0x47b2c590  <==== Leak 143M  (26%)
    

Local fix

  • use PERFORMANCE-DEBUG mode instead DEBUG mode
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED:  All users of IBM WebSphere Application      *
    *                  Server versions 6.1, 7.0, and 8.0           *
    ****************************************************************
    * PROBLEM DESCRIPTION: When WebSphere Application Server is    *
    *                      configured in "DEBUG" mode for          *
    *                      Performance Monitoring Infrastructure   *
    *                      (PMI) Request Metrics using ARM         *
    *                      agents, OutOfMemory occurs which leads  *
    *                      to JVM crash.                           *
    ****************************************************************
    * RECOMMENDATION:                                              *
    ****************************************************************
    This is not seen for small number of requests, but as the
    number of request keeps on increasing the number of
    PmiRmCorrelator objects keeps growing and are not released.
    This
    ultimately leads to OutOfMemory and JVM crash.
    

Problem conclusion

  • This problem is fixed by making sure that PmiRMCorrelator
    objects are reused from cache and a new one is only created
    when the cache is empty. This avoids too many PmiRmCorrelator
    objects getting created for every request at a component which
    keeps the number of PMIRMCorrelator objects under control and
    avoids the OutOfMemory and JVM Crash.
    
    The fix for this APAR is currently targeted for inclusion in
    fix packs 6.1.0.41, 7.0.0.21 and 8.0.0.2.  Please refer to the
    Recommended Updates page for delivery information:
    http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980
    

Temporary fix

Comments

APAR Information

  • APAR number

    PM43581

  • Reported component name

    WEBS APP SERV N

  • Reported component ID

    5724H8800

  • Reported release

    61I

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt

  • Submitted date

    2011-07-12

  • Closed date

    2011-08-12

  • Last modified date

    2011-08-12

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    WEBS APP SERV N

  • Fixed component ID

    5724H8800

Applicable component levels

  • R61A PSY

       UP

  • R61H PSY

       UP

  • R61I PSY

       UP

  • R61P PSY

       UP

  • R61S PSY

       UP

  • R61W PSY

       UP

  • R61Z PSY

       UP

  • R700 PSY

       UP

  • R800 PSY

       UP



Document information

More support for: WebSphere Application Server
General

Software version: 6.1

Reference #: PM43581

Modified date: 12 August 2011