JR42522: BusinessDataAliasCache performance and memory leak fix for IBM Business Process Manager

Version 7.5.1 Fix Pack 1 for the IBM Business Process Manager products

APAR status

• Closed as program error.

Error description

• The inital load of the BusinessDataAliasCache, which is an
  internal cache that is associated with portal queries, scales
  poorly as the number of named snapshots in the database
  increases. The performance problem is also associated with
  large amounts of memory being consumed and heavy load on the
  database.
  .
  Heap Analyzer shows 90% of memory with the branch manager.
  .
  One instance of
  "com.lombardisoftware.server.ejb.persistence.versioning.BranchMa
  nager"
  loaded by "com.ibm.oti.vm.BootstrapClassLoader @ 0x40f91a48"
  occupies
  1,944,715,592 (90.52%) bytes. The memory is accumulated in one
  instance
  of "java.util.concurrent.ConcurrentHashMap$Segment[]" loaded by
  "com.ibm.oti.vm.BootstrapClassLoader @ 0x40f91a48".
  

Local fix

• n/a
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:  Users of BPM Express, BPM Standard and BPM  *
  *                  Advanced                                    *
  ****************************************************************
  * PROBLEM DESCRIPTION: The first login to the portal           *
  *                      triggers the initial load of the        *
  *                      cache and can take a long time. It      *
  *                      sometimes results in page time outs     *
  *                      and hung thread stack traces in the     *
  *                      server log files. During this initial   *
  *                      load, there is excessive load on the    *
  *                      database. Users will see stack traces   *
  *                      with content like:                      *
  *                      at                                      *
  *                      com/lombardisoftware/server/ejb/psapi/B *
  *                      usinessDataAliasesCache.getBusinessData *
  *                      Aliases(BusinessDataAliasesCache.java:4 *
  *                      5)                                      *
  *                      at                                      *
  *                      com/lombardisoftware/servlet/util/Cache *
  *                      Manager.init(CacheManager.java:112)     *
  *                      at                                      *
  *                      com/ibm/ws/webcontainer/servlet/Servlet *
  *                      Wrapper.init(ServletWrapper.java:358)   *
  *                      at                                      *
  *                      com/ibm/ws/webcontainer/servlet/Servlet *
  *                      WrapperImpl.init(ServletWrapperImpl.jav *
  *                      a:169)                                  *
  *                      at                                      *
  *                      com/ibm/ws/webcontainer/servlet/Servlet *
  *                      Wrapper.initialize(ServletWrapper.java: *
  *                      1809                                    *
  ****************************************************************
  * RECOMMENDATION:  Recommended IFIX                            *
  ****************************************************************
  The first login to the portal
  triggers the initial load of the
  cache and can take a long time. It
  sometimes results in page time outs
  and hung thread stack traces in the
  server log files. During this initial
  load, there is excessive load on the
  database. Users will see stack traces
  with content like:
  at
  com/lombardisoftware/server/ejb/psapi/B
  usinessDataAliasesCache.getBusinessData
  Aliases(BusinessDataAliasesCache.java:4
  5)
  at
  com/lombardisoftware/servlet/util/Cache
  Manager.init(CacheManager.java:112)
  at
  com/ibm/ws/webcontainer/servlet/Servlet
  Wrapper.init(ServletWrapper.java:358)
  at
  com/ibm/ws/webcontainer/servlet/Servlet
  WrapperImpl.init(ServletWrapperImpl.jav
  a:169)
  at
  com/ibm/ws/webcontainer/servlet/Servlet
  Wrapper.initialize(ServletWrapper.java:
  1809
  On runtime servers with many deployed snapshots, large amounts
  of memory might be consumed, which sometimes leads to out of
  memory errors. Heap dump analysis points to a class named
  BranchManager as a leak suspect.
  

Problem conclusion

• The fix:
  The algorithm for loading and incrementally updating this
  cache has been optimized to dramatically reduce the number of
  queries to the database and the server time needed to process
  the results.
  The fix also addresses the memory leak by avoiding the code
  path that exposed the leak and by putting a configurable bound
  on the number of entries in the BranchManager cache.
  
  On systems with very large number of snapshots and versions of
  business process definitions (BPDs) (especially Process
  Center), the first login to the portal after a server start
  might still have a delay.
  Details available here:
  http://www-304.ibm.com/support/docview.wss?uid=swg27025011
  >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  
  The interim fix is available in fix central for the following
  versions:
  7.5.0.1
  7.5.1
  It is also fixed in future fix packs and releases.
  
  >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  
  In version 7.5.0 Fix Pack 1 (7.5.0.1), this fix is available
  as a part of the JR42581 ifix package.
  JR42581 can be downloaded from Fix Central:
  http://www-933.ibm.com/support/fixcentral/swg/quickorder?parent=
  ibm~WebSphere&product=ibm/WebSphere/IBM+Business+Process+Manager
  +Advanced&release=7.5.0.1&platform=All&function=fixId&fixids=7.5
  .0.1-WS-BPM-IFJR42581&includeSupersedes=0&source=fc
  The installation instruction can be downloaded along with
  JR42581 ifix file.
  
  >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  
  Install/Dependency information on 7.5.1:
  
  Is available in the readme.txt on FixCentral.
  When the user goes to the download files page in FixCentral the
  readme will be available with the fix package as a seperate
  download.
  
  >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
  THIS FIX REQUIRES POST INSTALL XML CONFIGURATION CHANGES.
  PLEASE CONSULT README FOR THE SAME.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  BPM ADVANCED

• Fixed component ID

  5725C9400

Applicable component levels

• R750 PSY

     UP