IBM Support

JR47818: BUSINESS DATA ALIASES CACHE TAKES A LONG TIME TO BUILD WHICH CAN LEAD TO HUNG THREADS

Subscribe

You can track all active APARs for this component.

APAR status

  • Closed as program error.

Error description

  • The Business data alias cache loads too slowly and is triggered
    at first Portal access. This fix adds different options for this
    cache. This fix also requires JR48884 to avoid issues after
    applying the fix.
    
    Business process definition (BPD) variables that are
    configured as  "available for search"  are given alias and
    are cached in the business data alias cache. The business data
    aliases cache is populated either when the server starts up or
    when you first access the cache, depending on the IBM BPM
    version. Furthermore, the number of BPDs and snapshots in the
    repository affects how long it takes to initialize the cache
    because a large amount of data might be retrieved from the
    database.
    
    The following stack trace snippet appears in the log file when
    the server tries to build the business data alias cache and it
    is taking a long time:
    
    com.ibm.ws.rsadapter.jdbc.WSJdbcResultSet.next
    (WSJdbcResultSet.java:3120)
    at
    com.lombardisoftware.server.ejb.psapi.BusinessDataAliasesBuilder
    .buildBusinessDataAliases(BusinessDataAliasesBuilder.java:119)
    at
    com.lombardisoftware.server.ejb.psapi.
    BusinessDataAliasesCacheHelper.findBusinessDataAliases
    (BusinessDataAliasesCacheHelper.java:13)
    at
    com.lombardisoftware.server.ejb.psapi.BusinessDataAliasesCache.
    loadCacheData(BusinessDataAliasesCache.java:50)
    
    
    While the cache is loading, other server functions must wait
    to access the cache and might show hung threads that include a
    stack trace similar to the following stack trace:
    
    
    at
    org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkD
    ynamicAopProxy.java:204)
    at $Proxy59.findSearchableBusinessDataAliases(Unknown Source)
    at
    com.lombardisoftware.server.ejb.psapi.SearchAPIFacadeBean$19.
    execute(SearchAPIFacadeBean.java:292)
    
    Furthermore, building the cache also blocks server functions
    that try to update BPD models, which affects Process Designer.
    
    .------------------------------------------------------
       Here is a WSVR0605W message that is logged when the problem
    occurs.
    .
    [5/14/13 10:36:01:627 CDT] 00000019 ThreadMonitor W   WSVR0605W:
     Thread "WebContainer : 2" (0000004a) has been active for 694532
     milliseconds and may be hung.  There is/are 1 thread(s) in
    total in the server that may be hung."
        at java.net.SocketInputStream.socketRead0(Native Method)
        at
    java.net.SocketInputStream.read(SocketInputStream.java:140)
        at com.ibm.db2.jcc.t4.z.b(z.java:199)
        at com.ibm.db2.jcc.t4.z.c(z.java:259)
        at com.ibm.db2.jcc.t4.z.a(z.java:755)
        at com.ibm.db2.jcc.t4.cb.Dc(cb.java:2219)
        at com.ibm.db2.jcc.t4.cb.d(cb.java:2210)
        at com.ibm.db2.jcc.t4.db.a(db.java:185)
        at com.ibm.db2.jcc.t4.db.c(db.java:31)
        at com.ibm.db2.jcc.t4.r.a(r.java:32)
        at com.ibm.db2.jcc.t4.j.Zb(j.java:263)
        at com.ibm.db2.jcc.am.vm.X(vm.java:3697)
        at com.ibm.db2.jcc.t4.d.f(d.java:1883)
        at com.ibm.db2.jcc.am.hc.a(hc.java:202)
        at com.ibm.db2.jcc.t4.d.a(d.java:109)
        at com.ibm.db2.jcc.am.vm.c(vm.java:381)
        at com.ibm.db2.jcc.am.vm.next(vm.java:308)
        at
    com.ibm.ws.rsadapter.jdbc.WSJdbcResultSet.next(WSJdbcResultSet.j
    ava:3120)
        at
    com.lombardisoftware.server.ejb.psapi.BusinessDataAliasesBuilder
    .buildBusinessDataAliases(BusinessDataAliasesBuilder.java:119)
        at
    com.lombardisoftware.server.ejb.psapi.BusinessDataAliasesCacheHe
    lper.findBusinessDataAliases(BusinessDataAliasesCacheHelper.java
    :13)
        at
    com.lombardisoftware.server.ejb.psapi.BusinessDataAliasesCache.l
    oadCacheData(BusinessDataAliasesCache.java:50)
        at
    com.lombardisoftware.server.ejb.psapi.BusinessDataAliasesCache.l
    oadCacheData(BusinessDataAliasesCache.java:14)
        at
    com.lombardisoftware.core.cache.GenericCache.getCacheData(Generi
    cCache.java:177)
        at
    com.lombardisoftware.server.ejb.psapi.BusinessDataAliasesCache.g
    etBusinessDataAliases(BusinessDataAliasesCache.java:44)
        at
    com.lombardisoftware.server.ejb.psapi.SearchAPIFacadeCore.findSe
    archableBusinessDataAliases(SearchAPIFacadeCore.java:312)
    ...
    

Local fix

  • Trigger the intial loading of the cache during an offshift time
    so that it does not impact users as often. Depending on the
    version it will be trigger at startup or you can trigger it by
    opening process portal or process admin console.
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED:  IBM BPM Express, Standard, and Advanced     *
    ****************************************************************
    * PROBLEM DESCRIPTION: Searchable business data aliases are    *
    *                      the columns that you can use in saved   *
    *                      searches, the TWSearch API, and the     *
    *                      REST Search API. If you set the         *
    *                      <use-business-aliases-for-process-      *
    *                      instances> property value to "false",   *
    *                      the list of searchable business data    *
    *                      aliases are returned based on           *
    *                      snapshot definitions. Before the        *
    *                      <use-business-aliases-for-process-      *
    *                      insta                                   *
    *                      nces> property was introduced,          *
    *                      business data aliases that were         *
    *                      defined in any snapshot were            *
    *                      available and reloaded after each       *
    *                      snapshot deployment, which slowed       *
    *                      down performance, especially as the     *
    *                      number of snapshots increased, such     *
    *                      as in a Process Center environment      *
    *                      with many developers.                   *
    ****************************************************************
    * RECOMMENDATION:                                              *
    ****************************************************************
    The BusinessDataAliasesCache takes a long time to build, which
    can cause threads to hang.
    

Problem conclusion

  • A fix is available to cache business data aliases based on BPD
    instances. With this fix applied, the business data alias
    cache is built by default when the server starts, which
    improves the loading performance.
    
    The <use-business-aliases-for-process-instances> property is
    set to "true" so that the list of searchable business data
    aliases are returned only when they are used in actual process
    instances. Because business data aliases are loaded as needed,
    performance might slow down as the number of instances
    increases (such as in a high-volume Process Server
    environment), but the number will not climb unnecessarily.
    Business data aliases might not be available immediately after
    instances are created.
    
    On Fix Central (http://www.ibm.com/support/fixcentral), search
    for JR47818:
    
    1. Select the product group, product, installed version, and
    platform, and click Continue.
    2. Select APAR or SPR, enter JR47818, and click Continue.
    
    When downloading fix packages, ensure that you also download
    the readme file for each fix. Review each readme file for
    additional installation instructions and information about the
    fix.
    
    In some circumstances, you might need to change the business
    data alias cache behavior by updating the following
    configuration property values in the 100Custom.xml file:
    
    In stand-alone profiles, go to
    Profile_home\config\cells\<~cellname~>\nodes\<~nodename~>\server
    s\<~servername~>\<~process-center/server~>\config\100Custom.xml
    
    In cluster profiles, go to
    DMGR_home\config\cells\<~cellname~>\nodes\<~nodename~>\servers\<
    ~servername~>\<~process-center/server~>\config\100Custom.xml
    
    The fix shortens the time that is required to load the
    business data alias cache when there are a large number of
    business data aliases because it is loaded only with aliases
    that are defined in BPDs for which there are instances in the
    system. The default value is "true". When the property value
    is set to "false", the business data alias cache is
    searching all BPDs in the system. If there are a large number
    of BPDs, searching all the BPDs can take a very long time.
    
    To build saved searches when there are no BPD instances that
    use the aliases that you want to include in the search, you
    might need to change the value of this property to "false".
    However, doing so will lengthen the time required to build the
    business data alias cache.
    
    The <bdac-refresh-interval>[integer]</bdac-refresh-interval>
    property specifies the length of time between automatic cache
    refreshes. However, it is effective only when the property
    <use-business-aliases-for-process-instances> value is set to
    "true". The default value is 30 minutes (1800 seconds).
    
    You might want to change the cache refresh interval when there
    are a small number of instances in the system or when
    instances start and end before the cache is refreshed. This
    property should have a shorter value in a Process Center than
    in a Process Server because Process Centers tend to have fewer
    BPD instances running and the BPDs change much more frequently.
    
    The <init-bdac-on-startup>[boolean]</init-bdac-on-startup>
    property is effective only when the property
    <use-business-aliases-for-process-instances> value is set to
    "false". This property controls when the cache is initially
    loaded. The default value is "true", which means
    the initialization happens when the server starts up. When the
    value is set to "false", initialization happens the first time
    the cache is accessed, such as when a search is being authored
    or when a saved search is running.
    
    After making the changes to 100Custom.xml in the DMGR profile,
    make sure the changes are carried to the nodes by doing a
    force synchronize and restarting the deployment manager
    (DMGR). Then copy the following XML snippet to 100Custom.xml
    (under <properties> tag):
    
    <server>
     <portal merge="mergeChildren">
      <use-business-aliases-for-process-instances
    merge="replace">true</use-business-aliases-for-process-instances
    >
      <init-bdac-on-startup
    merge="replace">true</init-bdac-on-startup>
     </portal>
     <repository merge="mergeChildren">
      <bdac-refresh-interval
    merge="replace">1800</bdac-refresh-interval>
     </repository>
    </server>
    
    Save the 100Custom.xml file and then open it in a browser to
    make sure that there are no special characters in it.
    

Temporary fix

  • Yes. You can speed the time that is required to build the
    business data alias cache by
    reducing the number of BPD variables that are configured as
     "available for search", removing the unused BPDs, and
    deleting unneeded snapshots.
    

Comments

APAR Information

  • APAR number

    JR47818

  • Reported component name

    BPM ADVANCED

  • Reported component ID

    5725C9400

  • Reported release

    751

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt

  • Submitted date

    2013-09-17

  • Closed date

    2014-02-05

  • Last modified date

    2016-01-04

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    BPM ADVANCED

  • Fixed component ID

    5725C9400

Applicable component levels

  • R750 PSY

       UP

  • R800 PSY

       UP



Document information

More support for: IBM Business Process Manager Advanced

Software version: 7.5.1

Reference #: JR47818

Modified date: 04 January 2016