JR53314: CALLING THE TASK QUERY ENTITY LIST COUNT REST API CAUSES AN OUTOFMEMORYERROR ON LARGE DATA SETS

8.0.1.3-WS-BPM-IFJR53314
8.0.1.2-WS-BPM-IFJR53314
Version 8.5 Refresh Pack 7 for the IBM Business Process Manager products

APAR status

• Closed as program error.

Error description

• When you use the "Task Query Entity List Count" REST API with a
  saved search that returns a large result set with tens of
  thousands or more results, you receive an OutOfMemoryError:
  
  CWTBG0019E: Unexpected exception during execution.
  
  UOWManager transaction processing failed; nested exception is
  com.ibm.wsspi.uow.UOWException: java.lang.OutOfMemoryError: Java
  heap space
  
  bulk operations the customer was advised to use the REST APIs:
  
  /rest/bpm/wle/v1/tasks/query/IBM.PI_TASKLIST_ALL/count
  
  for a count of the tasks.
  
    If there are a large number of task instances, then an
  OutOfMemoryError may occur.
  
  The following SQL query is run as part of this processing:
  
  [4/17/15 16:19:37:040 CEST] 00000090 wle_search    1 com.
  lombardisoftware.server.core.pse.impl.ProcessSearchEngineImpl
  executeSearchQuery executing search query:
  
  select i.instance_name as instanceName, t.bpd_instance_id as
    instanceId, bpd.name as bpdName,
    i.due_date as instanceDueDate,
    i.create_datetime as instanceCreateDate,
    i.last_modified_datetime as instanceModifyDate,
    istatus.name as instanceStatus,
    iprj.short_name as instanceProcessApp,
    tuser.user_name as assignedToUser,
    trole.group_name as assignedToRole,
    tstatus.name as taskStatus,
    t.subject as taskSubject,
    t.activity_name as taskActivityName,
    t.due_date as taskDueDate,
    tpriority.name as taskPriority,
    t.rcvd_datetime as taskReceivedDate,
    t.close_datetime as taskClosedDate,
    t.sent_datetime as taskSentTime,
    t.read_datetime as taskReadTime,
    treceivedUser.user_name as taskReceivedFrom,
    tclosedUser.user_name as taskClosedBy,
    t.task_id as taskId,
    t.attached_form_ref as taskAttachedInfoPathFormRef,
    t.attached_ext_activity_ref as taskAttachedExtActivityRef,
    cbb.bpd_id as taskCreatedByBpdId,
    t.created_by_bpd_flow_object_id as
       taskCreatedByBpdFlowObjectId,
    tpriority.ranking as taskPriorityRanking
  from lsw_task t
  left join lsw_bpd_instance i on t.bpd_instance_id =
    i.bpd_instance_id
  left join lsw_task_status_codes tstatus on t.status = tstatus.
    status_value
  left join lsw_bpd_status_codes istatus on i.execution_status =
    istatus.status_id
  left join lsw_priority tpriority on t.priority_id = tpriority.
    priority_id
  left join lsw_bpd bpd on i.cached_bpd_version_id =
    bpd.version_id
  left join lsw_usr_xref tuser on t.user_id = tuser.user_id
  left join lsw_usr_grp_xref trole on t.group_id = trole.group_id
  left join lsw_usr_xref treceivedUser on t.rcvd_from =
    treceivedUser.user_id
  left join lsw_usr_xref tclosedUser on t.close_by =
    tclosedUser.user_id
  left join lsw_bpd cbb on t.cached_cbb_version_id =
    cbb.version_id
  left join lsw_snapshot isnap on i.snapshot_id =
    isnap.snapshot_id
  left join lsw_project iprj on isnap.project_id = iprj.project_id
  order by taskDueDate, taskPriorityRanking, instanceId, taskId
  with ur
  
  .------------------------------------------------------------
     If one analyzes the generated heapdump or core, there will be
  a large number of java.util.TreeMap objects in an array like
  this.
  .
  java.lang.Object[3771757] @ 0x1b7e7eae8                       |
    15,087,040 | 4,004,103,952
  |- <class> class java.lang.Object[] @ 0x1000006e8 System Class|
  96 |           112
  |- [2198409] java.util.TreeMap @ 0x104505bf0                  |
  48 |         1,096
  |- [1654225] java.util.TreeMap @ 0x106c2df80                  |
  48 |         1,136
  |- [1654264] java.util.TreeMap @ 0x106fa4598                  |
  48 |         1,096
  |- [2198408] java.util.TreeMap @ 0x1070c17d8                  |
  48 |         1,096
  |- [2198407] java.util.TreeMap @ 0x1070c1808                  |
  48 |         1,096
  |- [2198406] java.util.TreeMap @ 0x1070c1838                  |
  48 |         1,096
  |- [2198405] java.util.TreeMap @ 0x1070c1868                  |
  48 |         1,096
  |- [2198404] java.util.TreeMap @ 0x1070c1898                  |
  48 |         1,136
  |- [2198403] java.util.TreeMap @ 0x1070c18c8                  |
  48 |         1,136
  |- [2198402] java.util.TreeMap @ 0x1070c18f8                  |
  48 |         1,096
  |- [2198401] java.util.TreeMap @ 0x1070c1928                  |
  48 |         1,136
  |- [2198400] java.util.TreeMap @ 0x1070c1958                  |
  48 |         1,096
  |- [2198399] java.util.TreeMap @ 0x1070c1988                  |
  48 |         1,136
  |- [2198398] java.util.TreeMap @ 0x1070c19b8                  |
  48 |         1,136
  |- [2198397] java.util.TreeMap @ 0x1070c19e8                  |
  48 |         1,096
  |- [2198396] java.util.TreeMap @ 0x1070c1a18                  |
  48 |         1,136
  |- [2198395] java.util.TreeMap @ 0x1070c1a48                  |
  48 |         1,136
  |- [2198394] java.util.TreeMap @ 0x1070c1a78                  |
  48 |         1,128
  |- [2198393] java.util.TreeMap @ 0x1070c1aa8                  |
  48 |         1,104
  |- [2198392] java.util.TreeMap @ 0x1070c1ad8                  |
  48 |         1,136
  |- [2198391] java.util.TreeMap @ 0x1070c1b08                  |
  48 |         1,096
  |- [2198390] java.util.TreeMap @ 0x1070c1b38                  |
  48 |         1,136
  |- [2198389] java.util.TreeMap @ 0x1075fb1c0                  |
  48 |         1,096
  |- [2198388] java.util.TreeMap @ 0x1075fb1f0                  |
  48 |         1,136
  '- Total: 25 of 3,573,798 entries; 3,573,773 more             |
  |
  

Local fix

Problem summary

• The "Task Query Entity List Count" REST API uses a saved search
  to run a query on the database, returning the results from the
  database in an array list that contains a map of results.  Then
  the count is calculated by iterating over the whole result set.
  The OutOfMemory exception is already thrown when the saved
  search tries to return the large data set before starting to
  count.
  

Problem conclusion

• A fix is/will be available for IBM BPM V8.0.1.2 that prevents
  the OutOfMemory exception by directly running a count query on
  the database so that it does not return all the results.
  
  On Fix Central (http://www.ibm.com/support/fixcentral), search
  for JR53314:
  
  1. Select IBM Business Process Manager with your edition from
    the product selector, the installed version to the fix pack
    level, and your platform, and then click Continue.
  
  2. Select APAR or SPR, enter JR53314, and click Continue.
  
  When you download fix packages, ensure that you also download
  the readme file for each fix. Review each readme file for
  additional installation instructions and information about the
  fix.
  

Temporary fix

• Not applicable
  

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  BPM STANDARD

• Fixed component ID

  5725C9500

Applicable component levels

• R801 PSY

     UP