Managing persisted data

Whenever data is persisted into a database, a time comes when its important to delete or archive some of that data. Periodically cleaning up saves disk space and improves performance.

The data

IBM Business Process Manager retains the following kinds of data. All of them are candidates for regular purging or archiving. The tables provide pointers to more specific guidance.

IBM Business Process Manager Standard
Process Center holds snapshots of process applications and toolkits as they are developed.
Process Server holds snapshots of process applications that have been deployed to it.
Process Server also holds instances of the Business Process Definition (BPD) processes and tasks created in Process Designer.
IBM Document Store is a built-in storage location for document attachments.
The Performance Data Warehouse holds event data collected from tracking groups and auto-tracking.
Durable subscription events for intermediate messages are retained.
Installation messages are stored in the system temporary folder (%temp%).
Some index data is stored in memory indices to support user interactions.
Shared business objects are retained.
IBM Business Process Manager Advanced
In addition to the data mentioned in IBM Business Process Manager Standard, Advanced stores information about service functions.
In addition to instances and tasks created in Process Designer, Process Server Advanced holds instances of BPEL processes and tasks created in Integration Designer.
Process Server also stores information about failed events. You can view that information through the Failed Event Manager.
WebSphere Application Server
System and trace logs are stored during production and when doing additional traces.
WebSphere Transaction Manager saves transaction logs from inflight transactions.
The system also saves transaction logs from databases used by IBM BPM.

Process Center

Table 1. Cleanup activities for Process Center administration
Activity Description Resources
Archiving in Process Center

Archiving a project does not delete it or reclaim its space in the database. Archiving marks the project so that it does not show by default in the Process Center project list. Archiving a project archives all its snapshots and artifacts.

You can archive individual named snapshots instead of archiving an entire project.
Deleting in Process Center

To delete a project, you must first archive it. To actually delete a project, select to show the archived projects; then click Delete beside the project that you want to remove. Deleting a project deletes all snapshots and process instances. This deletion capability is available only from the user interface. There is no scripted way to delete projects.

An unnamed snapshot is created every time you save artifacts in Process Designer. A named snapshot is simply an unnamed snapshot that has been given a name. These unnamed snapshots are saved in the database to provide a project history, but they should be regularly purged. The LSW_PO_VERSIONS table incurs the most growth when unnamed snapshots are not regularly deleted.

You can delete any snapshot manually. You can also configure IBM BPM to delete unnamed snapshots automatically when they are no longer needed.

You can delete snapshots using the BPMSnapshotCleanup wsadmin command. To delete a named snapshot, you must first archive it.

Deleting in Process Center (IBM BPM Advanced)

In addition to the actions described for IBM BPM Standard, you need to be aware of business level applications (BLAs) and enterprise applications that are created on the Process Center playback server. If process applications or toolkits in Process Center contain a module or library, either directly or through a toolkit, a BLA is created for the tip of every process application or toolkit that contains advanced content and for every named snapshot that contains advanced content. This advanced content can quickly accumulate, and that accumulation can affect the time it takes to start the server, memory use, and general performance.

If a snapshot contains Advanced Integration Services (for example, SCA modules or BPEL processes), you can use the Process Center console to undeploy it from the Process Center server.

If you are following the façade pattern, You usually only need one BLA for the most current snapshot of a process application. To delete BLAs, click the Undeploy option on the Snapshots page of Process Center. To delete named snapshots, click Deactivate then Undeploy.
Undeploying a process application or toolkit tip (IBM BPM Advanced)

The tip is the current working version of a process application or toolkit. When you undeploy a tip, all Advanced Integration Service artifacts and the associated business level application (BLA) are removed from the server. The process application definition and other artifacts are still present in the repository.

If you are using the façade pattern and testing the advanced content in an IBM Integration Designer Unit Test Environment (UTE), you almost never need the tip to be deployed. However, since the tip is automatically deployed when changes occur, you need to undeploy the tip if it is not really needed.

Process Server

Table 2. Cleanup activities for Process Server administrators
Activity Description Resources
Deactivating a snapshot

Use the BPMDeactivate command to deactivate a snapshot. Deactivating allows all existing instances to complete processing, but no new requests are processed. In a network deployment environment, you must run this command on the node containing the application cluster member that handles Process Server or Process Center applications. Do not run this command from the deployment manager profile.

You can deactivate a running process application snapshot on the Process Center server or a Process Server. If your process application uses a BPEL process as the main entry component, you must stop the corresponding BPEL template in the WebSphere administrative console. If this BPEL process invokes a BPD, you must allow any existing instances to complete after you stop the template but before you deactivate the snapshot.

Deleting snapshots from IBM BPM Standard on Process Server

Over time, a Process Server can accumulate many snapshots of a process application. You can delete snapshots that are no longer in use. When you delete a snapshot, you also delete any business process definitions that are associated with it.

The snapshot cannot have any running instances. The snapshot cannot be active.

Toolkit snapshots are not automatically deleted when process application snapshots are deleted. Use the BPMShowSnapshot command to obtain a list of toolkit snapshots on which a process application depends. If those toolkit snapshots are not needed by any other process application or toolkit, delete them using the BPMDeleteSnapshot command.
Deleting BPMN process instances When you delete process instances, task instances are also deleted. The BPMProcessInstanceCleanup wsadmin command allows you to delete specific instances or delete instances that completed within a specified date range. BPMProcessInstancesCleanup command
Deleting system tasks when the tasks complete By default, data about system tasks is saved after the task is complete. This data cannot be deleted after it has been saved. If you do not want to store audit data for system tasks on Process Server, select Delete task on completion when you create a system task in Process Designer. Implementing activities in a BPD
Deleting execution data about user tasks when the task completes By default, the runtime execution state of an activity is saved after it is complete. This data cannot be deleted after it has been saved. If you do not want to store execution data (such as variable values) for viewing after the process is complete, select Clean State from the Task Header section of the BPD diagram in Process Designer. Implementing activities in a BPD
Deleting snapshots on Process Server (IBM BPM Advanced)

If a process application contains content from IBM Integration Designer, a business-level application (BLA) with EAR files is created. After the process application is installed on the Process Server, the BLA and its constituent EAR files are deployed on the server. You need to stop the processes from IBM BPM Advanced and undeploy the content before you delete the snapshot.

Deleting BPEL process instances (IBM BPM Advanced)

Use the deleteCompletedProcessInstances.py administrative script to selectively delete from the BPC database or the Business Process Archive database any top-level BPEL process instances that have reached an end state of finished, terminated, or failed.

Deleting BPEL human task instances (IBM BPM Advanced)

Use Business Process Choreographer Explorer to delete human tasks individually. You can also use the Jython scripts deleteCompletedTaskInstances.py or deleteCompletedProcessInstances.py.

Use the cleanup service in the Human Task Manager or Business Flow Manager administrative console pages to schedule jobs to delete task or process instances.

 Cleanup procedures for Business Process Choreographer
Deleting Business Process Choreographer objects Various database objects accumulate in a running system, for example, audit log entries, task and process instances, task and process templates, and people queries. Regularly running administrative scripts to delete objects that are no longer needed from the Business Process Choreographer databases can help prevent wasting storage space. Deleting Business Process Choreographer objects
Deleting groups
  • Security groups come from a configured user repository, such as LDAP. Security groups are never deleted when process instances or tasks are deleted.
  • A team, formerly called a participant group, is associated with a snapshot. An interim fix has been published that ensures that a team is deleted when the snapshot is deleted.
  • A temporary group is formed if you assign a task to a list of users or groups. Temporary groups are deleted from the server automatically when no task uses them any more.
  • Dynamic groups are defined by an expression, such as all users from Canada. The membership of the group can change each time the server is started, Dynamic groups are scoped to a snapshot. An interim fix has been published that ensures that a dynamic group is deleted when the snapshot is deleted.
  • An internal group can contain users from more than one user repository. Internal groups are created and deleted from the Process Admin Console.
Deleting unused ad hoc groups An ad hoc group is shared among tasks that make use of it. Because it can be shared, an ad hoc group is not deleted when the task is deleted. You can clean up unused ad hoc groups in the following ways:
  • Use the Admin Console during task cleanup.
  • Delete unused ad hoc groups while deleting a process instance.
  • Add a step to custom cleanup scripts.
 IBM Business Process Manager default security groups
Deleting durable messages

Durable subscription messages exist and persist in the LSW_DUR_MSG_RECEIVED database table if durable subscriptions are enabled for certain events.

 BPMDeleteDurableMessages command

Performance Data Warehouse

Table 3. Cleanup activities for database administrators
Activity Description Resources
Purging old instances Task and process instances are recorded in the database, even after the task and process have been completed.
Archiving data in the Performance Data Warehouse You can archive snapshots and mark all the metadata in those snapshots with an ARCHIVED time stamp. IBM Business Process Manager does not use archived metadata when it generates the Performance Data Warehouse schema and views. Archiving and restoring data in the Performance Data Warehouse database
Purging data in the Performance Data Warehouse BPD events that are being tracked are saved in the database of the Performance Data Warehouse. Use the prune command to remove data that you no longer need from the Performance Data Warehouse database.

File system

Table 4. Cleanup activities for the temporary directories
Activity Description Resources
Purging the IBM BPM temporary directory During installations and during some other runtime operations, files are placed in the system temporary directory (%temp%). You can delete those files when the installation has completed. The server must be stopped when you delete files from the temp directory.  
Managing WebSphere log files
IBM Websphere Application Server creates temporary files in the following directories:
  • profile root/profile name/config/temp
  • profile root/profile name/temp
  • profile root/profile name/wstemp
You can delete the temp files after stopping the servers on the profile node. You can delete the content of the wstemp directory after shutting down the deployment manager.
Some other WebSphere log files purge data automatically.
  • The SystemOut.log file contains messages about significant events from the JVM process. Log files automatically roll over and purge the oldest data. You can use a background batch process to capture the oldest logs before rollover.
  • Never delete the transaction log. It is self-cleaning.
Parent topic: Administering applications and processes in the runtime environment