Using the pelog tool in place of vwlog with FileNet workflows

Content

I. Overview

The pelog tool is an enhanced version of the vwlog tool. The vwlog tool is no longer supported; however, some of the vwlog command options can still be used with the pelog command. These earlier vwlog options are described later in this document.

Following are some pelog tool features:

• The pelog tool improves the performance of pruning of event logs by doing bulk deletion of rows in the event log. The improved pruning process can handle large event logs in a high load environment.
• The pelog command options are simpler and more intuitive.
• During the pruning process, progress is displayed periodically with totals of event log records and tracker records deleted. A summary report is displayed when the pruning task is complete.
• Some of the earlier vwlog tool options are supported by the pelog command.

II. Using the pelog command

The pelog command prunes event log records and tracker work items for the workflow system event logs.

The pelog command is case-sensitive in a UNIX environment. That is, in a UNIX environment, you must type pelog.

However, the pelog command is case-insensitive in the Microsoft Windows environment. That is, in the Microsoft Windows environment, you can enter PELog, pelog, or any other case variation.

The pelog options are case-sensitive for either operating system environment as specified in this document and in the online help.

If the workflow system runtime option Enable CBE adapter for WebSphere® Business Monitor is enabled or if Case Analyzer is enabled, then only the event records that were processed by the BPM Common Base Event (CBE) adapter and Case Analyzer are pruned. Incompletely processed records are not deleted. However, this restriction can be overridden with the -P option. The -P option causes the specified event log records to be pruned regardless if the BPM Common Base Event (CBE) adapter or Case Analyzer has finished processing the records.

The pelog command options

The pelog command has several options. The -Terminated and -TimeOnly options are the most commonly used options.

For more information on the pelog command options, refer to this topic in the P8 documentation: https://www.ibm.com/docs/en/filenet-p8-platform/5.5.x?topic=tools-pelog

The -Terminated option:

• Prunes event log records associated with completed workflows only. To specify event log records to save, indicate the number of days from the present day to save. When the number of days is specified, pelog calculates the time range by determining the start and end times. You no longer specify a time range, such as with the earlier vwlog –s, -S, –e, and -E options.
• Prunes tracker items for completed workflows within the same time range that is specified for pruning event logs. Tracker items associated with active workflows are not affected.

  The -Terminated option is equivalent to the vwlog –L –T –D options.

The -TimeOnly option:

Prunes event log records within the specified time range regardless of whether the workflows have completed. Prunes tracker items for completed workflows within the specified time range. Tracker items associated with active workflows are not affected.

The -TimeOnly option is equivalent to the vwlog –L –D options.

Examples of the -Terminated and -TimeOnly options

The -Terminated and -TimeOnly options are simpler than the earlier vwlog command options because you no longer have to determine the start and stop dates and times. Simply specify the number of days from the present day that need to be kept.

The following pelog command prunes event log records and tracker records for terminated workflows only, keeping 90 days of event log records. Because a batch size is not specified, the batch size defaults to 10000. The diagram illustrates the time range in which event log records are pruned.

pelog <connection_point> -Terminated 90

When Case Analyzer is not configured or the –P option is used

* ‘oldest table time’ is the oldest time that exists in the event log database table.

Note: Previously, vwlog would scan for all terminated events from the A-C time range (that is, scan the entire event log), and then prune events from the C-B time range that belong to all terminated workflows. The pelog tool scans for terminated workflows only in the C-B time range, and then prunes only the events that belong to terminated workflows in that C-B time range.

When Case Analyzer is configured and the –P option is not used

Case Analyzer processed events passed the time B.

* ‘oldest table time’ is the oldest time that exists in the event log database table.

When Case Analyzer is configured and the –P option is not used
Case Analyzer has processed events up to the time C.

* ‘oldest table time’ is the oldest time that exists in the event log database table.

Use the pelog command and options to do the following:

• Delete older event log records and tracker records for terminated workflows only, keeping the most recent 90 days of event log records, with a batch size of 50000.

  pelog <connection_point> -Terminated 90 -b 50000

• Delete all event log records and tracker records for terminated workflows, with a batch size of 50,000. Specify the value ‘0’ to indicate that there are 0 (zero) number of days of events to keep. Specify 0 (zero) for small event log files. For large event log files, specify a greater number of days to keep, such as 100 days. Then, incrementally decrease the number of days to keep in subsequent runs, such as 70, then 30, and then 0 days. By incrementally decreasing the number of days to keep, the pruning performance is not degraded.

  pelog <connection_point> -Terminated 0 -b 50000

• Delete all event log records and tracker records for terminated workflows, with a batch size of 50,000. The –P option ignores whether Case Analyzer has processed the events or not. Use this option with caution if Case Analyzer is in use.

  pelog <connection_point> -Terminated 0 -b 50000 -P

• Delete event log records older than 30 days and delete tracker records that are associated with terminated workflows older than 30 days. Because the batch size is not specified, the batch size defaults to 10000.

  pelog <connection_point> -TimeOnly 30

When Case Analyzer is not configured or the –P option is not used.

* ‘oldest table time’ is the oldest time that exists in the event log database table.

When Case Analyzer is configured and Case Analyzer has processed events that have passed the time C

* ‘oldest table time’ is the oldest time that exists in the event log database table.

When Case Analyzer is configured and Case Analyzer has processed events up to the time C

* ‘oldest table time’ is the oldest time that exists in the event log database table.

Use the pelog command and options to do the following:

• Delete event log records older than 30 days and delete tracker records associated with terminated workflows older than 30 days, with a batch size of 30,000.

  pelog <connection_point> -TimeOnly 30 -b 30000

• Delete all event log records and delete tracker records associated with terminated workflows, with a batch size of 30,000.

  pelog <connection_point> -TimeOnly 0 -b 30000

• To combine the new options with –P (ignore Case Analyzer configuration), -t (specify the working event table) or –v (for verbose), use the following command to delete all event log records of LoadTestLog table and delete tracker records associated with terminated workflows, with a batch size of 30,000. The -t option is used to specify the name of the event log table. The -P option causes Case Analyzer configuration to be ignored. The -v option displays the progress of the pruning process.

  pelog <connection_point> -TimeOnly 0 -b 30000 –t LoadTestLog –P -v

• The previous command is used, but the progress information is redirected to the text file PELogProgress.out in the c:\temp directory. The summary information is appended to the end of the text file when the pruning task is done.

  pelog <connection_point> -TimeOnly 0 -b 30000 –t LoadTestLog –P –v >> c:\temp\PELogProgress.out

III. Supported earlier vwlog options

While the new options Terminated and TimeOnly simplify the pruning process, there might be cases when Tracker items and events must be pruned separately, such as in special cases or as directed by engineering. The following legacy options can be used to serve these specific instances.

NOTE: These legacy options cannot be used in combination with either the -Terminated or -TimeOnly options.

The following are the earlier vwlog parameters and options that are supported by the pelog tool.

Examples of the pelog command with earlier vwlog options

Use the pelog command and earlier vwlog options to do the following:

• Use the -D option to delete tracker records associated with terminated workflows. In the case, use the advanced property PELOG_TRACKER_BATCH_SIZE to change the batch size for the tracker section during tracker pruning. The recommended value for the PELOG_TRACKER_BATCH_SIZE property is within the range 500 to 1000. If you do not specify the -b option, the default batch size for tracker pruning is 10,000, which is not efficient for tracker pruning. Because tracker records are in the VWRoster and Tracker queue, pruning the tracker records does not reduce the size of the event logs.

  pelog <connection_point> -D

• To delete all event log records associated with terminated workflows with a batch size of 30,000. For a large event log, this command is not recommended. If the –t option is not specified, the default event log is DefaultEventLog.

  pelog <connection_point> -L –T -b 30000

When Case Analyzer is not configured

* ‘oldest table time’ is the oldest time that exists in the event log database table.

When Case Analyzer is configured and Case Analyzer has processed events up to time B

* ‘oldest table time’ is the oldest time that exists in the event log database table.

Use the pelog command and earlier vwlog options to do the following:

• To delete all event log records associated with terminated workflows for the event log UPIEventLog with a batch size of 30,000. For a large event log, this command is not recommended.

  pelog <connection_point> -L –T –t UPIEventLog -b 30000

• To delete event log records and tracker records for terminated workflows only, with a batch size of 30,000. For a large event log, this command is not recommended.

  pelog <connection_point> -L –T -D -b 30000

• To delete event log records and tracker records for terminated workflows, beginning from start time to end time, with a batch size of 30,000.

  pelog <connection_point> -L –T -D –s <start time> –e <end time> -b 30000

  The start time and end time should be in the format of "dd/mm/yyyy 00:00:00". If specified, the start time should be always the date/time where the first event gets started from the event log. You should not specify a start time as the pelog tool determines the start time to cover the first event in the event log.

In the case where Case Analyzer is not configured.

Note: Previously, vwlog would scan for all terminated events from the A-C time range (that is, scan the entire event log), and then prune events from C-B time range that belong to all terminated workflows. The pelog tool scans for terminated workflows only in the C-B time range, and then prunes only the events that belong to terminated workflows in that C-B time range.

In the case where Case Analyzer is configured and Case Analyzer has processed events passed the time B:

In the case where Case Analyzer is configured and Case Analyzer has processed events up to the time C:

• To delete event log records and tracker records for terminated workflows, beginning from the first event in the event log to specified end time, with a batch size of 30,000. You should not specify a start time as the pelog tool will determine the start time to cover the first event in the event log.

  pelog <connection_point> -L –T -D –e <end time> -b 30000

• To delete event log records and tracker records for terminated workflows, beginning from the start time to the end time, with a batch size of 30,000. The start time –S is the number of hours from the current time. The end time –E is the number of hours from the current time.

  pelog <connection_point> -L –T -D –S <start time> –E <end time> -b 30000

• To delete event log records and tracker records for terminated workflows, beginning from the start time to the end time, with a batch size of 30,000. The end time –E is the number of hours from the current time. If you do not specify the –S option, the start time defaults to the time when the first event gets started from the event log.

  pelog <connection_point> -L –T -D –E <end time> -b 30000

• To delete all event log records and tracker records for terminated workflows, with a batch size of 30,000. The -P option causes Case Analyzer configuration to be ignored during pruning. Use this option with caution if Case Analyzer is in use.

  pelog <connection_point> -L –T -D -P -b 30000

• To prune all event log records regardless whether the workflows were completed or not with a batch size of 30,000. At the same time, tracker items are pruned from completed workflows. Tracker items belong to active workflows are not touched. Do not use this option if Case Analyzer is in use.

  pelog <connection_point> -L –D –E 240 –b 30000

In the case where Case Analyzer is configured and the –P option is not used.

* ‘oldest table time’ is the oldest time that exists in the event log database table.
 

In the case where Case Analyzer is configured and Case Analyzer has processed events passed the time B.

* ‘oldest table time’ is the oldest time that exists in the event log database table.

In the case where Case Analyzer is configured and Case Analyzer has processed events up to the time D.

* ‘oldest table time’ is the oldest time that exists in the event log database table.

• To prune all event log records regardless whether the workflows were completed or not with a batch size of 30,000. Tracker items are pruned from completed workflows. Tracker items that belong to active workflows are not touched. The -P option causes Case Analyzer configuration to be ignored during pruning. Use this option with caution if Case Analyzer is in use.

  pelog <connection_point> -L -D -P -b 30000

  IV. Progress and summary messages that are displayed while pelog is running

  The pruning progress is displayed for tracking purposes. The progress messages inform you about the state of the pruning progress and the pruning rate. A summary report at the completion of the pruning process is displayed. Use the -v (verbose) option for detailed progress messages and summary. Only a summary of the pruning progress is provided if the -v option is not specified.

  For detailed examples of the pruning progress messages, see Appendix A – Examples of progress and summary messages.

  V. Troubleshooting

  The vwtool command asyncrpc shows information about the latest pelog commands that were run and whether the status of the process has completed or not. However, the actual pelog command that was issued is not shown. You get that level of information with the –v option.

  For example:

  <vwtool:10>asyncrpc
  List/Clear/Abort/Attach (CR=List All):

  ID Status Start Time Last Update Tasks/Regions
  ====================================================================================================
  1601 Completed 2012/10/31 15:31:37 2012/10/31 15:31:38 PELog/n/a
  1602 Completed 2012/10/31 15:35:46 2012/10/31 15:35:48 PELog/n/a
  1603 Completed 2012/10/31 17:53:29 2012/10/31 17:53:30 PELog/n/a
  1604 Completed 2012/10/31 17:53:52 2012/10/31 17:53:53 PELog/n/a

  VI. Recommendations and restrictions

  1. Even though it appears to work for some cases, hitting CTRL-C is not intended to abort a running task.  In that case the last on-going batch deletion will be completed. Refer to the progress and summary messages that are displayed when the -v (verbose) option is specified. You can see at which phase the pruning process was terminated. If you run the same pelog command again, the pruning task resumes from the batch where the pruning stopped.
  2. The new advanced property for the pelog tool is:

     PELOG_TRACKER_BATCH_SIZE (int type).

     This property is used to change the batch size for the tracker section during tracker pruning. By tuning the value of PELOG_TRACKER_BATCH_SIZE you might gain some performance improvement during pruning the tracker objects (-D option, or with -Terminated or -TimeOnly). This advance property can be used in addition to the -b option to tune performance when deleting tracker items. If this advanced property is not set, then pelog will prune both tracker items and events by using the -b option.

     When the PELOG_TRACKER_BATCH_SIZE property and the -b option are not assigned, the batch size defaults to 10,000. When the PELOG_TRACKER_BATCH_SIZE property and the -b option are specified, the PELOG_TRACKER_BATCH_SIZE property setting affects the tracker deletion, while the -b option affects the event deletion.

     Through the Process Engine Process Task Manager, Advanced tab, enter the following in the property name, and an integer for the property value:

     PELOG_TRACKER_BATCH_SIZE

     Setting the property value to a smaller number could result in better performance when deleting tracker items.

     Start with the value of 500 for the PELOG_TRACKER_BATCH_SIZE property, and then gradually increase that value. If this new advance property is not assigned, it defaults to the –b option setting.

  3. The legacy pruning parameters (options) with the individual options are still supported, which allows greater flexibility in special cases. However, there is also a greater possibility of encountering problems depending on the combination of the options.
  4. When using these -L -T -D options to delete all trackers and log records associated with terminated workflows. If the performance of this operation is unacceptable, consider dividing this single operation to 2 steps. That is, run the -D option to delete tracker records before running the -L -T options.
  5. Don't specify a start time when pruning terminated workflows. When using the -L -T -D options together with -s -e to delete all trackers and log records associated with terminated workflows within specific time range, it is best not to specify the start time value (-s or –S). With no start time specified, pruning begins with the oldest event log record. No Start time specification ensures that the pelog tool prunes all event log records from the beginning of the event log in the database table. If -s and -e option is specified, only events in that range are pruned including the terminated event 165. Part of events older than -s (start time) will not be removed, and that may result in some events left over without the terminated event.
  6. Here is an example of how to use the –v (verbose) option to redirect the pelog output to a file for later reference.

     pelog pecp1 -TimeOnly 3 -Y <UserID>+<password> -v -b 30000 > c:\pelog.txt

     The following example shows that the pelog options are parsed and shown with the –v option.

     PELog: (null, 72)
Connecting to cmintel8d:8080/peengine/api/petoolsapi
PELog: -- connection point name: pecp1, associated reg# 1, (null, 72)
--------------------------------------------------------------------------------
PELogTask id 3
.
2012/12/18 16:50:37.248-0800 f_sw.451SQLDS: Task Started.
Neither CA Store or CBEAdaptor is configured

Starting -- Log table processing!
parameters: LogTable=null, StartTime=-1500000000, EndTime=1355619037, RegionId=1, BatchSize=30000
verbose=true, purgeLog=true, onlyTerminate=false, deleteTracker=true, ignorePA: =true

  7. The pelog tool does not restore unused table space after pruning. Reclaiming unused table space in database after large deletion is a DBA administrative task. The following articles may be helpful for an informative purpose. Users should seek help from their DBA.

     DB2 - Steps to reclaim all available space from a reclaimable storage DMS automatic storage tablespace

     https://www.ibm.com/support/pages/steps-reclaim-all-available-space-reclaimable-storage-dms-automatic-storage-tablespace

     Oracle - Reclaiming Unused Space in Data Files.

     https://oracle-base.com/articles/misc/reclaiming-unused-space

     SQL Server - Database console command CLEANTABLE

     https://docs.microsoft.com/en-us/sql/t-sql/database-console-commands/dbcc-cleantable-transact-sql?view=sql-server-ver15

  8. The -TimeOnly option will not work once Custom Tasks are enabled.  When users enable Custom Tasks in IBM Case Manager, they turn on ad hoc transfer -- so-called dynamic workflow transfer and its correlated dynamic workspaces. The number of the ad hoc transfers may become very large (as its name, ad hoc, implies, each may correlate to a one-time activity in a case). Those dynamic workspaces need to be cleaned. The pelog tool with -Terminated option is able to clean those dynamical workspaces safely by finding 165 terminated events. It's not safe to clean those dynamic workspaces while their Case work objects are alive and still referencing their own dynamic workspaces. That is the reason why users are being forced to use the -Terminated option instead of the -TimeOnly option once the Custom Tasks are enabled.

     For users having the Custom Tasks enabled and having existing dynamic workspaces (for instance, ws.dyn.author and ws.dyn.runtime with a suffix ID greater than or equal to 100,000,000) they can try the -Terminated option to clean them up provided that the work objects have terminated event 165 associated with them.

     If a work objects terminated event is missing from the event log, and their correlated work objects were terminated or no longer in use, then the only choice is to manually delete those ws.dyn.author and ws.dyn.runtime entries in the VWObject<regionid> table. We do not recommend this option unless clients have no other choice.

  9. The pruning task can have a heavy impact on input/output activity, particularly when the table is large. Avoid executing the tool when the server is under high load. It's best to run at night, off hours, or when there is less activity on the system.
  10. Using the batch size setting (-b option), the pruning task uses memory and TEMP table space. Truncating the table uses the transaction log, but that's not the case with pelog pruning. Transaction log can be configured to grow automatically on demand, but it relies on available disk space. Managing disk space is a Database Administrator responsibility, it is beyond the scope of pelog.

  VII. Appendix A - Examples of progress and summary messages

  The pelog pruning progress messages are in the following format:

  PELog output section meanings (all Phases within time range):
  Phase 1: event log pruning for bulk deletes non-terminator event log records
  Phase 2: event log pruning for tracker deletes
  Phase 3: event log pruning for bulk deletes of terminator event log records
  Phase 4: event log pruning for bulk deletes of event log records where workflownumber = 0x0

  Example 1:

  pelog pecp1 -TimeOnly 3 -Y <UserID>+<password> -v -b 30000 > c:\pelog.txt

  The output of pelog.txt shows the progress of the pelog execution and the summary when it's done.

  PELog: (null, 72)
Connecting to cmintel8d:8080/peengine/api/petoolsapi
PELog: -- connection point name: pecp1, associated reg# 1, (null, 72)
--------------------------------------------------------------------------------
PELogTask id 3
.
2012/12/18 16:50:37.248-0800 f_sw.451SQLDS: Task Started.
Neither CA Store or CBEAdaptor is configured

Starting -- Log table processing!
parameters: LogTable=null, StartTime=-1500000000, EndTime=1355619037, RegionId=1, BatchSize=30000
verbose=true, purgeLog=true, onlyTerminate=false, deleteTracker=true, ignorePA: =true

PELog output section meanings (all Phases within time range):
Phase 1: event log pruning for bulk deletes non-terminator event log records
Phase 2: event log pruning for tracker deletes
Phase 3: event log pruning for bulk deletes of terminator event log records
Phase 4: event log pruning for bulk deletes of event log records where workflownumber = 0x0

2012/12/18 16:50:37.451-0800 PELog: starting Phase 1 bulk delete.
.........
2012/12/18 16:51:02.404-0800 PELog: Phase 1 batch delete = 30000, total deleted phase 1 = 30000

Phase 1: Batch delete average rate = 1200 records/sec

.......
2012/12/18 16:51:23.264-0800 PELog: Phase 1 batch delete = 30000, total deleted phase 1 = 60000

Phase 1: Batch delete average rate = 1428 records/sec

.......
2012/12/18 16:51:40.936-0800 PELog: Phase 1 batch delete = 30000, total deleted phase 1 = 90000

Phase 1: Batch delete average rate = 1764 records/sec

.......
2012/12/18 16:52:00.576-0800 PELog: Phase 1 batch delete = 30000, total deleted phase 1 = 120000

Phase 1: Batch delete average rate = 1578 records/sec

......
2012/12/18 16:52:15.936-0800 PELog: Phase 1 batch delete = 19007, total deleted phase 1 = 139007

Phase 1: Batch delete average rate = 1267 records/sec

..........
2012/12/18 16:52:48.701-0800 PELog: Phase 1 batch delete = 0, total deleted phase 1 = 139007

Phase 1: final average delete rate = 1061 records/sec

2012/12/18 16:52:48.733-0800 PELog: starting Phase 2 tracker delete
.......
2012/12/18 16:53:08.733-0800 PELog: Phase 2 batch -- deleted 437 trackers for 477 terminated workflows.
2012/12/18 16:53:08.764-0800 PELog: Phase 2 batch -- deleted 437 trackers for 477 terminated workflows.
2012/12/18 16:53:08.811-0800 PELog: completed deleting 437 tracker records for 477 terminated workflows.

Processed terminated workflow final average rate = 23 records/sec

2012/12/18 16:53:08.873-0800 PELog: starting Phase 3 bulk delete.
....
2012/12/18 16:53:16.014-0800 PELog: Phase 3 batch delete = 477, total deleted = 477

Phase 3 batch delete rate = 59 records/sec

.......
2012/12/18 16:53:33.233-0800 PELog: Phase 3 batch delete = 0, total deleted = 477

Phase 3 final average rate = 18 records/sec


SUMMARY:
Deleted record counts: Phase 1: 139007, Phase 2: 477, Phase 3: 477, Phase 4: 0

Time in seconds totals: Phase 1: 131, Phase 2: 20, Phase 3: 26, Phase 4: 0


Overall bulk delete rate (Phases 1,3,4): 888 records per sec,

Overall tracker delete rate (Phase 2): 23/records per sec,

Total duration: 0 hours, 2 minutes, 178 seconds

2012/12/18 16:53:35.186-0800: Finish PELog Process.
Task 3 completed
PELog: done

  Example 2:

  pelog <pe-connectionpoint> -Terminated 3 -Y <UserID>+<password> -v -b 10000 > c:\pelog.txt

PELog: (null, 0)
Connecting to cmintel16b:9080/peengine/api/petoolsapi
PELog: -- connection point name: cmintel16b_3, associated reg# 3, (null, 0)
--------------------------------------------------------------------------------
PELogTask id 719
..
2013/01/09 14:57:13.142-0800 OSDBUSER.DESIGNOSDS: Task Started.
Neither CA Store or CBEAdaptor is configured

Starting -- Log table processing!
parameters: LogTable=null, StartTime=-1500000000, EndTime=1357772233, RegionId=3, BatchSize=10000
verbose=true, purgeLog=true, onlyTerminate=true, deleteTracker=true, ignorePA: =true

PELog output section meanings (all Phases within time range):
Phase 1: event log pruning for bulk deletes non-terminator event log records
Phase 2: event log pruning for tracker deletes
Phase 3: event log pruning for bulk deletes of terminator event log records
Phase 4: event log pruning for bulk deletes of event log records where workflownumber = 0x0

2013/01/09 14:57:13.220-0800 PELog: starting Phase 1 bulk delete.
.....
2013/01/09 14:57:24.185-0800 PELog: Phase 1 batch delete = 10000, total deleted phase 1 = 10000

Phase 1: Batch delete average rate = 909 records/sec

......
2013/01/09 14:57:36.322-0800 PELog: Phase 1 batch delete = 10000, total deleted phase 1 = 20000

Phase 1: Batch delete average rate = 833 records/sec

.....
2013/01/09 14:57:47.146-0800 PELog: Phase 1 batch delete = 10000, total deleted phase 1 = 30000

Phase 1: Batch delete average rate = 909 records/sec

....
2013/01/09 14:57:57.065-0800 PELog: Phase 1 batch delete = 10000, total deleted phase 1 = 40000

Phase 1: Batch delete average rate = 1000 records/sec

.....
2013/01/09 14:58:07.890-0800 PELog: Phase 1 batch delete = 10000, total deleted phase 1 = 50000

Phase 1: Batch delete average rate = 1000 records/sec

......
2013/01/09 14:58:18.808-0800 PELog: Phase 1 batch delete = 10000, total deleted phase 1 = 60000

Phase 1: Batch delete average rate = 909 records/sec

.....
2013/01/09 14:58:30.242-0800 PELog: Phase 1 batch delete = 10000, total deleted phase 1 = 70000

Phase 1: Batch delete average rate = 833 records/sec

.....
2013/01/09 14:58:40.738-0800 PELog: Phase 1 batch delete = 10000, total deleted phase 1 = 80000
….

2013/01/09 15:03:00.826-0800 PELog: Phase 1 batch delete = 10000, total deleted phase 1 = 500000

Phase 1: Batch delete average rate = 5000 records/sec

..
2013/01/09 15:03:03.966-0800 PELog: Phase 1 batch delete = 10000, total deleted phase 1 = 510000

Phase 1: Batch delete average rate = 3333 records/sec

..
2013/01/09 15:03:06.856-0800 PELog: Phase 1 batch delete = 10000, total deleted phase 1 = 520000

Phase 1: Batch delete average rate = 5000 records/sec

..
2013/01/09 15:03:10.198-0800 PELog: Phase 1 batch delete = 9950, total deleted phase 1 = 529950

Phase 1: Batch delete average rate = 2487 records/sec

.
2013/01/09 15:03:11.463-0800 PELog: Phase 1 batch delete = 0, total deleted phase 1 = 529950

Phase 1: final average delete rate = 1480 records/sec

2013/01/09 15:03:11.495-0800 PELog: starting Phase 2 tracker delete
.............
2013/01/09 15:03:58.761-0800 PELog: Phase 2 batch -- deleted 10000 trackers for 10000 terminated workflows.
.....
2013/01/09 15:04:09.273-0800 PELog: Phase 2 batch -- deleted 13482 trackers for 13482 terminated workflows.
2013/01/09 15:04:09.304-0800 PELog: Phase 2 batch -- deleted 13482 trackers for 13482 terminated workflows.
2013/01/09 15:04:09.320-0800 PELog: completed deleting 13482 tracker records for 13482 terminated workflows.

Processed terminated workflow final average rate = 232 records/sec

2013/01/09 15:04:09.335-0800 PELog: starting Phase 3 bulk delete.
....
2013/01/09 15:04:16.677-0800 PELog: Phase 3 batch delete = 10000, total deleted = 10000

Phase 3 batch delete rate = 1428 records/sec

2013/01/09 15:04:18.098-0800 PELog: Phase 3 batch delete = 3482, total deleted = 13482

Phase 3 batch delete rate = 1741 records/sec

2013/01/09 15:04:18.129-0800 PELog: Phase 3 batch delete = 0, total deleted = 13482

Phase 3 final average rate = 1498 records/sec

2013/01/09 15:04:18.129-0800 PELog: starting Phase 4
2013/01/09 15:04:18.145-0800 PELog: Phase 4 batch delete = 0, total deleted = 0


SUMMARY:
Deleted record counts: Phase 1: 529950, Phase 2: 13482, Phase 3: 13482, Phase 4: 0

Time in seconds totals: Phase 1: 358, Phase 2: 58, Phase 3: 9, Phase 4: 1


Overall bulk delete rate (Phases 1,3,4): 1476 records per sec,

Overall tracker delete rate (Phase 2): 232/records per sec,

Total duration: 0 hours, 7 minutes, 425 seconds

2013/01/09 15:04:18.207-0800: Finish PELog Process.
Task 719 completed
PELog: done
  
   

  Example 3: Without the –v (verbose) option, only the SUMMARY is shown
  pelog cmintel16b_3 -L -e 1353222455 -Y <UserID>+<password>

log4j:WARN No appenders could be found for logger (filenet.vw.server).
log4j:WARN Please initialize the log4j system properly.
Connecting to cmintel16b:9080/peengine/api/petoolsapi
--------------------------------------------------------------------------------
PELogTask id 201
..
2013/01/23 13:28:15.049-0800 OSDBUSER.DESIGNOSDS: Task Started.
Neither CA Store or CBEAdaptor is configured

Starting -- Log table processing!
2013/01/23 13:28:15.128-0800 PELog: starting Phase 1 bulk delete.
...
Phase 1: final average delete rate = 7632 records/sec


SUMMARY:
Deleted record counts: Phase 1: 38160, Phase 2: 0, Phase 3: 0, Phase 4: 0

Time in seconds totals: Phase 1: 5, Phase 2: 0, Phase 3: 0, Phase 4: 0


Overall bulk delete rate (Phases 1,3,4): 7632 records per sec,

Total duration: 0 hours, 0 minutes, 5 seconds

2013/01/23 13:28:20.423-0800: Finish PELog Process.
Task 201 completed

  Example 4: With the –v (verbose) option, detailed pruning progress is displayed.

  pelog cmintel16b_3 -L -e 1353222555 -v -Y <UserID>+<password>

  Starting -- Log table processing!
parameters: LogTable=null, StartTime=-1500000000, EndTime=1353222555, RegionId=3, BatchSize=10000
verbose=true, purgeLog=true, onlyTerminate=false, deleteTracker=false, ignorePA: =true

  PELog output section meanings:

  Phase 1: event log pruning for bulk deletes of all event log records

  2013/01/23 13:31:55.064-0800 PELog: starting Phase 1 bulk delete.
.
2013/01/23 13:31:56.891-0800 PELog: Phase 1 batch delete = 10000, total deleted phase 1 = 10000

Phase 1: Batch delete average rate = 10000 records/sec

.
2013/01/23 13:31:58.266-0800 PELog: Phase 1 batch delete = 10000, total deleted phase 1 = 20000

Phase 1: Batch delete average rate = 5000 records/sec

.
2013/01/23 13:31:59.375-0800 PELog: Phase 1 batch delete = 10000, total deleted phase 1 = 30000

Phase 1: Batch delete average rate = 10000 records/sec

2013/01/23 13:32:00.328-0800 PELog: Phase 1 batch delete = 10000, total deleted phase 1 = 40000

Phase 1: Batch delete average rate = 10000 records/sec

..
2013/01/23 13:32:02.421-0800 PELog: Phase 1 batch delete = 10000, total deleted phase 1 = 50000

Phase 1: Batch delete average rate = 5000 records/sec

2013/01/23 13:32:02.624-0800 PELog: Phase 1 batch delete = 1828, total deleted phase 1 = 51828

Phase 1: Batch delete average rate = 1828 records/sec

2013/01/23 13:32:02.702-0800 PELog: Phase 1 batch delete = 0, total deleted phase 1 = 51828

Phase 1: final average delete rate = 7404 records/sec


SUMMARY:
Deleted record counts: Phase 1: 51828, Phase 2: 0, Phase 3: 0, Phase 4: 0

Time in seconds totals: Phase 1: 7, Phase 2: 0, Phase 3: 0, Phase 4: 0


Overall bulk delete rate (Phases 1,3,4): 7404 records per sec,

Total duration: 0 hours, 0 minutes, 7 seconds

2013/01/23 13:32:02.749-0800: Finish PELog Process.
Task 202 completed
PELog: done

  The vwtool/async output shows the Task 201 and Task 202 status as “Completed”:

  <vwtool:3>async
List/Clear/Abort/Attach (CR=List All):

ID Status Start Time Last Update Tasks/Regions Client

201 Completed 2013/01/23 13:28:12 2013/01/23 13:28:21 PELog/n/a Started by pwtestadmin on 9.39.61.110.
202 Completed 2013/01/23 13:31:53 2013/01/23 13:32:03 PELog/n/a Started by pwtestadmin on 9.39.61.110.

  VIII. Appendix B – The pelog online help

  The pelog tool online help provides some basic command option information. You can find some basic examples available in the usage output.

  There are two versions of the help usage output:

  • The h option displays the current description of the pelog options
  • The H option displays the description for the legacy options

  The pelog –h provides the new usage.

  Usage: pelog <connection_point>
       [-h | -H]
       [-O <OutPut File Name> ]
       [-Terminated <# of days> ]
       [-TimeOnly <# of days> ]
       [-t <Table Name> ]
       [-b <Batch Size> ]
       [-P ]
       [-v ]
       [-C ]
       [-N <tenantName>]
       [-Y<username>+<password>]

    Where:
    <connection_point>  Connection point name.  The connection point name is a
                      required property.
    -h                Help. Displays these PELog properties and terminates.
    -H                Help menu for legacy options. Legacy pruning options cannot
                      be combined with current pruning options.
    -O                Output file name.
    -N <tenantName>   Tenant domain to work in.
    -Terminated <number of days>
                      Prunes 'eligible' event log records associated with Terminated
                      Workflows only and prunes Tracker items for terminated
                      workflows older than the number of days.There is no default
                      number of days.Entering 0 (zero)number of days, will process
                      the whole event log table.
                      Note: if CA|CH|CPE is enabled, only records that are already
                      processed by CA|CH|CPE will be pruned. See -P for overriding.
    -TimeOnly <number of days>
                      Prunes 'eligible' event log records based on time. There
                      is no default number of days. Entering 0 (zero)number of
                      days, will process the whole event log table. This option
                      can impact Tracker view of running workflows as event records
                      are deleted by time.
                      Note: if CA|CH|CPE is enabled, only records that are already
                      processed by CA|CH|CPE will be pruned. See -P for overriding.
    -t <Table Name>   Name of event log to process. Default='DefaultEventLog'
    -b <Batch Size>   Batch. Number of records processed per transaction.
                      Default is 10,000.  Minimun batch size is 1000.
    -P                Ignore Case Analyzer|Case History| CBE configuration.
                      This option will force the pruning of log records
                      NOT collected by Case Analyzer or Case History or
                       CBE.
    -v                Verbose. Provides verbose output to stdout.
    -C                Generate an encrypted token for the clear text user name
                       and password specified with -Y.
    -Y <username>+<password>|token Login using either a clear text user name/pwd
                      or an encrypted token generated with -C. Depending on the shell
                      you use to run pelog, the encrypted token might need to be
                      enclosed within either a single or double quote. If the token
                      contains the quoted character, you would also need to escape
                      the internal character.

  Examples:
  To prune event log records and tracker records for terminated workflows only,
  for records older than 90 days, with a batch size 50,000.
    pelog <connection_point> -Terminated 90 -b 50000

    To prune event log records and tracker records for terminated workflows only,
  for records older than 0 days from now. With a batch size 50,000. This will
  process and prune all event logs in the event log table.
    pelog <connection_point> -Terminated 0 -b 50000

    To prune all event log records older than 30 days plus delete tracker records
  associated with terminated workflows older than 30 days. With a batch 50,000.
    pelog <connection_point> -TimeOnly 30 -b 50000

    To generate an encrypted token for user/pwd to use with -Y later. Note: this
  user needs to be a member of PE Administrator group.
    pelog <conection_point> -Y <user>+<pwd> -C

    To run pelog with an encrypted token on the bash shell (default for zLinux),
  you need to enclose the encrypted token with quote and escape any internal
  quote within the token.
  For example: if the encrypted token is !xyz'123, passed to pelog this value
  as '!xyz\'123'. Use \ for escape character.
    pelog <conection_point> -Y '!xyz\'123' -Terminated 90 -b 50000

  The pelog –H provides the legacy usage.

  Usage: pelog <connection_point>
       [-h | -H ]
       [-O <OutPut File Name> ]
       [-L [-T] [-D] [-P]] | [-D]
       [-s <Start Time> | -S <Hours> ]
       [-e <End Time> | -E <Hours> ]
       [-t <Table Name> ]
       [-b <Batch Size> ]
       [-v ]
       [-Y<username>+<password> ]
    Where:
    <connection_point>  Connection point name.  The connection point name is a required property.
    -h                Help. Displays non-Legacy current PELog properties and terminates.
    -H                Help menu for legacy options. Legacy pruning options
                      can not be combined with current pruning options.
    -O                Output file name.
    -L                Log. Prunes event log records.
                      If the Case Analyzer option has been enabled,
                      then the -L parameter will only prune
                      records that have been processed by the Case Analyzer.
                      All records will be pruned regardless of the Case
                      Analyzer option if the Case Analyzer has not been
                      configured for the event log (the Case Analyzer may be
                      configured to only process records from a subset of the
                      event logs).
    -T                Terminated Workflows only. Prune only log records that are
                      associated with terminated workflows.
    -D                Delete tracker items. This option will only delete tracker records
                      associated with terminated workflows.
                      Prerequisite: This operation requires the existence of Workflow
                      Termninated event record (event type 165 record).
    -P                Ignore Case Analyzer configuration. This option will
                      force the deletion of log records regardless if Case Analyzer is configured or not.
    Time range properties -- if either start or end time is not provided then:
                           the start time range will be set to -1500000000 (06/20/1922 13:20:00) and/or
                           the end time range will be set to the current time.
    -s <Start Time>   start. Provide the exact start time with either time number
                      (# seconds since 1970) or time string format matches
                      the system date/time mask.
    -S <Hours>        Start. Provide the start time as the number of hours from the current time.
    -e <End  Time>    end. Provide the exact end time in either time number
                      (# seconds since 1970) or time string format matches
                      the system date/time mask.
    -E <Hours>        End. Provide the end time as hours from the current time.
      -t <Table Name>   Name of event log to process.  Default = 'DefaultEventLog'
    -b <Batch Size>   Batch. Number of records processed per transaction.
                      Default is 10,000.  Minimun batch size is 1000.
    -v                Verbose. Provides verbose output to stdout.
    -Y <username>+<password> Login using the user name and password.
  Examples:
    pelog <connection_point> -D: within the provided time range,
                                 delete all tracker records associated with terminated workflows.
    pelog <connection_point> -D -L: within the provided time range,
                                    delete all tracker records associated with terminated workflows
                                    and also delete all event log records.
    pelog <connection_point> -L -T: delete all log records associated with terminated workflows.
    pelog <connection_point> -L -T -D: delete all trackers records and log records
                                       associated with terminated workflows.
  Recommendation:
Use -L -T -D to delete all trackers and log records associated with terminated workflows. If
the performance of this operation is unacceptable, you might consider breaking this single operation
to 2 steps: run -D to delete tracker records before running -L -T.