Runtime changes of note for WebSphere Batch in WebSphere V8.5.5.2

Answer

Using database sequence generator to create job number

This is a new and optional feature in WebSphere Application Server v8.5.5.2. It provides a streamlined process for the Job Scheduler to obtain a job number. It is aimed at reducing the complexity at runtime, especially when job numbers roll over and are reclaimed.


Enable this feature in WebSphere Application Server v8.5.5.2 in 2 steps:

Note:

• Step 1 is not required if the Batch Job Scheduler database is set up using the DDL from the WebSphere Application Server v8.5.5.2 file system. If the Batch Job Scheduler database already existed and the runtime is updated to fix pack 8.5.5.2, step 1 is required.
• Using “START WITH=0” and “MAXVALUE=999999” will result in the jobId starting at 0 and the max jobId set to 999999
• To have the jobId continue from the last jobId before the upgrade, set “START WITH” to the value of the last jobId.

Step1: Update the batch job scheduler database to create the sequence generator.

If using DB2 on distributed:
Modify file <was_home>/util/Batch/updateSchedulerDBtoUseSequenceDB2.ddl with appropriate values, and run the command below:

db2 -tf updateSchedulerDBtoUseSequenceDB2.ddl

If using DB2 on z/OS:
Modify and run this SQL query:

CREATE SEQUENCE "LRSSCHEMA"."JOBNUMBER"
START WITH 0
INCREMENT BY 1
MINVALUE 0
MAXVALUE 999999
ORDER
CYCLE;

If using Oracle:
Modify and run this SQL query:

CREATE SEQUENCE JOBNUMBER
START WITH 0
INCREMENT BY 1
MINVALUE 0
MAXVALUE 999999
CACHE 20
CYCLE;

There is no support for this feature in Derby.

Step 2: Enable the batch job scheduler custom property “UseSequenceForJobNumber”
To set this custom property, navigate to Job Scheduler > Custom properties
Click on 'new' to create a new Batch custom property and specify
custom property name = UseSequenceForJobNumber
value = true
Click 'Save' to save this setting and restart the server(s) which host the job scheduler.

New Endpoint joblog location and behaviour.

In the previous fixpack, the default location of joblogs generated on the endpoint was
<was_home/joblogs/<server_name>/<section>/<jobId>

For example:
/WebSphere/ND/AppServer3/profiles/default/joblogs/endpoint_ndnode3/section1/XDCGIVT_000007> ls
part.1.log part.2.log part.3.log

If a job was restarted on the same endpoint server, the joblogs of the restarted execution were placed in the same location as the previous execution.
In this example, part.1.log and part.2.log are from the initial execution and part.3.log is from the restarted execution.

In WebSphere Application Server v8.5.5.2, the default behaviour is all endpoint joblogs will be in a similar path as the scheduler joblogs, where each execution will have its joblogs in a separate directory. The new default path is <was_home>/joblogs/<server_name>/<section>/<jobid>/<timestamp>

For example: /WebSphere/ND/AppServer3/profiles/default/joblogs/endpoint_ndnode3/section1/XDCGIVT_000009/20140214_152254

Each job execution will have joblogs written to a new location, and the first joblog name will be part.1.log.

For example, if a job executed the first time, and then went to restartable, upon restart of the job, users will see

/WebSphere/ND/AppServer3/profiles/default/joblogs/endpoint_ndnode3/section1/XDCGIVT_000009> ls
20140214_151921 20140214_152254

/WebSphere/ND/AppServer3/profiles/default/joblogs/endpoint_ndnode3/section1/XDCGIVT_000009> cd 20140214_151921
/WebSphere/ND/AppServer3/profiles/default/joblogs/endpoint_ndnode3/section1/XDCGIVT_000009/20140214_151921> ls
part.1.log part.2.log part.3.log part.4.log

/WebSphere/ND/AppServer3/profiles/default/joblogs/endpoint_ndnode3/section1/XDCGIVT_000009> cd 20140214_152254
/WebSphere/ND/AppServer3/profiles/default/joblogs/endpoint_ndnode3/section1/XDCGIVT_000009/20140214_152254> ls
part.1.log part.11.log part.13.log part.15.log part.17.log part.19.log part.20.log part.22.log part.3.log part.5.log part.7.log part.9.log
part.10.log part.12.log part.14.log part.16.log part.18.log part.2.log part.21.log part.23.log part.4.log part.6.log part.8.log

In WebSphere Application Server v8.5.5.2, the Job Management Console will show the logs from the latest execution. This allows customers to quickly view the logs and determine a restart action.


Prior to bringing up WebSphere Application Server v8.5.5.2, customers can decide to stay with the old behaviour. If that is the case, customers can set a WebSphere variable at the cell scope,
GRID_ENDPOINT_USE_TIMESTAMP_FOR_JOBLOG=false

Once the customer switches to WebSphere Application Server v8.5.5.2 behaviour, it is not possible to switch back to the previous joblog behaviour. Meaning if customers have jobs writing out joblogs using the WebSphere Application Server v8.5.5.2 behaviour and then switch back to the old behaviour, they will not be able to view the joblogs of the WebSphere Application Server v8.5.5.2 jobs correctly.

Introduction of the ability to specify different jobclass values for top level jobs and subjobs in xjcl for Parallel Jobs.

For parallel jobs using WCG 8 xJCL with the “run element”, the top level job will run under the job class that is specified by the "class" attribute of the job element.

To specify a job class for the subjobs or parallel steps, add the "job-class" attribute to the run element of the xJCL. If the "class" attribute of the job element or "job-class" attribute of the run element is not specified, the job will run under the default job class.

Example:
<job name="XDCGIVT" default-application-name="XDCGIVT_cg8" class="TLJ_JobClass" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
...
<run instances="multiple" jvm="multiple" job-class="SubJob_JobClass">

<props>
<prop name="com.ibm.websphere.batch.parallel.parameterizer" value="com.ibm.ws.batch.parallel.BuiltInParameterizer"/>
<prop name="com.ibm.wsspi.batch.parallel.jobs" value="3"/>
<prop name="com.ibm.wsspi.batch.parallel.prop.numberRecords.1" value="100"/>
<prop name="com.ibm.wsspi.batch.parallel.prop.inputDataStream.1" value="/tmp/input-text-sj1.txt" />
....
</props>
...

</job>

New JobClass computation SPI

WebSphere Application Server v8.5.5.2 introduces a new service provider interface to dynamically calculate the batch job class of each job that is submitted to the Job Scheduler. This SPI is enabled through the use of a job scheduler custom property. The given job class interceptor is invoked before job submission and is calculated using the job name, submitter, and original job class that is defined in the job definition (xJCL).

Below are the instructions on how to invoke the Job Class SPI:

1) Extend the com.ibm.wsspi.batch.JobClassInterceptor class.
ex. public class MyJobClassInterceptor extends JobClassInterceptor


2) Implement the one abstract method:
/**
* @param jobName The XJCL name of the job as specified in the name attribute of the job element.
*
* @param submitter The job submitter
*
* @param jobClass The current job associated with the job.
*
* @return The new Job Class to assign to the job.
**/
public abstract String classifyJob(String jobName, String submitter, String jobClass);


3) Package the new class in a jar and place in the $WAS_HOME/classes directory. Create the classes directory if it doesn't already exist. Then define a shared library to point to the jar package.


4) Create the following Job Scheduler custom property, "com.ibm.wsspi.batch.job.class.interceptor", and set the value to the fully qualified name of the new class.
ex.com.ibm.wsspi.batch.job.class.interceptor=com.ibm.example.MyJobClassInterceptor


If the Job scheduler custom property is set and the associated class can be loaded, the custom job class interceptor will be called each time a Job is submitted and job class will be modified to the job class returned from the classifyJob method.

Runtime Update that requires new database tables

When migrating from V8.5.5.0 or 8.5.5.1 to WAS V8.5.5.2, there is a runtime update that requires a new table.

The following DDLs/SPUFIs are provided to assist with the table changes for the various databases that are supported with WebSphere Batch in V8.5.x. These files can be found in the WAS_HOME/util/Batch directory. Attempting to run WebSphere Batch without the necessary table changes will result in failures on the Job Scheduler during job processing and management.

Please note that if you have previously upgraded to WebSphere V8.5.5.1, and subsequently installed the fix 8.5.5.1-WS-WAS-DistOnly-IFPI07496.zip (capacity leak detection/recovery) you do not have to proceed as this update is effected by that fix and will be in place on your system.

DB2 ULW ()
updateSchedulerDBForJobClassRecoveryDB2.ddl
UpdateLocalJobStatusConstraintDB2.ddl

Oracle
updateSchedulerDBForJobClassRecoveryOracle.ddl
UpdateLocalJobStatusConstraintOracle.ddl

Derby
updateSchedulerDBForJobClassRecoveryDerby.ddl


There is no script to update the local job status constraint for Derby. User can drop their existing Derby database and recreate it using the CreateLRSCHEDTableDerby.ddl to take advantage of the constraints update or the user can create their own script that would model the updates found in CreateLRSCHEDTableDerby.ddl

All database updates must be done after the Compute Grid code is updated to the latest fix pack level, but before any application server that uses Compute Grid is started.

The following command can be issued from DB2 command line processor to run script to update DB2 database.

db2 -tf <file_name>.ddl

The following command can be used to update Derby database. Process this script in the ij command line processor.

java -Djava.ext.dirs=was_install_root/derby/lib -Dij.protocol=jdbc:derby:
org.apache.derby.tools.ij file_name.ddl
SQL*Plus can be used to run the script update for Oracle database.