Recovery to point-of-failure

This backup and recovery process is used to recover the Tivoli Workload Scheduler for z/OS environment to point-of-failure. The strategy relies on the dual job-tracking facility. The dual logs must be allocated on DASD in the secondary center connected to the primary center by either channel-to-channel connectors or fiber-optic channel extenders.

Although the backup process is complex, when in place, this setup provides quick recovery of the environment.

Backup requirements

Table 41 defines the backup intervals required to ensure you can successfully restore to point-of-failure. Some data sets need only be backed up weekly; others are required to be backed up after every CP or JS backup. N/A shows that a backup is not applicable for DRP purposes.

Table 41. Backup cycle for point-of-failure DRP
ddname Format Defines Backup interval
EQQADDS VSAM Applications and JCL variables Daily
EQQCKPT PS Checkpoint data set N/A
EQQCP1DS VSAM Current-plan-1 data set N/A
EQQCP2DS VSAM Current-plan-2 data set N/A
EQQCXDS VSAM Current-plan-extension data set N/A
EQQDLnn PS Dual job-tracking-log data set N/A (already in secondary)
EQQEVDS PS Event data set for the event-writer task N/A
EQQEVDnn PS Event data set for an event-reader task N/A
EQQHTTP0 PS Event data set for end-to-end scheduling with z-centric capabilities N/A
EQQINCWK PS JCC incident work file N/A
EQQJBLIB PDS JCL library data set Minimum weekly, daily if high activity
EQQJCLIB PDS JCC message-table data set Minimum weekly, N/A if same as primary
EQQJS1DS VSAM JCL-repository-1 data set See note 1
EQQJS2DS VSAM JCL-repository-2 data set See note 1
EQQJTARC PS Job-tracking-archive data set See note 2
EQQJTnn PS Job-tracking-log data set N/A
EQQLDDS VSAM Work data set for long-term-plan batch jobs N/A
EQQLTBKP VSAM Long-term plan backup See note 3
EQQLTDS VSAM Long-term plan data set See note 3
EQQMLIB PDS Message library N/A (already in secondary)
EQQMLOG PS Message log N/A
EQQNCPDS VSAM New-current-plan data set After every NCP
EQQNCXDS VSAM New-current-plan extension data set After every NCP
EQQOIDS VSAM Operator-instruction database Minimum weekly, daily if high activity
EQQPARM PDS

Initialization-statement-parameter library

Initialization-statement- parameter library

 Minimum weekly, daily if high activity
EQQPRLIB PDS Automatic-recovery-procedure library Minimum weekly, daily if high activity
EQQRDDS VSAM Special resource descriptions Daily
EQQSCLIB PDS Scripts and Commands definition library Minimum weekly, daily if high activity
EQQSIDS VSAM ETT criteria and configuration data Minimum weekly, daily if high activity
EQQSTC PDS Started-task-submit data set N/A
EQQSUDS PS Submit/release data set N/A
EQQTWSCS PDSE End-to-end data set for centralized script support N/A
EQQTWSIN PS Input events for end-to-end scheduling with fault tolerance capabilities N/A
EQQTWSOU PS Output events for end-to-end scheduling with fault tolerance capabilities N/A
EQQWSDS VSAM Workstation, calendar, and period definitions Minimum weekly, daily if high activity
STEPLIB PDS IBM Tivoli Workload Scheduler for z/OS load-module library N/A, already in secondary
user-defined PS Submit/release data set N/A
Notes:
  1. EQQJSnDS: after every JS copy is complete, make a backup of the inactive file. Tivoli Workload Scheduler for z/OS issues message EQQN015I to show when the copy is complete and to identify the inactive JS ddname. You should update the message to include WTO=YES to trigger the backup using NetView®.
  2. EQQJTARC: after every CP backup, the contents of the current job-tracking log are copied to this data set. Take the backup after message EQQN090I is issued to show that the JT data is copied to the archive data set.
  3. When an LTP or daily-planning batch job runs a copy of the LTP is written to the EQQLTBKP data set. Use EQQLTBKP for your DRP backup to ensure that no updates have occurred before the backup is taken. Perform the DRP backup after every NCP.
  4. Tivoli Workload Scheduler for z/OS issues message EQQN057I to show that a CP backup is complete. You should update the message to include WTO=YES so you can use NetView to trigger DRP data set backups. Message EQQN051I shows why the CP backup occurred. The DRP backups can be triggered when the scheduler issues message EQQN057I following message EQQN051I with reason "DP END". In fact, now, EQQLTBKP, EQQNCPDS and EQQNCXDS are all synchronized with the other scheduler data.

Recovery process

Follow the steps listed below to recover your Tivoli Workload Scheduler for z/OS environment to the point-of-failure.

  1. Allocate all required data sets. The JCL required should be based on the sample library (SEQQSAMP) members EQQPCS01 and EQQPCS02.
  2. Data backed up daily or weekly should be recovered from the most recent backup. LTP, NCP, and JTARC should be recovered from the most recent backup.
  3. Restore data backed up at regular intervals. Copy the backup of JS to both EQQJS1DS and EQQJS2DS.
  4. Browse EQQJTARC and obtain the time stamp of the last record. The time stamp starts at decimal location 12 and is in the format 00YYMMDDFHHMMSSTH. Examine EQQDLnn data sets, and identify the files that contain job-tracking records not included in the archive log.
  5. Copy the required EQQDLnn data set to EQQJT01. If more than one dual log contains job-tracking data not included in the archive log, append all the records to EQQJT01 in strict time order.
  6. Specify JOBSUBMIT(NO) and CURRPLAN(NEW) on the JTOPTS statement and then start the controller address space. Start additional address spaces required for the tracker subsystems.
  7. Use the Tivoli Workload Scheduler for z/OS dialogs to delete or complete occurrences that you do not need to process in the secondary center. Check the status of all occurrences and resources before starting job submission.
  8. Change CURRPLAN(NEW) to CURRPLAN(CURRENT) on the JTOPTS statement.

If you are scheduling end-to-end with fault tolerance capabilities, perform the following manual actions to make sure that the Symphony™ file is aligned with the rebuilt current plan:

  1. From Tivoli Workload Scheduler for z/OS dialog select the option 3, DAILY PLANNING. The Producing OPC Daily Plans dialog is displayed.
  2. Select option 5, SYMPHONY RENEW.
  3. Submit the symphony renew batch job to create a Symphony file aligned with the Current® Plan.