IBM Support

Recommended DSMSERV RESTORE DB point-in-time procedure

Question & Answer


Question

What are the steps to perform and correct any inconsistencies caused by DSMSERV RESTORE DB to a point-in-time?

Cause

Restore of the IBM Spectrum Protect or Tivoli Storage Manager server database to an earlier point in time can cause the server to encounter data on disk and tapes for which it has no record, or records in its database for which the data in question no longer exists (whether at all or in its original location), resulting in ANR1330E and ANR1331E errors and/or ANR0340E messages for servers using node replication.

Answer

Restoring a server database to a point in time

Make sure you have fully read this document and understand all the steps before proceeding.

To restore a database to point in time, you need the latest full backup before the point in time. You also need the latest incremental backup after that last full backup. You can also use snapshot database backups to restore a database to a specific point in time.

Before you begin


Before restoring the database, have available the following infrastructure setup files:
  • Server options file
  • Volume history file:
  • Device configuration file:
  • Detailed query output about the database and recovery log

Copy the volume history file pointed to by the server options file. The backup copy must be a different name. If the restore fails and you must try it again, you might need the backup copy of the volume history file. After the database is restored, any volume history information pointed to by the server options is lost. This information is required to identify the volumes to be audited.

You might need to modify the device configuration file based on the hardware available at the recovery site. For example, the recovery site might require a different device class, library, and drive definitions.



NOTE: Configurations that may require a manual device configuration file update at a recovery site include the following:
  • If definitions for paths are included when SRCTYPE is set to SERVER
  • If an automated tape library is used at the recovery site
  • If server-to-server virtual volumes are used
For more information review "Updating the device configuration file" for the server version closest to the version in your environment:
V6.3.3 server documentation
V7.1.0 server documentation
If files were migrated, reclaimed, or moved after a backup, the files might be lost and the space occupied by those files might be reused. In the future, you can minimize this loss by using the REUSEDELAY parameter when defining or updating sequential-access storage pools. This parameter delays volumes from being returned to scratch or being reused.

If your old volume history file shows that any of the copy storage pool volumes that are required to restore your storage pools were reused (STGREUSE) or deleted (STGDELETE), you might not be able to restore all your files. In the future you can avoid this problem by including the REUSEDELAY parameter when you define your copy storage pools.



UPDATE STGPOOL note REUsedelay
For more information review "UPDATE STGPOOL" for the server version closest to the version in your environment:
V6.3.3 server documentation
V7.1.0 server documentation
V8.1.0 server documentation

Procedure

To restore the database to a point-in-time, complete the following steps:


1. If the database or recovery log directories were lost, recreate the directories. For example:

AIX operating systemsHP-UX operating systemsLinux operating systemsOracle Solaris operating systems
mkdir /tsmdb001
mkdir /tsmdb002
mkdir /tsmdb003
mkdir /activelog
mkdir /archlog
mkdir /archfaillog


Windows operating systems
mkdir e:\tsm\db001
mkdir f:\tsm\db001
mkdir g:\tsm\db001
mkdir h:\tsm\activelog
mkdir i:\tsm\archlog
mkdir j:\tsm\archfaillog


2. Use the DSMSERV RESTORE DB utility. For example, to restore the database to a backup series that was created on April 19, 2016, enter:
dsmserv restore db todate=04/19/2016

There are some additional parameters that can be used with DSMSERV RESTORE DB. For example if your environment uses encrypted container storage pools (V7.1.3 and higher) in a cloud environment you will want to read and understand the RESTOREKeys and PASSword parameters. Other parameters include the ability to change paths used for database and log information and the ability to preview the operation.


For more information review "DSMSERV RESTORE DB" for the server version closest to the version in your environment:
V6.3.3 server documentation
V7.1.1 server documentation
V7.1.3 server documentation
V8.1.0 server documentation

The server completes the following actions:
a. Reads the volume history file to locate the last full backup that occurred on or before the specified date and time.
b. Using the device configuration file, requests a mount of the first volume. The first volume contains the beginning of the full backup.
c. Restores the backup data from the first volume.
d. Continues to request mounts and to restore data from the backup volumes that contain the full backup and any incremental backups that occurred on or before the specified date.

NOTE: You may want to temporarily disable server maintenance processes and client sessions. This prevents any client/server operations from attempting to access any of the data on the storage pool volumes until the following steps are complete. If you want to do this, set the following options in the dsmserv.opt file before starting the server:
NOMIGRRECL
EXPINTERVAL 0
DISABLESCHEDS YES
V7 Tivoli Storage Manager servers below 7.1.1 do not honor DISABLESCHEDS YES. APAR IC99843
Then run the following command after starting the server:
DISABLE SESSIONS
Be sure to ENABLE SESSIONS and change the options in dsmserv.opt back to their original values (will require a server restart) before resuming normal production activity.

NOTE: A point-in-time database restore may invalidate the server-to-server verification key. Use UPDATE SERVER <server> FORCESYNC=YES to create a new server-to-server verification key.
For more information review "UPDATE SERVER" for the server version closest to the version in your environment:
V6.3.3 server documentation
V7.1.0 server documentation
V8.1.0 server documentation

NOTE: A point-in-time restore of a library manager server can create inconsistencies between the volume inventories of the library manager and library client servers. Steps must be taken to prevent this problem.
For more information review "Restoring to a point-in-time a library manager server" for the server version closest to the version in your environment:
V6.3.3 server documentation
V7.1.0 server documentation

NOTE: A point-in-time restore of a library client server can cause volumes to be removed from the volume inventory of a library client server and later overwritten.
For more information review "Restoring to a point-in-time a library client server" for the server version closest to the version in your environment:
V6.3.3 server documentation
V7.1.0 server documentation

3. From the old volume history information that was generated by the QUERY VOLHISTORY command, obtain a list of all the volumes that were reused (STGREUSE), added (STGNEW), and deleted (STGDELETE) since the original backup. Use this list to perform the remaining steps in this procedure. It might also be necessary to update the device configurations in the restored database.

NOTE: You may want to temporarily disable access to the sequential volumes in the list until the remaining steps in this procedure are complete. If you want to do this, issue the UPDATE VOLUME command with ACCESS=UNAVAILABLE. If you do this, you will need to issue the UPDATE VOLUME command with ACCESS=READONLY before running the AUDIT VOLUME commands in the remaining steps.

4. Issue AUDIT VOLUME command and specify the FIX=YES parameter the following volume types:
  • all disk volumes
  • STGREUSE and STGDELETE volume from step 3 (TAPE, VTL and non-deduplicated FILE)
Do NOT run AUDIT VOLUME with FIX=YES on FILE volumes in a deduplicated storage pool. The "Recovering from lost or damaged FILE volumes in deduplicated storage pool" note later in this procedure has special considerations for FILE volumes in a deduplicated storage pool.
The audit volume process identifies files recorded in the database that can no longer be found on a volume. If a copy of the file is in a copy storage pool or an active-data pool, the file on the audited volume is marked as damaged. Otherwise, the file is deleted from the database and is lost.

5. If the audit detects any damaged files, issue the RESTORE STGPOOL command to restore those files after you audit the volumes in the storage pool.

6 a. Mark as "destroyed" any volumes that cannot be located.

6 b. Directory-container storage pools have additional procedures for recovering from data loss or system outages that should be followed after a point-in-time database restore. For more information review "Recovering from data loss or system outages" for the server version closest to the version in your environment:
V7.1.3 server documentation
V7.1.7 server documentation
V8.1.0 server documentation

6 c. Node replication configurations have additional considerations that should be reviewed after a point-in-time database restore. A point-in-time database restore of a source replication server will disable replication operations originating from the source server to prevent it from deleting copies of data that might exist on the target replication servers . To preserve the data that exists on target replication servers, determine whether copies of data that are on the target replication server are needed. If they are, you must replicate data that is on the target replication server to the source replication server.
If on V7 and below 7.1.1.304 or V6 and below 6.3.5.109 be aware of APAR IT06923 and its corrective actions before using the following:
Reference the steps from the messages manual for message ANR0340E for the server version closest to the version in your environment:
V6.3.3 server documentation
V7.1.0 server documentation
V8.1.0 server documentation

6 d. FILE volumes in a deduplicated storage pool have additional procedures for recovering from data loss or damage that should be followed after a point-in-time database restore. For more information review the following technote:
Recovering from lost or damaged FILE volumes in deduplicated storage pool

6 e. NOTE: Any "destroyed" FILE volumes in deduplicated storage pools should already be addressed in "Recovering from lost or damaged FILE volumes in deduplicated storage pool" above. Recover any additional "destroyed" volumes from copy storage pool backups. If no backups are available, delete the volumes from the database by using the DELETE VOLUME command with the DISCARDDATA=YES parameter.

7. Redefine any storage pool volumes that were added since the database backup.

Additional considerations:
  • After a restore, the volume inventories for Tivoli® Storage Manager and for your tape management system might be inconsistent. For example, after a database backup, a new volume is added to Tivoli Storage Manager. The tape management system inventory records the volume as belonging to Tivoli Storage Manager. If the database is restored from the backup, Tivoli Storage Manager has no record of the added volume, but the tape management system does. You must synchronize these inventories.
  • Similarly, the volume inventories for Tivoli Storage Manager and for any automated libraries might also be inconsistent. Issue the AUDIT LIBRARY command to synchronize these inventories.

  • When you restore a database on a server that uses virtual volumes (either the source or target server), you should reconcile the virtual volumes on the source server and the archive files on the target server.

  • For more information review "RECONCILE VOLUMES" for the server version closest to the version in your environment:
    V6.3.3 server documentation
    V7.1.0 server documentation
    V8.1.0 server documentation
    FLASH 1655392 - Data integrity problem running RECONCILE VOLUMES with backup operations. Fixed in 6.3.5.0, 6.2.6.0 and 7.1.1.0 with APAR IC93989.
  • If the IBM Tivoli Storage Manager gets out-of-sync with the LDAP directory server, you might notice some unexpected errors. To put the data in-sync, issue the AUDIT LDAPDIRECTORY command.

  • For more information review "AUDIT LDAPDIRECTORY" for the server version closest to the version in your environment:
    V6.3.3 server documentation
    V7.1.0 server documentation
    V8.1.0 server documentation

Related Information

[{"Product":{"code":"SSGSG7","label":"Tivoli Storage Manager"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Component":"Server","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF010","label":"HP-UX"},{"code":"PF016","label":"Linux"},{"code":"PF027","label":"Solaris"},{"code":"PF033","label":"Windows"}],"Version":"Version Independent","Edition":"","Line of Business":{"code":"LOB26","label":"Storage"}}]

Document Information

Modified date:
17 June 2018

UID

swg21294024