Possible data corruption during incremental FlashCopy backup, or during a FlashCopy restore following an incremental FlashCopy Backup, with TSM for ACS, or Tivoli Storage FlashCopy Manager

Flash (Alert)


Abstract

Possible data corruption during incremental FlashCopy backup, or during a FlashCopy restore following an incremental FlashCopy Backup, with Tivoli Storage Manager for Advanced Copy Services or Tivoli Storage FlashCopy Manager when using DS6000, DS8000, or ESS Series Hardware

Content

This document is divided into linked sections for ease of navigation. You may use the links below to jump to the desired section of the document.
Problem
Background Context
Scenario 1 Reversal of Source-target Relationships During Backup by Mistake
Scenario 2 Failure to Reverse Source-target Relationships During Restore As Required
Affected Products, Components, and Levels
Conditions Required for Problem to Occur
How Do I Know if I am Affected?
Workaround
Fixing Levels

Further Assistance & Related APARs


Problem:
The algorithm used by the Tivoli Storage FlashCopy Manager product running on the AIX, Solaris, and Linux platforms, and by Tivoli Storage Manager for Advanced Copy Services (TSM for ACS), to determine whether a FlashCopy source-target relationship is reversed in the storage hardware definition structure, employs a search for the target LUN id that is not sufficiently precise. Under the conditions named in this document, Tivoli Storage FlashCopy Manager and TSM for ACS do not properly handle some source-target relationships resulting in corruption of both the production data and the local FlashCopy backup image.

Background Context:
For a FlashCopy backup, the user defines source and target volumes.

When either the Tivoli Storage FlashCopy Manager product, running on the AIX, Solaris, or Linux platforms, or the TSM for Advanced Copy Services (TSM for ACS) product, initiates a FlashCopy restore operation from target volumes that have an existing source-target relationship due to a previous incremental FlashCopy backup, the storage system reverses the source-target relationships in the storage hardware definition structure as a part of the FlashCopy operation.

Therefore, it is necessary for the Tivoli Storage FlashCopy Manager product running on the AIX, Solaris, or Linux platforms, or the TSM for Advanced Copy Services (TSM for ACS) product, to revert the source-target relationships back to their original state during the next incremental FlashCopy backup operation performed following a FlashCopy restore.

There are 2 scenarios where the source-target relationships are not properly handled.


Scenario 1: Reversal of Source-target Relationships During Backup by Mistake

During an incremental FlashCopy backup operation (also known as a FLASHCOPY_TYPE INCR operation), a verification check is performed to determine if the source-target relationship is reversed. This verification check searches the source volume list using the LUN IDs associated with the target volumes.

If the LUN ID associated with the target volume is the same value as the storage hardware device's Type identifier (e.g. "2107 for DS8k devices, "1750" for DS6K devices, or "2105" for ESS devices), or is contained within the string that represents the serial number of the device, the relationship is incorrectly considered to be reversed. Therefore, when the source-target relationship is reversed by the FlashCopy backup process, it causes the FlashCopy operation to be performed from the target volume to the source volume by mistake.

As a consequence, the data on the source volume (2106 in the example below) is replaced by the data from the target volume (2107 in the example below) which could result in corruption of the data stored on the affected source volumes.





Scenario 2: Failure to Reverse Source-target Relationships During Restore As Required

During the FlashCopy restore of an incremental backup, a verification check similar to the one in Scenario 1 is performed to determine if the source-target relationship is reversed.This verification check uses the same incorrect search for the LUN ids as described in Scenario 1.

Therefore, if the LUN ID associated with the target volume is the same value as the storage hardware device's Type identifier (e.g. "2107 for DS8k devices, "1750" for DS6K devices, or "2105" for ESS devices), or is contained within the string that represents the serial number of the device, then the relationship is not considered to be reversed as it should be. Therefore, the source-target relationship is not reversed by the FlashCopy restore process which causes the FlashCopy operation to be performed from the source volume to the target volume by mistake.

The failure to reverse the source-target relationship corrupts the data on the target volume thereby invalidating the FlashCopy backup image. Additionally, the FlashCopy backup is not restored to the source volume.




In Scenario 1 and Scenario 2, the data and backup image stored on the affected volumes is corrupted.

The data stored on the affected volumes must be recovered from:
  • another copy of the same backup image stored in a Tivoli Storage Manager Server (TSM), if available.
  • a backup taken at an earlier point in time

If a copy of the incremental FlashCopy backup is stored in a TSM Server, then the corrupted data can be recovered because the restore operation can be repeated using this copy of the backup image. However, this approach is optional and some customers may not have another copy of the same backup image. In cases where another copy of the same backup image is not available via a TSM Server, then the production volumes would need to be restored from a different backup image if available. In this event, and when volumes are used by database applications, the database may need to be restored from an earlier database backup and log files applied as required by the database to bring the database to its current state before the data volumes were corrupted.



Affected Products, Components, and Levels

Product and Version Affected Components
IBM Tivoli Storage FlashCopy Manager v2.2
  • FlashCopy Manager for DB2 v2.2.0.0- v2.2.1.0
  • FlashCopy Manager for Oracle v2.2.0.0- v2.2.1.0
  • FlashCopy Manager for Oracle in SAP Environments v2.2.0.0- v2.2.1.0
  • FlashCopy Manager for Custom Applications v2.2.0.0- v2.2.1.0
IBM Tivoli Storage FlashCopy Manager v2.1
  • FlashCopy Manager for DB2 v2.1.0.0 - v2.1.0.2
  • FlashCopy Manager for Oracle v2.1.0.0 - v2.1.0.2
  • FlashCopy Manager for Oracle in SAP Environments v2.1.0.0 - v2.1.0.2
IBM Tivoli Storage Manager for Advanced Copy Services v6.1
  • Data Protection for Snapshot Devices v6.1.0.0
  • Data Protection for Disk Storage & SAN VC for Oracle v5.5.1.0
  • Data Protection for Snapshot Devices for mySAP v5.4.0 - v5.4.3.1
  • HW Devices Snapshot Integration Module v5.3.0.0 - v5.3.4.8
IBM Tivoli Storage Manager for Advanced Copy Services v5.5
  • Data Protection for Snapshot Devices for DB2 ACS v5.5.0
  • Data Protection for Disk Storage & SAN VC for Oracle v5.3.1.0 & v5.5.1.0
  • Data Protection for Snapshot Devices for mySAP v5.4.0 - v5.4.3.1
  • HW Devices Snapshot Integration Module v5.3.0.0 - v5.3.4.8
IBM Tivoli Storage Manager for Advanced Copy Services v5.4
  • Data Protection for Disk Storage & SAN VC for Oracle v5.3.1.0
  • Data Protection for Snapshot Devices for mySAP v5.4.0 - v5.4.3.1
  • HW Devices Snapshot Integration Module v5.3.0.0 - v5.3.4.8



Conditions Required for Problem to Occur

The data corruption, and loss of backup image, can occur when all of the following conditions are met:
    1. The storage subsystem being used with the affected products is one of the following:
    • IBM System Storage DS6000 Series
    • IBM System Storage DS8000 Series
    • IBM TotalStorage® Enterprise Storage Server™ (ESS)

    2. The LUN id of the target volume matches one of the following:
    • the storage hardware device's Type identifier (e.g. "2107 for DS8k devices, "1750" for DS6K devices, or "2105" for ESS devices)
    • a portion of the string that represents the serial number of the storage hardware (e.g. storage hardware has serial number like "IBM.2107-7504491" and target LUN id is "2107", "7504", "5044", "0449", or "4491).

    3. One of the following storage subsystem FlashCopy operations is performed:
    • Incremental FlashCopy backup (e.g. FLASHCOPY_TYPE INCR operation)
    • FlashCopy restore of an incremental FlashCopy backup

    4. The affected LUN ID must also the be first LUN ID in the list of relationships checked. **
        **Note: This condition applies only to customers using one of the components and levels of TSM for Advanced Copy Services listed below:
    • Data Protection for Disk Storage & SAN VC for Oracle 5.3.x and 5.5.x
    • Data Protection for Snapshot Devices for mySAP 5.4.x



How Do I Know if I am Affected?

Since the operations that manipulate the source-target volume relationships are normal FlashCopy operations, there is not an error message or other direct symptom that will alert the end user that the relationship has been incorrectly handled.

As a result, if you use are using the Tivoli Storage FlashCopy Manager product running on the AIX, Solaris, or Linux platforms or the Tivoli Storage Manager for Advanced Copy Services (TSM for ACS) product to perform:
  • an incremental FlashCopy backup operation (also known as a FLASHCOPY_TYPE INCR operation), OR
  • a FlashCopy restore of an incremental FlashCopy backup operation (also known as a FLASHCOPY_TYPE INCR operation)

an examination of LUN IDs configured on the system is required to determine if you are affected. You should use either the DSCLI or ESSCLI commands available via the storage subsystem to generate a list of LUN IDs defined to the storage subsystem. Then, the content of that list should be compared against the criteria defined in the Conditions for Problem to Occur section of this document. If any of the configured LUN IDs meet the defined criteria, and those LUN IDs are allocated for use by the affected products, then you are likely affected.

IBM is making a script available that will perform this examination for you and identify volume(s) that may be affected by the issue. The script, and details for using it, are described in this tech note document: http://www.ibm.com/support/docview.wss?uid=swg21502016



Workaround:

To avoid encountering this problem prior to fix application:
    1) Do not configure target LUN IDs on the storage subsystem where:
    • the LUN id is the same value as the storage hardware device's Type identifier (e.g. "2107 for DS8k devices, "1750" for DS6K devices, or "2105" for ESS devices)
    • the LUN id is contained within the string that represents the serial number of the storage hardware.
    • E.g. Issue happens in case the storage hardware has serial number "IBM.2107-7504491" and target LUN id is "2107", "7504", "5044", "0449", or "4491".

    OR

    2) If you already have LUN IDs configured that meet the affected criteria above:
      A) Do not allocate those for use by Tivoli Storage FlashCopy Manager or TSM for Advanced Copy Services (TSM for ACS) for use as FlashCopy target volumes

      B) If you have already allocated volumes for use as target volumes by Tivoli Storage FlashCopy Manager or TSM for ACS you can:
      • delete any backups using such target volumes, if any
      • remove the volumes from the storage subsystem's configuration file of eligible LUN IDs for use by one of the affected products
      • create a new, unaffected LUN ID configuration
      • perform a new backup


Fixing Levels

The tables below detail the affected products, components, fix levels, and details regarding availability of the fixing level.


Tivoli Storage FlashCopy Manager V2.2 Fixes

Affected Components
Fixing Level
Cumulative Update?
Package Availability Info
FlashCopy Manager for DB2
v2.2.0.0- v2.2.1.1
2.2.1.2
Yes
Availability Date: June 10, 2011

Package Names:
AIX: 2.2.1.2-TIV-TSFCMFTP-AIX.bin
Solaris: 2.2.1.2-TIV-TSFCMFTP-Linux.bin
Linux: 2.2.1.2-TIV-TSFCMFTP-Solairs.bin

Package download information:
http://www.ibm.com/support/docview.wss?uid=swg24030173
FlashCopy Manager for Oracle
v2.2.0.0- v2.2.1.1
2.2.1.2
Yes
Target Availability Date: June 10, 2011

Package Names:
AIX: 2.2.1.2-TIV-TSFCMFTP-AIX.bin
Solaris: 2.2.1.2-TIV-TSFCMFTP-Linux.bin
Linux: 2.2.1.2-TIV-TSFCMFTP-Solairs.bin

Package download information:
http://www.ibm.com/support/docview.wss?uid=swg24030173
FlashCopy Manager for Oracle in SAP Environments
v2.2.0.0- v2.2.1.1
2.2.1.2
Yes
Target Availability Date: June 10, 2011

Package Names:
AIX: 2.2.1.2-TIV-TSFCMFTP-AIX.bin
Solaris: 2.2.1.2-TIV-TSFCMFTP-Linux.bin
Linux: 2.2.1.2-TIV-TSFCMFTP-Solairs.bin

Package download information:
http://www.ibm.com/support/docview.wss?uid=swg24030173
FlashCopy Manager for Custom Applications
v2.2.0.0- v2.2.1.1
2.2.1.2
Yes
Target Availability Date: June 10, 2011

Package Names:
AIX: 2.2.1.2-TIV-TSFCMFTP-AIX.bin
Solaris: 2.2.1.2-TIV-TSFCMFTP-Linux.bin
Linux: 2.2.1.2-TIV-TSFCMFTP-Solairs.bin

Package download information:
http://www.ibm.com/support/docview.wss?uid=swg24030173



Tivoli Storage FlashCopy Manager V2.1 Fixes

Affected Components
Fixing Level
Cumulative Update?
Package Availability Info
FlashCopy Manager for DB2
v2.1.0.0 - v2.1.0.3
2.1.0.4 Yes Target Availability Date: June 10, 2011

Package Names:
2.1.0.4-TIV-TSFCMFTP-AIX.bin

Package download information:
http://www.ibm.com/support/docview.wss?uid=swg24030175
FlashCopy Manager for Oracle
v2.1.0.0 - v2.1.0.3
2.1.0.4 Yes Target Availability Date: June 10, 2011

Package Names:
2.1.0.4-TIV-TSFCMFTP-AIX.bin

Package download information:
http://www.ibm.com/support/docview.wss?uid=swg24030175
FlashCopy Manager for Oracle in SAP Environments
v2.1.0.0 - v2.1.0.3
2.1.0.4 Yes Target Availability Date: June 10, 2011

Package Names:
2.1.0.4-TIV-TSFCMFTP-AIX.bin

Package download information:
http://www.ibm.com/support/docview.wss?uid=swg24030175



Tivoli Storage Manager (TSM) for Advanced Copy Services V5.3, V5.4, V5.5, and V6.1 Fixes

Affected Components
Fixing
Level
Cumulative Update?
Package Availability Info
Data Protection for Snapshot Devices
v6.1.0.0

which includes:
  • Data Protection for Snapshot Devices for DB2 ACS
  • Data Protection for Snapshot Devices for Oracle
  • Data Protection for Snapshot Devices for Oracle with SAP
v6.1.0.1
Yes
Target Availability Date: June 10, 2011

Package Names:
6.1.0.1-TIV-TSMACSFTP-AIX.bin

This product and release reached End of Support on April 30, 2014. Tivoli Storage FlashCopy Manager is the replacement product.
Data Protection for Snapshot Devices for DB2 ACS
v5.5.0
v5.5.0.1
Yes
Target Availability Date: June 10, 2011

Package Names:
5.5.0.1-TIV-TSMACSDB2-AIX.bin

This product and release reached End of Support on April 30, 2014. Tivoli Storage FlashCopy Manager is the replacement product.
Data Protection for Disk Storage & SAN VC for Oracle v5.5.1.0
v5.5.1.1
Yes
Target Availability Date: June 27, 2011

Package Names:
5.5.1.1-TDPHDW.ORACLE.tar.Z

This product and release reached End of Support on April 30, 2014. Tivoli Storage FlashCopy Manager is the replacement product.
Data Protection for Disk Storage & SAN VC for Oracle v5.3.1.0
v5.5.1.1
Yes
Please install the v5.5.1.1 level fix. There are no intermediate release levels between 5.3.1 and 5.5.1. The 5.5.1.1 release contains the fix for this issue plus additional currency and APAR fixes.

This product and release reached End of Support on April 30, 2014. Tivoli Storage FlashCopy Manager is the replacement product.
Data Protection for Snapshot Devices for mySAP
v5.4.x
v5.4.3.2
Yes
Target Availability Date: June 10, 2011

Package Names:
5.4.3.2-TIV-TSMACSSAP-AIX.bin

This product and release reached End of Support on April 30, 2014. Tivoli Storage FlashCopy Manager is the replacement product.
DB2 UDB Integration Module
v5.4.x

Hardware Devices Snapshot Integration Module
v5.3.x
See Note at right.
See Note at right.
Note: The DB2 UDB Integration Module V5.3.x is used with the HW Devices Snapshot Integration Module V5.3.x. Only the HW Devices Snapshot Integration Module is affected. The newer "Data Protection for Snapshot Devices for DB2 ACS" component provides the same environment support and functionality available in these two components except for the AIX 5.2 and DB UDB 8.2 environments. However, the AIX 5.2 and DB UDB 8.2 environments are end of support and outside of the scope of this fix delivery.

Therefore, the fix for these 2 components it to install and use Data Protection for Snapshot Devices for DB2 ACS v5.5.0.1.

Target Availability Date: See row for Data Protection for Snapshot Devices for DB2 ACS v5.5.0

Package Names:
See row for Data Protection for Snapshot Devices for DB2 ACS v5.5.0

This product and release reached End of Support on April 30, 2014. Tivoli Storage FlashCopy Manager is the replacement product.

For releases of the specified products earlier than those shown in the table or TSM for Hardware, please contact IBM Support.


Further Assistance & Related APARs

After reviewing this document, if you think your data might be affected and have questions, or if you are not sure whether your data might be affected, contact IBM technical support for further assistance. Please refer to one of the following APARs as applicable to your environment:
  • IC74961 Tivoli Storage FlashCopy Manager for DB2
  • IC75616 Tivoli Storage FlashCopy Manager for Oracle
  • IC75617 Tivoli Storage FlashCopy Manager for Oracle in SAP environments
  • IC75617 Tivoli Storage FlashCopy Manager for Custom Applications
  • IC75765 Tivoli Storage Manager for Advanced Copy Services: Data Protection for Snapshot Devices for mySAP
  • IC76761 Tivoli Storage Manager for Advanced Copy Services: DP for Snapshot Devices
  • IC76761 Tivoli Storage Manager for Advanced Copy Services: DP for Snapshot Devices for DB2 ACS
  • IC76763 Tivoli Storage Manager for Advanced Copy Services: DP for Snapshot Devices for Oracle
  • IC76764 Tivoli Storage Manager for Advanced Copy Services: DP for Snapshot Devices for Oracle with SAP
  • IC76767 Tivoli Storage Manager for Advanced Copy Services: Data Protection for Disk Storage & SAN VC for Oracle
  • IC76768 Tivoli Storage Manager for Advanced Copy Services: HW Devices Snapshot Integration Module

IBM technical support will work with you to:
  • Help you determine whether your data might be affected.
  • Help you locate the appropriate fix level for your product and application environment.

Cross reference information
Segment Product Component Platform Version Edition
Storage Management Tivoli Storage FlashCopy Manager FlashCopy Manager for Custom Applications AIX, Linux/x86, Solaris 2.2, 2.2.1
Storage Management Tivoli Storage FlashCopy Manager FlashCopy Manager for Oracle AIX, Linux/x86, Solaris 2.1, 2.2, 2.2.1
Storage Management Tivoli Storage FlashCopy Manager FlashCopy Manager for Oracle (SAP® environments) AIX, Linux/x86, Solaris 2.1, 2.2, 2.2.1
Storage Management Tivoli Storage Manager for Advanced Copy Services DB2 Snapshot AIX 5.5, 5.4, 6.1
Storage Management Tivoli Storage Manager for Advanced Copy Services DP for DS/SVC for mySAP AIX 5.4
Storage Management Tivoli Storage Manager for Advanced Copy Services DP for DS/SVC for Oracle AIX 5.5

Rate this page:

(0 users)Average rating

Document information


More support for:

Tivoli Storage FlashCopy Manager
FlashCopy Manager for DB2

Software version:

2.1, 2.2, 2.2.1

Operating system(s):

AIX, Linux/x86, Solaris

Reference #:

1495839

Modified date:

2011-06-29

Translate my page

Machine Translation

Content navigation