IT19408: EXTENTS CAN BE MISSED BY PROTECT STGPOOL PROCESS WHEN USING CROSS-PROTECT

7.1.7.200-TIV-TSMSRV-Linuxs390x
7.1.7.200-TIV-TSMSRV-Linuxppc64
7.1.7.200-TIV-TSMSRV-WIN
7.1.7.200-TIV-TSMSRV-SolarisSPARC
7.1.7.200-TIV-TSMSRV-Linuxx86_64
7.1.7.200-TIV-TSMSRV-HP-UX
7.1.7.200-TIV-TSMSRV-AIX
8.1.1.100-IBM-SPOC-WindowsX64
8.1.1.100-IBM-SPOC-Linuxx86_64
8.1.1.100-IBM-SPOC-Linuxs390x
8.1.1.100-IBM-SPOC-AIX
8.1.1.000-IBM-SPSRV-WindowsX64
8.1.1.000-IBM-SPSRV-Linuxs390x
8.1.1.000-IBM-SPSRV-AIX
8.1.1.000-IBM-SPSRV-Linuxx86_64
IBM Spectrum Protect Server V8.1 Fix Pack 1 (V8.1.1) Downloads
IBM Spectrum Protect Server V7.1.7.X interim fix downloads
IBM Spectrum Protect Server V7.1 Fix Pack 8 (7.1.8.000) Downloads

APAR status

• Closed as program error.

Error description

• Customer/L2 Diagnostics:
  
  During PROTect STGpool processing, some extents can be missed
  without error or warning message when using cross-protect
  configuration as below:
  
  a) Pool1 on ServerA is protected to Pool2 on ServerB, and
  meantime,
  b) Pool2 on ServerB is protected to Pool1 on ServerA.
  
  With the said configuration, some extents might not have been
  protected and might not be able to be recovered when the extents
  are damaged.
  
  Use below steps to determine if the problem exists.
  
  1. On serverA, run DB2 SQL command to determine the
  Poolid_of_Pool1.
  db2 "select poolid as Poolid_of_Pool1 from tsmdb1.ss_pools where
  poolname=upper('Pool1')"
  
  2. On serverB, run DB2 SQL command to determine the
  Poolid_of_Pool2.
  db2 "select poolid as Poolid_of_Pool2 from tsmdb1.ss_pools where
  poolname=upper('Pool2')"
  
  3. On serverA, run DB2 SQL command to determine the
  Serverid_of_ServerB.
  db2 "select serverid as Serverid_of_ServerB from
  tsmdb1.replicating_servers where servername=upper('ServerB')"
  
  4. On serverB, run DB2 SQL command to determine the
  Serverid_of_ServerA.
  db2 "select serverid as Serverid_of_ServerA from
  tsmdb1.replicating_servers where servername=upper('ServerA')"
  
  5. run below 2 DB2 SQL commands on both servers, replace
  <LOCAL_POOL_ID>, <PARNER_POOL_ID> and <PARNER_SERVER_ID> with
  the actual Ids. When running on serverA, <LOCAL_POOL_ID> =
  Poolid_of_Pool1 and <PARNER_POOL_ID> = Poolid_of_Pool2,
  <PARNER_SERVER_ID> = Serverid_of_ServerB. When running on
  ServerB, <LOCAL_POOL_ID> = Poolid_of_Pool2 and <PARNER_POOL_ID>
  = Poolid_of_Pool1, <PARNER_SERVER_ID> = Serverid_of_ServerA.
  
  db2 "select 1 as PROBLEM_DETECTED from tsmdb1.sd_chunk_locations
  sdcl left join tsmdb1.sd_replicated_chunks sdrc on
  (sdcl.chunkid=sdrc.chunkid  and sdrc.replpoolid=<PARNER_POOL_ID>
  and sdrc.srvid=<PARNER_SERVER_ID>) left join
  tsmdb1.sd_containers sdcn on (sdcl.cntrid=sdcn.cntrid and
  sdcl.poolid=sdcn.poolid) left join tsmdb1.sd_replicated_stgpools
  sdrp on (sdcn.poolid=sdrp.poolid and
  sdrp.replpoolid=<PARNER_POOL_ID> and
  sdrp.srvid=<PARNER_SERVER_ID>) where sdrc.chunkid is null and
  sdrp.poolid is not null and sdcl.poolid=<LOCAL_POOL_ID> and
  sdrp.replpoolid=<PARNER_POOL_ID> and sdcn.lastwr<sdrp.synctime
  and ( sdcl.refcount>=1 or sdcl.last_ref>(current timestamp -
  current timezone) ) fetch first 1 rows only"
  
  db2 "select 1 as PROBLEM_DETECTED from
  tsmdb1.sd_non_dedup_locations sdnd left join
  tsmdb1.sd_replicated_chunks sdrc on (sdnd.chunkid=sdrc.chunkid
  and sdrc.replpoolid=<PARNER_POOL_ID> and
  sdrc.srvid=<PARNER_SERVER_ID>) left join tsmdb1.sd_containers
  sdcn on (sdnd.cntrid=sdcn.cntrid and sdnd.poolid=sdcn.poolid)
  left join tsmdb1.sd_replicated_stgpools sdrp on
  (sdcn.poolid=sdrp.poolid and sdrp.replpoolid=<PARNER_POOL_ID>
  and sdrp.srvid=<PARNER_SERVER_ID>) where sdrc.chunkid is null
  and sdrp.poolid is not null and sdnd.poolid=<LOCAL_POOL_ID> and
  sdrp.replpoolid=<PARNER_POOL_ID> and sdcn.lastwr<sdrp.synctime
  and ( (sdnd.refcount is null or sdnd.refcount=1) or
  (sdnd.last_ref is null or sdnd.last_ref>'1956-09-05 00:00:00') )
  fetch first 1 rows only"
  
  6. The problem exists on the server if either of DB2 SQL
  commands returns PROBLEM_DETECTED = 1.
  
  REPLicate Node is not affected by this issue. Users of Replicate
  Node will not have unprotected extents.
  
  IBM Spectrum Protect Versions Affected: 7.1 and 8.1
  
  Initial Impact: High
  Additional Keywords: protect stgpool extents
  

Local fix

• Protect Stgpool with FORCEREConcile=yes on the server(s) that
  the problem exists.
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * All Tivoli Storage Manager and IBM Spectrum Protect Server   *
  * users of directory container pools and PROTECT STGPOOL.      *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See ERROR DESCRIPTION.                                       *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Apply fixing level when available.This problem is currently  *
  * projected to be fixed in levels 7.1.7.200, 7.1.8 and 8.1.1.  *
  * Note that this is subject to change at the discretion of     *
  * IBM.                                                         *
  ****************************************************************
  

Problem conclusion

• This problem was fixed.
  Affected platforms:  AIX, Solaris, Linux, and Windows.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  TSM SERVER

• Fixed component ID

  5698ISMSV

Applicable component levels

• R71A PSY

     UP

• R71L PSY

     UP

• R71S PSY

     UP

• R71W PSY

     UP

• R81A PSY

     UP

• R81L PSY

     UP

• R81W PSY

     UP