IC90488: IDENTIFY DEDUP PROCESSES PERFORMANCE DEGRADATION AFTER UPGRADE FROM V6.1.X OR V6.2.0

APAR status

• Closed as program error.

Error description

• After upgrade, the Query Process for Identify Duplicates for
  storage pools has very slow performance. These processes
  are run as part of server-side and client-side deduplication
  processing. If you do not have deduplication enabled on your
  server, you are not affected by this APAR.
  
  Example of Query Process reporting very few duplicate bytes
  found:
  Process     Process Description      Process Status
  Number
  --------     --------------------     -------------------------
     147     Identify Duplicates      Storage pool: xxx. Volume:
                /sss/yyyy.BFS. State: active.
                State Date/Time: 02/19/2013 01:37:47 PM. Current
                Physical File(bytes): 1,635,566. Total Files
                Processed: 8. Total Duplicate Extents Found: 0.
                Total Duplicate Bytes Found: 0.
  
  Tivoli Storage Manager Versions Affected:
  Any 6.1.X or 6.2.0 server upgrading to 6.2.1.00 or later, or
  upgrading to 6.3.0.00 or later.
  Any new server 6.2.1.00  or later, 6.3.0.00 or later, that
  formats a new database will not experience this problem.
  
  Customer/L2 Diagnostics
  Data collection from the Tivoli Storage Manager V6 server
  monitoring data:
  http://www.ibm.com/support/docview.wss?uid=swg21432937
  Will show very little Dedup Thread activity.
  This is a 15 minute example:
  Thread 22372 BfDedupVolumeThread parent=22371
  16:02:00.820-->16:16:06.764
   DB2 Reg Exec        5  845.813  169.163    0.000  721.997
  
   Thread 22373 BfDedupVolumeThread parent=22371
  16:02:00.821-->16:16:06.764
   DB2 Reg Exec        5  845.722  169.144    0.000  791.955
  
  There is a missing index. Do these steps to verify that the
  index is missing:
  From the instance owner account, do:
  db2 connect to tsmdb1
  db2look -d tsmdb1 -t bf_bitfile_extents -e >bfbitfileextents.out
  View the output file and look for:
  
  CREATE INDEX "TSMDB1  "."BFBE_PENDING_NDX" ON "TSMDB1
  "."BF_BITFILE_EXTENTS"
    ("PENDING_ID" ASC,
     "LINKBFID" ASC,
     "BFID" ASC)
    MINPCTUSED 20
    COMPRESS YES ALLOW REVERSE SCANS;
  
  If this index isn't found in the "bfbitfileextents.out" output,
  you are experiencing this APAR.
  
  Initial Impact: Medium
  Additional Keywords: poor performance Dedup
  

Local fix

• Create the missing index.  From the instance owner account, do:
  db2 connect to tsmdb1
  db2 "CREATE INDEX TSMDB1.BFBE_PENDING_NDX ON
  TSMDB1.BF_BITFILE_EXTENTS
                  (PENDING_ID ASC,
                   LINKBFID ASC,
                   BFID ASC)
   MINPCTUSED 20 COMPRESS YES ALLOW REVERSE SCANS collect
  statistics"
  
  This can be run while the server is operating, but do not halt
  the server until the command completes, which
  may take several hours depending on the size of your database
  and how busy your server is.
  
  Before the index is created, if you are experiencing
  unacceptable performance degradation on your server, you can
  disable the identify processes on storage pools for which
  deduplication is enabled do the following:
       UPDATE STGPOOL <stgpool> IDENTIFYPROCESS=0
  
  After the index creation completes, use the UPDATE STGPOOL
  command to restore the IDENTIFYPROCESS values back to their
  previous values.
  

Problem summary

• ****************************************************************
  * USERS AFFECTED: All Tivoli Storage Manager server users.     *
  ****************************************************************
  * PROBLEM DESCRIPTION: See ERROR DESCRIPTION.                  *
  ****************************************************************
  * RECOMMENDATION: Apply fixing level when available.           *
  *                 This problem is currently projected          *
  *                 to be fixed in level 6.3.4.                  *
  *                 Note that this is subject to change at the   *
  *                 discretion of IBM.                           *
  ****************************************************************
  *
  

Problem conclusion

• This problem was fixed.
  Affected platforms:  AIX, HP-UX, Solaris, Linux, and Windows.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  TSM SERVER

• Fixed component ID

  5698ISMSV

Applicable component levels

• R63A PSY

     UP

• R63H PSY

     UP

• R63L PSY

     UP

• R63S PSY

     UP

• R63W PSY

     UP