IC98497: DEADLOCK WHEN USING SELECT FROM PROCESSES AND IDENTIFY DUPLICATES SIMULTANEOUSLY

IBM Tivoli Storage Manager V6.3 Fix Pack 5 (6.3.5.000) Server Downloads
IBM Tivoli Storage Manager V7.1 Fix Pack 1 (7.1.1.000) Server Downloads

APAR status

• Closed as program error.

Error description

• A deadlock can occur when all of the following are true:
  - There are one or more IDENTIFY DUPLICATES processes currently
    running
  - A QUERY PROCESS or SELECT FROM PROCESSES command is invoked,
  - An IDENTIFY DUPLICATES or UPDATE STGPOOL <pool>
    IDENTIFYPROCESS=<n> command is invoked
  
  Customer/L2 Diagnostics (If Applicable):
  Thread 7320 is the SELECT from PROCESSES command thread. It is
  holding PROCV->mutex and waiting for BFV->mutex:
  
  $t7320  run  blocked  48824773 u no sys  _global_lock_common
  pth_spinlock._global_lock_common(??, ??, ??) at
  0x90000000095085c
  pth_mutex._mutex_lock(??, ??, ??) at 0x90000000095deb0
  pkmon.pkAcquireMutexTracked(??, ??, ??) at 0x1000067c8
  bfdedup.BfDedupQueryCounts(??, ??, ??, ??) at 0x1007a4e50
  process.procQueryProcess(??, ??, ??, ??) at 0x10025d9d0
  admproc.AdmQueryProcess(??) at 0x100d56d54
  admcmd.AdmCommandLocal(??, ??, ??, ??, ??) at 0x10039d900
  admcmd.admCommand(??, ??, ??, ??, ??) at 0x10039bb78
  admsql.PreFlushDataForSQL(??, ??, ??, ??) at 0x100b0eb90
  admsql.ScrubCmdInput(??, ??, ??) at 0x100b0df38
  admsql.PreProcessQuery(??, ??, ??) at 0x100b07ba0
  admsql.AdmSQLExecute(??) at 0x100b0aa94
  admcmd.AdmCommandLocal(??, ??, ??, ??, ??) at 0x10039d900
  admcmd.admCommand(??, ??, ??, ??, ??) at 0x10039bb78
  smadmin.SmAdminCommandThread(??) at 0x100b00490
  pkthread.StartThread(??) at 0x10000c260
  
  Thread 7321 is the IDENTIFY DUPLICATES command thread. It is
  holding BFV->mutex and waiting for PROCV->mutex.
  
  $t7321  run blocked  59375737 u no sys  _global_lock_common
  pth_spinlock._global_lock_common(??, ??, ??) at
  0x90000000095085c
  pth_mutex._mutex_lock(??, ??, ??) at 0x90000000095deb0
  pkmon.pkAcquireMutexTracked(??, ??, ??) at 0x1000067c8
  process.procBeginProcess(??, ??, ??, ??, ??, ??, ??, ??) at
  0x10025ca24
  bfdedup.bfManageDedupVolumeProcesses(??, ??, ??, ??) at
  0x1007a3a88
  bfdedup.bfResetIdentifyProcesses(??, ??, ??, ??, ??) at
  0x1007a7744
  bfdedup.bfUpdateDedupProcesses(??, ??, ??, ??, ??, ??) at
  0x1007a7428
  admpool.AdmIdentifyDuplicates(??) at 0x100d997dc
  admcmd.AdmCommandLocal(??, ??, ??, ??, ??) at 0x10039d900
  admcmd.admCommand(??, ??, ??, ??, ??) at 0x10039bb78
  smadmin.SmAdminCommandThread(??) at 0x100b00490
  pkthread.StartThread(??) at 0x10000c260
  

Local fix

• Avoid simultaneously querying identify processes and adjusting
  their number.
  

Problem summary

• ****************************************************************
  * USERS AFFECTED: All users of Tivoli Storage Manager server.  *
  ****************************************************************
  * PROBLEM DESCRIPTION: A deadlock can occur when all of the    *
  *                      following are true:                     *
  *                      - At least one of the following         *
  *                      processes is running:  BACKUP NODE,     *
  *                      BACKUP STGPOOL, COPY ACTIVEDATA,        *
  *                      IDENTIFY DUPLICATES, MIGRATE STGPOOL,   *
  *                      MOVE DATA, MOVE NODEDATA, RECLAIM       *
  *                      STGPOOL, RESTORE STGPOOL, RESTORE       *
  *                      VOLUME                                  *
  *                      - A QUERY PROCESS or SELECT FROM        *
  *                      PROCESSES command is invoked            *
  *                      - An IDENTIFY DUPLICATES                *
  *                      NUMPROCESS=<n> or UPDATE STGPOOL        *
  *                      <pool> IDENTIFYPROCESS=<n> command is   *
  *                      invoked.                                *
  ****************************************************************
  * RECOMMENDATION: Apply fixing level when available.  This     *
  *                 problem is currently projected to be fixed   *
  *                 in levels 6.3.5 and 7.1.1.  Note that this   *
  *                 is subject to change at the discretion of    *
  *                 IBM.                                         *
  ****************************************************************
  *
  

Problem conclusion

• The problem was fixed.
  Affected Platforms: AIX, HP-UX, Solaris, Linux, and Windows.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Modules/Macros

• BFDEDUP  BFDEFS
  

Fix information

• Fixed component name

  TSM SERVER

• Fixed component ID

  5698ISMSV

Applicable component levels

• R63A PSY

     UP

• R63H PSY

     UP

• R63L PSY

     UP

• R63S PSY

     UP

• R63W PSY

     UP

• R71A PSY

     UP

• R71H PSY

     UP

• R71L PSY

     UP

• R71S PSY

     UP

• R71W PSY

     UP