IT20830: LOCAL PROTECT MOVE DRMEDIA WHERESTATE=VAULTRETRIEVE OR COURIERRETRIEVE TOSTATE=ONSITERETRIEVE HANGS

8.1.2.000-IBM-SPSRV-AIX
8.1.2.000-IBM-SPSRV-Linuxppc64le
8.1.2.000-IBM-SPCMS-Linuxx86_64
8.1.2.000-IBM-SPCMS-WindowsI32
8.1.2.000-IBM-SPSRV-Linuxs390x
8.1.2.000-IBM-SPCMS-WindowsX64
8.1.2.000-IBM-SPSRV-Linuxx86_64
8.1.2.000-IBM-SPSRV-WindowsX64
8.1.2.000-IBM-SPOC-AIX
8.1.2.000-IBM-SPOC-LinuxPPC64le
8.1.2.000-IBM-SPOC-Linuxs390x
8.1.2.000-IBM-SPOC-Linuxx86_64
8.1.2.000-IBM-SPOC-WindowsX64
8.1.1.100-IBM-SPSRV-AIX
8.1.1.100-IBM-SPSRV-Linuxppc64le
8.1.1.100-IBM-SPSRV-Linuxs390x
8.1.1.100-IBM-SPSRV-Linuxx86_64
8.1.1.100-IBM-SPSRV-WindowsX64
7.1.7.400-TIV-TSMSRV-AIX
7.1.7.400-TIV-TSMSRV-HP-UX
7.1.7.400-TIV-TSMSRV-Linuxppc64
7.1.7.400-TIV-TSMSRV-Linuxs390x
7.1.7.400-TIV-TSMSRV-Linuxx86_64
7.1.7.400-TIV-TSMSRV-SolarisSPARC
7.1.7.400-TIV-TSMSRV-WIN
IBM Spectrum Protect Server V8.1 Fix Pack 2 (V8.1.2) Downloads
IBM Spectrum Protect Server V7.1 Fix Pack 8 (7.1.8.000) Downloads
IBM Spectrum Protect Server V8.1.1.X interim fix downloads
IBM Spectrum Protect Server V7.1.7.X interim fix downloads

APAR status

• Closed as program error.

Error description

• When recalling DRM volumes back on site, the process hangs but
  will complete on its own.  Example of command:
  MOVE DRMEDIA vol_name wherest=vaultr tost=onsiter
  
  IBM Spectrum Protect Versions Affected:  7.1.7 and above
  
  Customer/L2 Diagnostics
  The Actlog will show a long time gap after starting the Move DRM
  process to the point the volume has been processed.  Example:
  11:13:56      ANR2017I Administrator xxx issued command: MOVE
                            DRMEDIA vvvv wherest=vaultr
  tost=onsiter  (SESSION:
                            67633)
  11:13:56      ANR0609I MOVE DRMEDIA started as process 132.
  (SESSION:
  11:13:56      ANR0610I MOVE DRMEDIA started by xxx as process
  132.
  .... looks hung as there are no messages for this process during
  this time ....
  14:20:54      ANR6684I MOVE DRMEDIA: Volume vvvv was deleted.
  14:20:54      ANR6682I MOVE DRMEDIA command ended: 1 volumes
  processed.
  
  Servermon.pl data or several Show Threads output will list the
  OrmMoveDrmediaThread is waiting for the database "semop" during
  the wait time.  Example:
  
  Thread 104604, Parent 104603: OrmMoveDrmediaThread, Storage
  156991, AllocCnt 2486 HighWaterAmt 203780
   tid=1259c, ptid=1249b, det=1, zomb=0, join=0, result=0, sess=0,
  procToken=132, sessToken=67633
    Stack trace:
      0x09000000001f5190 semop
      0x090000000527a1e8 sqloSSemP
      0x0900000005279d08
  sqlccipcrecv__FP17SQLCC_COMHANDLE_TP12SQLCC_COND_T
      0x090000000527960c sqlccrecv
      0x09000000052797f0 sqljcReceive__FP10sqljCmnMgr
      0x0900000004cccdb0
  sqljrDrdaArOpen__FP14db2UCinterfaceP15db2UCCursorInfo
      0x0900000004772ba0
  csmOpen__FP14db2UCinterfaceP15db2UCCursorInfo
      0x0900000005174dac
  CLI_sqlOpen__FP17CLI_STATEMENTINFOP19CLI_ERRORHEADERINFO
      0x090000000525b3a8
  SQLExecute2__FP17CLI_STATEMENTINFOP19CLI_ERRORHEADERINFO
      0x090000000525a2fc SQLExecute
      0x0000000100189a00 RdbPrepareAndExecuteStmt
      0x00000001001872e4 tbCliMRFetch
      0x00000001006020f0 DeleteScratch
      0x00000001006000ac AsSetVolAccess
      0x00000001005ff544 AsUpdateVolAccess
      0x0000000100a5bc80 ssUpdateVolAccessX
      0x0000000100fe7c60 UpdateVolumeState
      0x0000000100fe7040 ProcessEligibleStgPoolVols
      0x0000000100fe4dc8 MoveStgPoolVolumes
      0x0000000100fe30a0 OrmMoveDrmediaThread
      0x000000010000da94 StartThread
    Holding mutex txnP->mutex (0x15232fc28), acquired at
  tbcli.c(1688)
   Thread context:
     COMMMETHOD: Tcp/Ip
     COMMAND: MOVE DRMEDIA
     THREAD_TYPE: PROCESS
     PROCESS_DESC: MOVE DRMEDIA
     PROCESS_NUMBER: 132
     SESSION_TYPE: ADMIN
     ADMIN_NAME: xxx
  
  Initial Impact: Low
  Additional Keywords: hung process hours Protect
  directory-container storage pools by using tape volumes
  

Local fix

• Process these volumes when they will not impact other tape
  activity
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * All Tivoli Storage Manager and IBM Spectrum Protect server   *
  * users.                                                       *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See error description.                                       *
  *                                                              *
  * In addition to the problem description, this problem can     *
  * happen on any operation that checks to see if we can return  *
  * a volume to scratch.                                         *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Apply fixing level when available. This problem is currently *
  * projected to be fixed in levels 7.1.8 and 8.1.2. Note that   *
  * this is subject to change at the discretion of IBM.          *
  ****************************************************************
  

Problem conclusion

• This problem was fixed.
  Affected platforms for reported release:  AIX, Solaris, Linux,
  and Windows.
  Platforms fixed:  AIX, Solaris, Linux, and Windows.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  TSM SERVER

• Fixed component ID

  5698ISMSV

Applicable component levels