IT10891: TIVOLI STORAGE MANAGER CRASH DUE TO MUTEX ACQUISITION FAILURE WHEN FREED REQP REMAIN IN THE MOUNT POINT WAITER QUEUE LIST

6.3.6.000-TIV-TSMRPT-AGENT-Linux
6.3.6.000-TIV-TSMRPT-AGENT-AIX
6.3.6.000-TIV-TSMRPT-AIX
6.3.6.000-TIV-TSMRPT-AGENT-Windows
6.3.6.000-TIV-TSMRPT-Linuxx86
6.3.6.000-TIV-TSMRPT-Linuxx86_64
6.3.6.000-TIV-TSMRPT-WindowsI32
6.3.6.000-TIV-TSMRPT-WindowsX64
6.3.6.000-TIV-TSMAC-WindowsX64
6.3.6.000-TIV-TSMAC-WindowsI32
6.3.6.000-TIV-TSMAC-SolarisSPARC
6.3.6.000-TIV-TSMAC-Linuxx86
6.3.6.000-TIV-TSMAC-Linuxs390x
6.3.6.000-TIV-TSMAC-AIX
6.3.6.000-TIV-TSMALL-SolarisSPARC
6.3.6.000-TIV-TSMALL-WindowsX64
6.3.6.000-TIV-TSMSTA-WindowsI32
6.3.6.000-TIV-TSMALL-HP-UX
6.3.6.000-TIV-TSMALL-Linuxppc64
6.3.6.000-TIV-TSMALL-AIX
6.3.6.000-TIV-TSMALL-Linuxs390x
6.3.6.000-TIV-TSMALL-Linuxx86_64
7.1.4.000-TIV-TSMSRV-WIN
7.1.4.000-TIV-TSMSRV-SolarisSPARC
7.1.4.000-TIV-TSMSRV-Linuxx86_64
7.1.4.000-TIV-TSMSRV-Linuxs390x
7.1.4.000-TIV-TSMSRV-Linuxppc64
7.1.4.000-TIV-TSMSRV-HP-UX
7.1.4.000-TIV-TSMSRV-AIX
IBM Spectrum Protect Server V7.1 Fix Pack 4 (7.1.4.000) Downloads
IBM Tivoli Storage Manager V6.3 Fix Pack 6 (6.3.6.000) Server Downloads

APAR status

• Closed as program error.

Error description

• Tivoli Storage Manager Server may crash under following
  conditions;
  
  - Migration with autocopy or simultaneous write to copy pools
  during backup are using.
  - Storage pools involved are a libtype=shared library(remote)
  and another device class that is not libtype=shared(local)
  - Both device classes are exceeding the MOUNTLIMIT causing
  operations to wait.
  
  The dsmserv.err shows Mutex acquisition failure message;
  
  07/11/2015 05:09:34  ANR9999D Mutex acquisition failure,
  errno=22 from asvolmnt.c.5220; thread 42 (tid d2a).
  07/11/2015 05:09:34  ANR7838S Server operation terminated.
  07/11/2015 05:09:34  ANR7833S Server thread 2 (tid 102)
  terminated in response to program abort.
  07/11/2015 05:09:34  ANR7833S Server thread 3 (tid 203)
  terminated in response to program abort.
  
  In the stack.out , it could show one of the following
  callstacks;
  
  1:stack.out
  
  pth_signal.pthread_kill(??, ??) at 0x9000000005a3e94
  pth_signal._p_raise(??) at 0x9000000005a36c8
  raise.raise(??) at 0x90000000002becc
  abort.abort() at 0x900000000082964
  psthread.PsAbortServer(??) at 0x100012b80
  pkthread.pkAbort(??) at 0x10000ba24
  pkmon.pkAcquireMutexTracked(??, ??, ??) at 0x100007208
  asvolmnt.TestSwMpReq(??) at 0x1005a0708
  asvolmnt.TestMpReq(??) at 0x10059e478
  asvolmnt.AsMPAgent(??) at 0x1005a2c90
  pkthread.StartThread(0x0) at 0x10000cadc
  
  2:stack.out
  
  pth_signal.pthread_kill(??, ??) at 0x9000000005a3e94
  pth_signal._p_raise(??) at 0x9000000005a36c8
  raise.raise(??) at 0x90000000002becc
  abort.abort() at 0x900000000082964
  psthread.PsAbortServer(??) at 0x100012b80
  pkthread.pkAbort(??) at 0x10000ba24
  pkmon.pkAcquireMutexTracked(??, ??, ??) at 0x100007208
  astxn.ReleaseSessionEnding(??) at 0x100581b38
  astxn.AsEndSession(??) at 0x1005813b4
  sssess.ssEndSession(??) at 0x1004d83cc
  bfutil.bfEndSession(??) at 0x1004fecb4
  afbackup.EndBackupProc(0x11b0d6888, 0xaf900000af9) at
  0x100bc6230
  afbackup.AfBackupPoolThread(??) at 0x100bc5fd8
  pkthread.StartThread(0x0) at 0x10000cadc
  
  3:stack.out
  
  pthread_kill(??, ??) at 0x90000000053af14
  _p_raise(??) at 0x90000000053a764
  raise.raise(??) at 0x900000000039468
  abort() at 0x900000000055ae4
  PsAbortServer(??) at 0x1000120a0
  pkAbort(??) at 0x10000b264
  pkDestroyCondition(??) at 0x100007f88
  AsChangeVolStatus(??, ??, ??, ??) at 0x10085ac0c
  AsReleaseVol(??, ??) at 0x1008556c4
  ReleaseSessionEnding(??) at 0x10047da1c
  AsEndSession(??) at 0x10047d570
  ssEndSession(??) at 0x100446c20
  bfEndSession(??) at 0x100495564
  EndBfSession(??, ??, ??, ??, ??, ??, ??) at 0x1006bd560
  CloseSessStats(??, ??) at 0x1006cef10
  SmNodeSession(??, ??) at 0x1006a5014
  HandleNodeSession(??, ??, ??) at 0x100553bb8
  DoNodeGeneral(??, ??) at 0x10054add0
  smExecuteSession(??, ??, ??, ??, ??, ??, ??, ??) at 0x100545dec
  psSessionThread(??) at 0x100079078
  StartThread(??) at 0x10000c260
  

Local fix

• N/A
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * All Tivoli Storage Manager server users of shared library    *
  * and simultaneous write.                                      *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See ERROR DESCRIPTION.                                       *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Apply fixing levels when available.                          *
  * This problem is currently projected to be fixed in levels    *
  * 6.3.6 and 7.1.4.                                             *
  * Note that this is subject to change at the discretion of     *
  * IBM.                                                         *
  ****************************************************************
  

Problem conclusion

• This problem was fixed.
  Affected platforms:  AIX, HP-UX, Sun Solaris, Linux, and
  Windows.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  TSM SERVER

• Fixed component ID

  5698ISMSV

Applicable component levels

• R63A PSY

     UP

• R63H PSY

     UP

• R63L PSY

     UP

• R63S PSY

     UP

• R63W PSY

     UP

• R71A PSY

     UP

• R71H PSY

     UP

• R71L PSY

     UP

• R71S PSY

     UP

• R71W PSY

     UP