IT14101: SERVER PROCESSES MIGHT HANG WHILE ADMSERVERMONITORTHREAD IS IN ADMGETFRONTENDCAPACTIY

7.1.6.000-TIV-TSMSRV-Linuxs390x
7.1.6.000-TIV-TSMSRV-Linuxppc64
7.1.6.000-TIV-TSMSRV-WIN
7.1.6.000-TIV-TSMSRV-SolarisSPARC
7.1.6.000-TIV-TSMSRV-Linuxx86_64
7.1.6.000-TIV-TSMSRV-HP-UX
7.1.6.000-TIV-TSMSRV-AIX
7.1.5.100-TIV-TSMSRV-SolarisSPARC
7.1.5.100-TIV-TSMSRV-WIN
7.1.5.100-TIV-TSMSRV-Linuxx86_64
7.1.5.100-TIV-TSMSRV-Linuxs390x
7.1.5.100-TIV-TSMSRV-Linuxppc64
7.1.5.100-TIV-TSMSRV-HP-UX
7.1.5.100-TIV-TSMSRV-AIX
IBM Spectrum Protect Server V7.1.5.X interim fix downloads
IBM Spectrum Protect Server V7.1 Fix Pack 6 (7.1.6.000) Downloads

APAR status

• Closed as program error.

Error description

• Process and session hang symptoms might be observerd if the
  Tivoli Storage Manager AdmServerMonitorThread is waiting for
  locks in the resource intensive admGetFrontEndCapactiy
  processing.
  
  See the following pstack example collected on a Linux box:
  pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  pkWaitConditionTracked (condP=<optimized out>, mutexP=<optimized
  out>, abstime=0, file=0x137d639 "tmlock.c", line=2214) at
  pkmon.c:1221
  WaitForLock (file=0x11d0a48 "imfs.c", lineNum=3964,
  lockP=0x7ffefc0af1f8, holdP=<optimized out>,
  txnP=0x7fff24000918, type=19002, nameSpace=0, key=0x7fffa09ffbc0
  "11.2", keyLen=4, mode=4 '\004', escalation=False) at
  tmlock.c:2214
  tmLockTracked (file=0x11d0a48 "imfs.c", lineNum=3964,
  txnId=0x7fff24000918, type=19002, nameSpace=0,
  key=0x7fffa09ffbc0 "11.2", keyLen=4, mode=4 '\004',
  condition=UnconditionalLock) at tmlock.c:750
  ImLockFsIdTracked (file=0x11d0a48 "imfs.c", lineNum=3964,
  txnId=0x7fff24000918, nodeId=11, fsId=2, lockMode=<optimized
  out>) at imutil.c:10530
  ImLockFileSpaceTracked (file=0x11d0a48 "imfs.c", lineNum=3964,
  txnId=0x7fff24000918, nodeId=11, fsId=2, lockMode=4 '\004') at
  imutil.c:10604
  imUpdateFsFrontEndCapacity (txnId=0x7fff24000918, nodeId=11,
  fsId=2, feCapacity=7234299501) at imfs.c:3964
  admGetFrontEndCapactiy (frontEndCap=0x7fffa09ffe20,
  frontEndClientCount=0x7fffa09ffe7c,
  lastFECapDate=0x7fffa09ffe30) at admutil.c:5740
  AdmServerMonitorThread (argP=<optimized out>) at admstart.c:4839
  StartThread (startInfoP=0x0) at pkthread.c:3779
  start_thread () from /lib64/libpthread.so.0
  clone () from /lib64/libc.so.6
  
  Under the APAR condition, other server processes or client
  sessions might hang and timeouts might not processed properly.
  For example, SHOW RESQ could report something similar to:
  This is a local waiter.
  status=resWaiting waitTime=3035 minutes          <= wait time
  significantly exceeds
  waiter Type=unknown (0) timeout value=60 minutes <= timeout
  value
  txnSeqNo=0:1438 resourceName=(11.2) lengthLen=4
  type=19002 and nameSpace=0
  waiter thead id is 250 condition=671231080
   mutex=28193944 abortFunc=ec6690
  It is expected that with the fix for
  IT13046: SERVER CAN DEADLOCK IN ADMSERVERMONITORTHREAD
  applied the symptom is seen less frequently.
    Tivoli Storage Manager Versions Affected: all supported
    Initial Impact: Medium
    Additional Keywords: TSM IBM Spectrum Protect
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * All Tivoli Storage Manager server users.                     *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See error description.                                       *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Apply fixing level when available. This problem is currently *
  * projected to be fixed in levels 7.1.5.100 and 7.1.6 . Note   *
  * that this is subject to change at the discretion of IBM.     *
  ****************************************************************
  

Problem conclusion

• This problem was fixed.
  Affected platforms:  AIX, HP-UX, Solaris, Linux, and Windows.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  TSM SERVER

• Fixed component ID

  5698ISMSV

Applicable component levels

• R71A PSY

     UP

• R71H PSY

     UP

• R71L PSY

     UP

• R71S PSY

     UP

• R71W PSY

     UP