IC88953: UNEXPECTED VOLUME HISTORY FILE FORMAT CAN CAUSE SERVER HANG DURI NG HALT

APAR status

• Closed as program error.

Error description

• The Tivoli Storage Manager Server can hang during shutdown
  processing, after an administrator issues the HALT command.
  For example:
  
    TSM:TSMSERVER> HALT
    ANR2017I Administrator SERVER_CONSOLE issued command: HALT
    ANR1912I Stopping the activity log because of a server
              shutdown.
    < HANG >
  
  Without administrator intervention, the dsmserv process will
  continue to run.  High CPU utilization can be seen during
  the hang.  Forcibly killing the dsmserv process is required
  to stop it.
  
  The cause of this hang is an unexpected volume history file
  format.  The volume history file may be in an unexpected or
  invalid format for a V6 server (for example, in a V5 format).
  The location of the volume history file is defined by the
  VOLUMEHISTORY option within the dsmserv.opt options file.
  
  Customer/L2 Diagnostics:
  A VOLHST trace will show the following behavior:
  
    12:17:30.355 ￭53￭output.c￭7271￭PutConsoleMsg:ANR2017I
      Administrator SERVER_CONSOLE issued command: HALT ~
    12:17:30.359 ￭1￭icvolhst.c￭5645￭icSavePre63VolHistFiles:
      starting
    12:17:30.359 ￭1￭icvolhst.c￭5682￭icSavePre63VolHistFiles:
      Reached the first stanza or file is empty
    12:17:31.195 ￭12￭output.c￭7271￭PutConsoleMsg:ANR1912I
      Stopping the activity log because of a server shutdown.~
    < HANG >
  
  Dumping the active threads for the dsmserv process will show
  a thread similar to the following (example is from pstack
  on Linux):
  
    read.read()
    filbuf.__filbuf()
    fgets.fgets()
    icvolhst.icSavePre63VolHistFiles()
    admstart.StartServer()
    admstart.admStartServer()
    dsmserv.main()
  
  procstack on aix will show:
  
  read()
  __filbuf()
  fgets()
  icSavePre63VolHistFiles()
  StartServer()
  admStartServer()
  main()
  __start()
  
  Initial Impact:
  Medium
  
  Tivoli Storage Manager Versions Affected:
  All V6.3 Tivoli Storage Manager servers.
  
  Additional Keywords:
  ZZTSM63 TSM HANG HUNG VOL HIST VOLHIST VOLUME HISTORY HALT
  STOP TSMHALT LOOP LP ICSAVEPRE63VOLHISTFILES STOPPED LOOPED
  
  | MDVREGR 6.3.3.0-TIV_5698MSV | (IC83120)
  

Local fix

• Rename the existing volume history file at the filesystem layer
  and create a new one using the BACKUP VOLHIST command.
  

Problem summary

• ****************************************************************
  * USERS AFFECTED: All Tivoli Storage Manager server users.     *
  ****************************************************************
  * PROBLEM DESCRIPTION: See error description.                  *
  ****************************************************************
  * RECOMMENDATION: Apply fixing level when available. This      *
  *                 problem is currently projected to be fixed   *
  *                 in level 6.3.4. Note that this is            *
  *                 subject to change at the discretion of IBM.  *
  ****************************************************************
  *
  

Problem conclusion

• This problem was fixed.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  TSM SERVER

• Fixed component ID

  5698ISMSV

Applicable component levels

• R63A PSY

     UP

• R63H PSY

     UP

• R63L PSY

     UP

• R63S PSY

     UP

• R63W PSY

     UP