IC62990: DSMSERV PROCESS CAN HANG IN THE UPGRADE SCRIPT "INSTALL_SERVER"

APAR status

• Closed as program error.

Error description

• The Tivoli Storage Manager server can hang during the upgradedb
  process when upgrading to any 5.5.X.X release on the Linux
  platform. After executing the upgrade install_server script, the
  command process can hang, and can only be terminated by issuing
  kill -xx <pid> command, where xx is the kill signal sent to
  terminate the process. To generate a core dump issue signal 11
  so the core file can be used to verify this APAR.
  During a normal upgrade of the Tivoli Storage Manager, the
  server will complete initialization process, and then issue the
  HALT command to end the upgrade process. Under an unknown
  circumstance there is a possibility that the HALT command will
  be issued before the Storage Manager can complete the
  initialization process, and this will block the upgrade from
  terminating.
  The core stack will show the main thread is not finished
  initializing duringa call to function dbUpgradeComplete when
  HALT is called, and is blocked in function LvmCheckpoint(). The
  call stack is as below:
  #0  0x00000200000c638e in pthread_cond_wait@@GLIBC_2.3.2 () from
  /lib64/tls/libpthread.so.0
  #1  0x000000008088481a in pkWaitConditionTracked ()
  #2  0x00000000804af9c0 in LvmCheckpoint ()
  #3  0x00000000804ace10 in lvmWriteRestartRecord ()
  #4  0x00000000802d9d50 in dbUpgradeComplete ()
  #5  0x00000000801396e4 in admStartServer ()
  #6  0x0000000080087be8 in main ()
  
  The current code is written so that function LvmCheckpoint()
  will be in a wait condition forever when the HALT command is in
  progress.
  If a core file is not available, or if the upgrade is currently
  in a hung state the APAR can be verified by checking the
  "tsm_upgrade.log". This log is located in the "server/bin"
  directory, and can be checked by looking for the following
  messages:
  
  1. ANR0129I Database upgrade completed successfully.
     Even though the upgrade hangs, this message shows that the
     upgrade is successful, and the 5.5.3.0 Storage Manager server
     is ready to use.
  2. ANR0993I Server initialization complete.
     This message is missing from the log file during a hang
     condition, and when missing means the server did not complete
     initialization.
  3. ANR2017I Administrator SERVER_CONSOLE issued command: HALT
     HALT is last message displayed in the log file during the
     hang, and should never be issued until initialization has
     completed.  This message should always follow message
     ANR0993I.
  4. ANR0991I Server shutdown complete.
     Server shutdown complete is always the last message
     displayed in the log and comes right after the HALT message.
     In a hang condition this message will not be displayed in the
     log.
  
  This hang condition has been seen on the zLinux platform, but
  could effect other Linux or Unix platforms, and other versions
  of the Tivoli Storage Manager. The problem is completely random
  and is not easily reproduced.
  Initial Impact: Low - the upgrade is successful.
  Additional Keywords: Hung Hang upgrade Linux install_server
  

Local fix

• None - the upgrade is successful and the Tivoli Storage Manager
  server is ready for use.
  

Problem summary

• ****************************************************************
  * USERS AFFECTED: All Tivoli Storage Manager server users.     *
  ****************************************************************
  * PROBLEM DESCRIPTION: See error description.                  *
  ****************************************************************
  * RECOMMENDATION: Apply fixing level when available. This      *
  *                 problem is currently projected to be fixed   *
  *                 in levels 5.5.5. Note that this is           *
  *                 subject to change at the discretion of IBM.  *
  ****************************************************************
  *
  

Problem conclusion

• This problem was fixed.
  Affected platforms:  AIX, HP-UX, Sun Solaris, Linux, z/OS, and
  Windows.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

  PK99375

Fix information

• Fixed component name

  TSM SERVER

• Fixed component ID

  5698ISMSV

Applicable component levels

• R55A PSY

     UP

• R55H PSY

     UP

• R55L PSY

     UP

• R55S PSY

     UP

• R55W PSY

     UP

• R55Z PSY

     UP