IBM Support

IT07398: MOVE NODEDATA AND QUERY PROCESS COMMANDS CAN TRIGGER SERVER CRAS H

Subscribe

You can track all active APARs for this component.

APAR status

  • Closed as program error.

Error description

  • The Tivoli Storage Manager server can crash when a MOVE
    NODEDATA process is active and an administrator issues the
    QUERY PROCESS command.
    
    This can only happen within a rare timing window and does not
    always occur.  This timing window becomes larger when there
    are a significant number source volumes required to complete
    the MOVE NODEDATA operation.  Also, the crash is more likely
    to occur the sooner that the QUERY PROCESS command is issued
    after the MOVE NODEDATA process has begun.
    
    Customer/L2 Diagnostics:
    The following failing callstack can be extracted from the
    dump file generated (Linux example):
    
      strlen ()
      OrderInsert ()
      OutGetMsg ()
      outFmtMsgf ()
      AfMoveDataQuery ()
      procQueryProcess ()
      AdmCommandLocal ()
      admCommand ()
      SmAdminCommandThread ()
      StartThread ()
      start_thread ()
      clone ()
    
    Initial Impact:
    High
    
    Tivoli Storage Manager Versions Affected:
    All supported Tivoli Storage Manager servers
    
    Additional Keywords:
    TSM CRASH ABEND SEG FAULT AFMOVEDATAQUERY ORDERINSERT PROC
    Q PROCESS AFMOVENODEDATATHREAD MOVE NODE DATA
    

Local fix

  • Do not issue the QUERY PROCESS command until after a MOVE
    NODEDATA process has completed (which can be verified based
    on activity log messaging).
    
    Alternatively, use system tools like gstack or pstack to ensure
    that the AfMoveNodeDataThread() thread is not still in the
    AsGetPool() function before issuing the QUERY PROCESS command:
    
      ...
      pkAcquireMutexTracked ()
      tmLockTracked ()
      AsLockVolIdTracked ()
      AsGetPoolStatus ()
      asGetPoolAttr ()
      AfUpdatePool ()
      AfGetPool ()               <-------------------
      AfMoveNodeDataThread ()    <-------------------
      StartThread ()
      ...
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED:                                              *
    * All Tivoli Storage Manager server users.                     *
    ****************************************************************
    * PROBLEM DESCRIPTION:                                         *
    * See error description.                                       *
    ****************************************************************
    * RECOMMENDATION:                                              *
    * Apply fixing level when available. This problem is currently *
    * projected to be                                              *
    * fixed in levels 6.3.6, 7.1.1.200 and 7.1.3. Note that this   *
    * is subject to change                                         *
    * at the discretion of IBM.                                    *
    ****************************************************************
    

Problem conclusion

  • This problem was fixed.
    Affected platforms:  AIX, HP-UX, Solaris, Linux, and Windows.
    

Temporary fix

Comments

APAR Information

  • APAR number

    IT07398

  • Reported component name

    TSM SERVER

  • Reported component ID

    5698ISMSV

  • Reported release

    63L

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2015-02-27

  • Closed date

    2015-03-05

  • Last modified date

    2015-03-05

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    TSM SERVER

  • Fixed component ID

    5698ISMSV

Applicable component levels

  • R63A PSY

       UP

  • R63H PSY

       UP

  • R63L PSY

       UP

  • R63S PSY

       UP

  • R63W PSY

       UP

  • R71A PSY

       UP

  • R71H PSY

       UP

  • R71L PSY

       UP

  • R71S PSY

       UP

  • R71W PSY

       UP



Document information

More support for: Tivoli Storage Manager

Software version: 63L

Reference #: IT07398

Modified date: 05 March 2015