IBM Support

IC99117: FREQUENT ANR0538I MESSAGES WITH HIGH AMOUNT OF "DB2 MFTCH EXEC" TIME DURING EXPIRATION

Subscribe

You can track all active APARs for this component.

APAR status

  • Closed as program error.

Error description

  • Frequent messages "ANR0538I A resource waiter has been aborted."
    may be reported in the actlog during client backups.
    This is especially the case during database intensive processes
    like EXPIRATION which do hold required locks for a longer time
     finally causing client backups to  terminate after the
    resourcetimeout has been  reached.
    
    It is not a best practice to let a client backup run
    concurrently with the expiration process but that may happen (as
    not possible otherwise) for a heavily used Tivoli Storage
    Manager server.
    
    A server instrumentation trace shows very slow "DB2 MFtch Exec"
    processing  for the expiration thread.
    
    Analysis has shown the following select as being responsible for
    this high value :
      "select BFID, OFFSET, LENGTH, OWNER,
    DIGEST_TYPE,DIGEST_VALUE,FLAGS,UNCOMP_LENGTH from
    BF_AGGREGATED_BITFILES where SRVID=? and SUPERBFID=? and
    OWNER=?"
    
    A special runstat needs to be done to allow DB2 to use a better
    access plan for this specific table:  BF_AGGREGATED_BITFILES
    
    Tivoli Storage Manager Versions Affected: All platforms  6.1  ,
    6.2 , 6.3 , 7.1
    
    Customer/L2 Diagnostics :
    
    - Servermon script instrumentation trace show the following
    statistics for the expiration thread :
    
    Thread 691414 ExpirationProcessThread parent=691411
    00:16:44.567-->00:41:08.037
     Operation       Count  Tottime  Avgtime  Mintime  Maxtime
    InstTput  Total KB
     ---------------------------------------------------------------
     DB2 Fetch Exec      5    0.082    0.017    0.000    0.042
     DB2 MFtch Exec      1 1391.037 1391.037 1391.037 1391.037
     DB2 Inser Exec 121537   70.183    0.001    0.000    2.580
     DB2 Updat Exec      5    0.082    0.017    0.000    0.042
     DB2 Fetch           5    0.000    0.000    0.000    0.000
     DB2 Commit          1    0.161    0.162    0.161    0.161
     DB2 Reg Prep        2    0.006    0.003    0.000    0.005
     DB2 Reg Exec        3    1.268    0.423    0.000    0.714
     DB2 Reg Fetch       3    0.000    0.000    0.000    0.000
     Tm Lock Wait        1    0.003    0.003    0.003    0.003
     Unknown                  0.644
     ---------------------------------------------------------------
     Total                 1463.470
    
    Please note the considerably high value of time being spent in
    "DB2 MFtch Exec".
    
    Initial Impact: Medium
    
    Additional Keywords:
    BFBF BFBE TSM
    

Local fix

  • To reduce the frequency of the ANR0538I messages to appear (may
    be not eliminate all of them as they can still appear due to
    other reasons) caused by the current runstats command, please
    do:
    
    - First be sure that the table BF_AGGREGATED_BITFILES and it's
      indexes does not need reorg.
      This technote explains how to check current reorg status for
      the tables and indexes using "db2 reorgchk" command.
      And it also explains how to do offline reorg if needed.
       "Database size, database reorganization, and performance
       considerations for Tivoli Storage Manager V6 and V7 servers"
       http://www.ibm.com/support/docview.wss?uid=swg21452146
    
    - Run the following runstats command (the command may run when
      server is up and running) :
     db2 "runstats on table TSMDB1.BF_AGGREGATED_BITFILES  ON ALL
     COLUMNS AND COLUMNS ((OWNER,SUPERBFID)) with distribution and
     sampled detailed  indexes all"
     The server should not be halted until the runstats completes.
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED:                                              *
    * All Tivoli Storage Manager server users.                     *
    ****************************************************************
    * PROBLEM DESCRIPTION:                                         *
    * See ERROR DESCRIPTION.                                       *
    ****************************************************************
    * RECOMMENDATION:                                              *
    * Apply fixing level when available.                           *
    * This problem is currently projected to be fixed in levels    *
    * 6.3.6 and 7.1.1.                                             *
    * Note that this is subject to change at the discretion of     *
    * IBM.                                                         *
    ****************************************************************
    

Problem conclusion

  • This problem was fixed.
    Affected platforms:  AIX, HP-UX, Solaris, Linux, and Windows.
    

Temporary fix

Comments

APAR Information

  • APAR number

    IC99117

  • Reported component name

    TSM SERVER

  • Reported component ID

    5698ISMSV

  • Reported release

    63L

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt

  • Submitted date

    2014-02-03

  • Closed date

    2014-04-16

  • Last modified date

    2014-04-16

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    TSM SERVER

  • Fixed component ID

    5698ISMSV

Applicable component levels

  • R63A PSY

       UP

  • R63H PSY

       UP

  • R63L PSY

       UP

  • R63S PSY

       UP

  • R63W PSY

       UP

  • R71A PSY

       UP

  • R71H PSY

       UP

  • R71L PSY

       UP

  • R71S PSY

       UP

  • R71W PSY

       UP



Document information

More support for: Tivoli Storage Manager

Software version: 63L

Reference #: IC99117

Modified date: 16 April 2014