IBM Support

IC94804: V6.3 SERVER ON AIX POWER 7 SYSTEMS CAN EXPERIENCE CLIENT PERFORM ANCE DEGRADATION.

Subscribe

You can track all active APARs for this component.

APAR status

  • Closed as program error.

Error description

  • Server users on AIX Power 7 (P7) systems can experience client
    backup and restore performance degradation.  Systems with a
    high rate of client connection activity (for example, thousands
    of sessions from HSM, TDP, CM, and CMOD workloads) are more
    likely to to be affected by this performance issue.
    
    This problem can cause client backup and restore operations
    to take significantly longer than normal.
    
    Customer/L2 Diagnostics:
    The most notable behavior will be an increase in concurrently
    active client sessions activity reported by the QUERY SESSION
    administrative command.  The following diagnostics can then
    be used to confirm this problem:
    
    1. Issue the SHOW THREADS administrative command against the
       Tivoli Storage Manager server every 5 minutes for 15
       minutes.
    2. Look for threads within the output generated that include
       the ICC_RAND_bytes function that end in the
       _global_lock_common function.  For example, there should be
       numerous threads that look similar to the following:
    
        Thread 4122, Parent 70: psSessionThread, Storage 169122,
        AllocCnt 12 HighWaterAmt 174272
          tid=4222, ptid=2922, det=1, zomb=0, join=0, result=0,
          sess=100924
        Stack trace:
          0x09000000007d37b8 _global_lock_common <---------------
          0x09000000007e0c80 _mutex_lock
          0x09000000031a8dac OSSL_locking_callback
          0x0900000003204f88 C98E_CRYPTO_lock
          0x09000000031ad69c efCRYPTO_lock
          0x09000000031dd974 fips_rand_bytes
          0x090000000321ca60 C98E_RAND_bytes
          0x09000000031b4914 METAC_RAND_bytes@AF377_226
          0x090000000129975c ICCC_RAND_bytes@AF323_193
          0x0900000001295110 ICCC_RAND_bytes
          0x090000000128b858 ICC_RAND_bytes      <---------------
          0x000000010014291c GenRandomKey
          .....
          0x000000010000c264 StartThread
    
    3. Each of the above threads will be waiting on a single
       thread, also including the ICC_RAND_bytes function, but
       this time ending with the getbyte() function.  For
       example, there should also be a single thread that looks
       similar to the following:
    
       Thread 1234, Parent 14: psSessionThread, Storage 152138,
       AllocCnt 60 HighWaterAmt 152138
         tid=9014, ptid=2946, det=1, zomb=0, join=0, result=0,
         sess=70983
       Stack trace:
           0x09000000031c9b20 getbyte            <---------------
           0x09000000031c89f8 trng_raw_gen
           0x09000000031c79bc CLiC_trng
           0x09000000031caa60 efGenerateRandomSeed
           0x09000000031c7028 PRNG_GenerateRandomSeed
           0x09000000031c71fc efRNG_ReSeed
           0x09000000031c74d8 OldefRNG_Generate
           0x09000000031c64b8 efRNG_Generate
           0x09000000031dd9b0 fips_rand_bytes
           0x090000000321ca60 C98E_RAND_bytes
           0x09000000031b4914 METAC_RAND_bytes
           0x090000000129975c ICCC_RAND_bytes
           0x0900000001295110 ICCC_RAND_bytes
           0x090000000128b858 ICC_RAND_bytes     <---------------
           0x000000010014291c GenRandomKey
           .....
           0x000000010000c264 StartThread
    
    Initial Impact:
    High
    
    Tivoli Storage Manager Versions Affected:
    All V6.3 Tivoli Storage Manager server users on AIX Power 7.
    
    Additional Keywords:
    ZZ63 gskit getbyte performance perf slow backups client upgrade
    pssessionthread session sessions ICC_RAND_BYTES GenRandomKey
    

Local fix

  • An emergency fix is available.  Contact Support for assistance.
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED: All Tivoli Storage Manager server users.     *
    ****************************************************************
    * PROBLEM DESCRIPTION: See ERROR DESCRIPTION.                  *
    ****************************************************************
    * RECOMMENDATION: Apply fixing level when available. This      *
    *                 problem is currently projected to be fixed   *
    *                 in level 6.3.5. Note that this is            *
    *                 subject to change at the discretion of IBM.  *
    ****************************************************************
    *
    

Problem conclusion

Temporary fix

Comments

APAR Information

  • APAR number

    IC94804

  • Reported component name

    TSM SERVER

  • Reported component ID

    5698ISMSV

  • Reported release

    63A

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt

  • Submitted date

    2013-08-12

  • Closed date

    2013-08-22

  • Last modified date

    2013-09-26

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    TSM SERVER

  • Fixed component ID

    5698ISMSV

Applicable component levels

  • R63A PSY

       UP



Document information

More support for: Tivoli Storage Manager

Software version: 63A

Reference #: IC94804

Modified date: 26 September 2013