IBM Support

IT02590: CLIENT SESSIONS HUNG IN RUN STATE DUE TO SESSION IN MEDIAW

Subscribe

You can track all active APARs for this component.

APAR status

  • Closed as program error.

Error description

  • Client session is hung in MediaW state.  New client and admin
    sessions hang in the Run state and in time the server can run
    out of sessions preventing new client/admin sessions.  These
    symptoms are the same as APAR IC99934, but the cause and
    identifiers are different.
    
    Tivoli Storage Manager Versions Affected: All V62, V63 and V7
    Servers
    
    Customer/L2 Diagnostics:
    Query Session will show growing number of client sessions in the
    run state:
     Sess       Comm.    Sess     Wait    Sess
     Number     Method   State    Time    Type
     -------    ------   ------  ------   -------
     1,336,400  Tcp/Ip   SendW    0 S     Node
     1,337,000  Tcp/Ip   MediaW 4.7 M     Node
     1,337,301  Tcp/Ip   Run      0 S     Node
     1,337,302  Tcp/Ip   Run      0 S     Node
     1,337,303  Tcp/Ip   Run      0 S     Node
     1,337,304  Tcp/Ip   Run      0 S     Node
     1,337,305  Tcp/Ip   Run      0 S     Node
     1,337,306  Tcp/Ip   Run      0 S     Admin
     ...
    
    Collect the servermonV6.pl script data during the time you get
    traces.
    Although this problem is difficult to trap with a trace, the
    following flags can be used:
    TRACE ENABLE INSTR AS MMS PVRNOIO
    
    Tracing with just INSTR and PVR seemed to help trigger the
    problem more frequently.
    
    The traces will show a latch deadlock.
    
    The deadlocked threads will both have the same stack.
    Example:
    Thread 3631144, Parent 1048: psSessionThread, Storage 7797835,
    AllocCnt 1621 HighWaterAmt 7867735
      tid=828, ptid=2218, det=1, zomb=0, join=0, result=0,
    sess=871784
       Awaiting cond latchP->sFree (0x110cdc6f0), using mutex
    PVRV->mutex (0x111b8c148), at latch.c(256)
       Stack trace:
         0x09000000004b9ab0 _cond_wait_global
         0x09000000004ba63c _cond_wait
         0x09000000004bb32c pthread_cond_wait
         0x00000001000079f4 pkWaitConditionTracked
         0x000000010068a51c SmNodeSession
         0x000000010053e39c HandleNodeSession
         0x0000000100535cb4 DoNodeGeneral
         0x0000000100530ce0 smExecuteSession
         0x000000010017827c psSessionThread
         0x000000010000c264 StartThread
    
    Trace data:
    06:56:18.143 [3631144][latch.c][238][AcquireLatchSpecific]:LATCH
    Acquire ATTEMPTED for latch
    12740150, mode 0, using mutex 11b8c008 by requestor
    pvrclass.c(3631144).
    06:56:18.143 [3631144][latch.c][294][AcquireLatchSpecific]:LATCH
    Acquire SUCCESSFUL for latch
    12740150, mode 0, using mutex 11b8c008 by requestor
    pvrclass.c(3631144).
    06:56:18.142 [3631155][latch.c][238][AcquireLatchSpecific]:LATCH
    Acquire ATTEMPTED for latch
    12750d90, mode 1, using mutex 11b8c008 by requestor
    pvrmp.c(938)(3631155).   ----> Another thread
    requesting for Xlatch blocked the 3631144's Slatch request
    
    Initial Impact: High
    Additional Keywords: Hang hung Login failed session client
    backups fail
    

Local fix

  • Restart the Tivoli Storage Manager server
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED:                                              *
    * All Tivoli Storage Manager server users.                     *
    ****************************************************************
    * PROBLEM DESCRIPTION:                                         *
    * See ERROR DESCRIPTION.                                       *
    ****************************************************************
    * RECOMMENDATION:                                              *
    * Apply fixing level when available. This problem is currently *
    * projected to be fixed in levels 6.3.5 and 7.1.1. Note that   *
    * this is subject to change at the discretion of IBM.          *
    ****************************************************************
    

Problem conclusion

  • This problem was fixed.
    Affected platforms:  AIX, HP-UX, Solaris, Linux, and Windows.
    

Temporary fix

Comments

APAR Information

  • APAR number

    IT02590

  • Reported component name

    TSM SERVER

  • Reported component ID

    5698ISMSV

  • Reported release

    63A

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt

  • Submitted date

    2014-06-16

  • Closed date

    2014-06-23

  • Last modified date

    2014-06-23

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    TSM SERVER

  • Fixed component ID

    5698ISMSV

Applicable component levels

  • R63A PSY

       UP

  • R63H PSY

       UP

  • R63L PSY

       UP

  • R63S PSY

       UP

  • R63W PSY

       UP

  • R71A PSY

       UP

  • R71H PSY

       UP

  • R71L PSY

       UP

  • R71S PSY

       UP

  • R71W PSY

       UP



Document information

More support for: Tivoli Storage Manager

Software version: 63A

Reference #: IT02590

Modified date: 23 June 2014