IV54339: ADD SERVICEABILITY ENHANCEMENTS TO THE KPX/KRA (IRA) CODE.

IBM Tivoli Monitoring 6.3.0 Fix Pack 3 (6.3.0-TIV-ITM-FP0003)

APAR status

• Closed as new function.

Error description

• Implement Serviceability changes in the TEMS IRA code to isolate
  problems identified with object deletion.  We have observed that
  the IRA code on the TEMS reports an Invariant exception when
  a RequestImpl object attempts to release possession of a mutex
  lock that has been either previously released by another thread
  or corrupted by a memory overwrite.
  
  When this Invariant exception occurs we find the following trace
  messages appear in the TEMS log,
  
  (52B7AE56.0072-7:krabulck.cpp,203,"CTIRA_RecursiveLock_UnLock")
  Attempt UnLock Lock 106946FC TID 1e1fb70 held 0 waiters 0
  (52B7AE56.0073-7:kraburas.cpp,44,"ctira_throw_invariant") CTIRA
  Invariant Violated: Mon Dec 23 03:30:30 2013
  +52B7AE56.0073
  (52B7AE56.0074-7:kraburas.cpp,45,"ctira_throw_invariant")
  Invariant (held && pthread_equal(tid, pthread_self())) in file
  krabulck.cpp at line 205.
  
  The consequence of this Invariant exception may eventually
  result in a "SitMon Busy" condition which renders the TEMS
  inoperable to monitor Situations.
  

Local fix

Problem summary

• Add serviceability enhancements, in the form of additional
  tracing statements, to the KPX/KRA components of IBM Tivoli
  Monitoring. Problems related to the use of recursive locks in
  the Tivoli Enterprise Monitoring Proxy are difficult to resolve
  with the current trace statements.
  

Problem conclusion

• Trace statements have been added to the Proxy code to make it
  easier to identify problems related to the use of the recursive
  locks.
  
  Additionally, there is an Invariant condition which can arise
  when a recursive lock is prematurely released which results in a
  "SitMon Busy" condition where the monitoring server is executing
  but no longer monitoring situations. The code has been changed
  to immediately terminate the Monitoring Server and produce a
  core file which can be examined by support  to provide useful
  information to help find the root cause of the Invariant
  condition.
  
  A new environment variable IGNOREHELD has been added  and when
  set to "Y"  will have the code work the same way it did prior to
  this APAR change when encountering the "SitMon Busy" condition.
  
  
  The fix for this APAR is contained in the following maintenance
  packages:
  
    | fix pack | 6.3.0-TIV-ITM-FP0003
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

  IV62665 OA45672

Fix information

• Fixed component name

  TEMS

• Fixed component ID

  5724C04MS

Applicable component levels

• R630 PSY

     UP