IV27549: DISCOVERY HANG DUE TO DATABASE LOCK.

APAR status

• Closed as program error.

Error description

• Discovery hang, due to all running topo pumps blocked at the
  database.  This can be seen in the logs and database activity
  reports. For example, on the DiscoveryServer, one might look at
  DiscoverObserver.log and see that the topopumps are in a storage
  state for some time;
  
  2012-08-07 19:12:41,561 DiscoverObserver [TopoPump1]  DEBUG
  
  topomgr.TopologyManagerFactory - Invoking
  persistQueuedModelObject on
  storage server......
  
  Nothing further is seen on TopoPump1 after starting to persist a
  sensor.
  
  In the above example, TopoPump1 has been storing for some time,
  the same case can be found on all 16 topopumps(16 is default
  topopumpcount, some customers may configure this differently)
  that they started to persist but never returned.  Reviewing
  database data such as a DB2 snapshot will show a DB lock for
  quite some time(it usually can take an hour + for all the
  threads to lock once the DB lock occurs). The lock could be on
  any data table related to CI storage, there is no specific table
  this issue occurs with.
  

Local fix

• Restart TADDM.
  

Problem summary

• Discovery hang, due to all running topo pumps blocked at the
  database. This can
  be seen in the logs and database activity reports. For example,
  on the
  DiscoveryServer, one might look at DiscoverObserver.log and see
  that the
  topopumps are in a storagestate for some time; 2012-08-07
  19:12:41,561
  DiscoverObserver [TopoPump1] DEBUG
  topomgr.TopologyManagerFactory - Invoking
  persistQueuedModelObject on storage server...... Nothing further
  is seen on
  TopoPump1 after starting to persist asensor. In the above
  example, TopoPump1 has
  been storing for some time, the same case can be found on all 16
  topopumps(16 is
  default topopumpcount, some customers may configure this
  differently) that they
  started to persist but never returned. Reviewing database data
  such as a DB2
  snapshot will show a DB lock for quite some time(it usually can
  take an hour +
  for all the threads to lock once the DB lock occurs). The lock
  could be on any
  data table related to CI storage, there is no specific tablethis
  issue occurs
  with.
  

Problem conclusion

• The fix for this APAR is contained in the following maintenance
  packages:
  | Fix Pack | 7.2.1-TIV-ITADDM-FP0004
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  APP DEPENDENCY

• Fixed component ID

  5724N5500

Applicable component levels

• R721 PSY

     UP