Subnodes go offline due to (503)Service Unavailable errors.
The VMware VI agent subnodes go offline on the Tivoli Enterprise Portal and the kvm_data_provider_<instance>_<number>,log file will have messages similar to:
java.lang.Object: (503)Service Unavailable
The "(503)Service Unavailable" message indicates that the agent lost connectivity to the data source, usually the VMware vCenter. This might happen for a variety of reasons, and the agent is designed to recover from such errors. The recovery might fail due to APARs IZ92794 and IZ93826, so ensure you are running at or above the following levels:
VMware VI Agent V6.2.2 with Interim Fix 6.2.2-TIV-ITM_VMWVI-IF0003
VMware VI Agent V6.2.3
Release and Interim Fix information is here:
If the kvm_data_provider_<instance>_<number>,log file has numerous occurrences of the
"java.lang.Object: (503)Service Unavailable" message and the agent is running on the same Windows Server 2003 SP2 machine as the vCenter, it might indicate a Microsoft TCP problem and the customer should install a Microsoft hotfix:
This is also documented on the VMware site because it can affect any program that uses the VMware SDK:
Note that this Microsoft TCP problem occurs only when the agent is running on the same machine as the vCenter. The problem can be resolved by applying the Microsoft hotfix or by moving the agent off the vCenter machine.