No Data Due to java.lang.Object: (503)Service Unavailable

The VMware VI agent subnodes go offline on the Tivoli Enterprise Portal and the kvm_data_provider_<instance>_<number>,log file will have messages similar to:

java.lang.Object: (503)Service Unavailable

The "(503)Service Unavailable" message indicates that the agent lost connectivity to the data source, usually the VMware vCenter. This might happen for a variety of reasons, and the agent is designed to recover from such errors. The recovery might fail due to APARs IZ92794 and IZ93826, so ensure you are running at or above the following levels:

VMware VI Agent V6.2.2 with Interim Fix 6.2.2-TIV-ITM_VMWVI-IF0003
or
VMware VI Agent V6.2.3

Release and Interim Fix information is here:

http://www.ibm.com/developerworks/wikis/display/tivolimonitoring/VMware+VI+Agent

If the kvm_data_provider_<instance>_<number>,log file has numerous occurrences of the
"java.lang.Object: (503)Service Unavailable" message and the agent is running on the same Windows Server 2003 SP2 machine as the vCenter, it might indicate a Microsoft TCP problem and the customer should install a Microsoft hotfix:

http://support.microsoft.com/kb/979230

This is also documented on the VMware site because it can affect any program that uses the VMware SDK:

http://kb.vmware.com/selfservice/documentLink.do?externalID=1021873

Note that this Microsoft TCP problem occurs only when the agent is running on the same machine as the vCenter. The problem can be resolved by applying the Microsoft hotfix or by moving the agent off the vCenter machine.