ESX/ESXi host keeps disconnecting and reconnecting when heartbeats are not received by vCenter Server

Symptoms

• ESX/ESXi hosts disconnects frequently
• vCenter Server shows ESX/ESXi host(s) as not responding
• ESX/ESXi host not responding in vCenter Server
• vCenter Server randomly not receiving ESX/ESXi heartbeats
• When verbose logging is enabled, the vpxd logs contain entries similar to:
  [2011-08-18 11:18:09.929 04068 verbose ‘App’] [VpxdIntHost] Missed 2 heartbeats for host esx03.it.test.local
• The vpxa logs contain entries similar to:
  [VpxaMoService::GetChangesInt] Forcing a full host synclastSentMasterGen = 67 MasterGenNo from vpxd = 66
  [VpxaMoService::GetChangesInt] Vpxa restarted or stolen by other server. Start a full sync

Cause

ESX/ESXi hosts send heartbeats every 10 seconds. vCenter Server has a window of 60 seconds to receive the heartbeats. If the UDP heartbeat message is not received by vCenter Server, it treats the host as not responding.This behavior can be an indication of a congested network between the ESX/ESXi host and vCenter server.

Resolution

To work around this issue, increase the timeout limit in vCenter Server to make the ESX host continuously connected. Increasing the timeout is a short-term solution until the network issues can be resolved.

To increase the timeout limit:  

1. Open the C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter\vpxd.cfg file (on Windows 2008, C:\ProgramData\VMware\VMware VirtualCenter\vpxd.cfg ) using a text editor.
2. Add this information in the <vpxd> tags:
   <heartbeat>
   <notRespondingTimeout>120</notRespondingTimeout>
   </heartbeat> 
3. Restart the VMware VirtualCenter Server service.