At work, I'm currently deploying a Windows 2008 R2 on a VMware ESXi 4.0, and noticed an odd behaviour. At irregular intervals, the console of the guest OS simply locks up and doesn't do anything anymore. Networking and remote access seem to work fine, it's just the console that is freezed. Restarting the vSphere client doesn't help, and the only way to get rid of the problem is by hard-resetting the VM. The issue only occurs on Windows 2008 R2, the other VMs run fine.
The problem is caused by the VMware SVGA II driver, and has two underlying issues: the driver itself, and an apparent lack of video RAM (due to the new HW-accelerated GDI of 2008R2/Win7).
The most quickest way is to get rid of the VMware SVGA driver but this makes console responsiveness incredibly slow. Therefore, it's not the best solution.
There are several things to do: increasing the video RAM, and upgrade/replace the driver with a newer (and alternate) version. Increasing the RAM stabilized the problem in my case, although I did not wait long enough to see whether the problem was permanently fixed. Either way, I recommend updating the driver as well as the new VMware SVGA 3D driver seriously increases performance over the old one.
In a quiet moment, turn off the VM and change its video adapter settings. For some reason, they were set to a fixed of 4MB, which is too low for proper operating 2008R2.
Set the memory size to 32MB, then start the VM again.
Additionally, try to install the new VMware SVGA 3D driver to resolve and increase your performance in the process.
To upgrade/replace the VMware SVGA II driver with the new VMware SVGA 3D driver, follow instructions: