I just patched a couple of ESXi hosts to 4.0.0 build 256968 and found that VMware Tools were no longer loading after upgrading them. After some digging everything is more or less back to normal.
Turns out that insserv(8) isn’t happy on Ubuntu 9.10 or 10.04, so now vmware-config-tools.pl falls back to using update-rc.d(8). However, since old rc.d links have been left behind, update-rc.d won’t do anything at all. On top of that vmware-config-tools.pl calls it incorrectly.
Before running vmware-config-tools.pl or vmware-tools-upgrader first remove all the rc.d links:
cd /etc rm -f rc?.d/*vmware-tools
On one of my systems I had multiple links at each run level, with different priorities, left behind from earlier VMware Tools installations. I guess regular manual inspection of the state of the links is warranted.
You may also want to apply this patch to vmware-config-tools.pl before running it:
--- vmware-config-tools.pl.orig 2010-06-06 19:45:49.176241344 -0400 +++ vmware-config-tools.pl 2010-06-06 20:25:56.000000000 -0400 @@ -1495,7 +1495,7 @@ if ($gHelper{'update-rc.d'} ne '') { if (0 == system(shell_string($gHelper{'update-rc.d'}) . " " . $service . " start " . $S_level . " S ." - . " start " . $K_level . " 0 6 .")) { + . " stop " . $K_level . " 0 6 .")) { return; } }
If your Linux kernel is 2.6.32 or later, the vmci module probably isn’t compiling for you either. Unpack the sources:
cd /tmp tar -xvf /usr/lib/vmware-tools/modules/source/vmci.tar
Then apply this patch:
--- vmci-only.old/vmciKernelIf.c 2010-05-07 00:11:18.000000000 -0400 +++ vmci-only/vmciKernelIf.c 2010-06-06 20:39:40.693377830 -0400 @@ -44,6 +44,7 @@ #include "compat_page.h" #include "compat_mm.h" #include "compat_highmem.h" +#include "compat_sched.h" #include "vm_basic_types.h" #include "pgtbl.h" #include <linux/vmalloc.h>
Then replace the source tar with the patched files:
cd /tmp tar -cvf /usr/lib/vmware-tools/modules/source/vmci.tar vmci-only
Now run vmware-config-tools.pl and everything should compile fine and you should have good start and stop links in your rc.d directories. Note that a start link is only installed in run level S: that’s okay, as according to the documentation that level is always run on boot. Seems to hold true, as the tools are now loaded fine on my systems.