This is mostly adapted from this long thread on the VMware community site.
I got VMware ESXi 5.5.0 running on upstream KVM today.
First I had to disable the “VMware backdoor”. When VMware runs, it detects that qemu underneath is emulating this port and tries to use it to query the machine (instead of using CPUID and so on). Unfortunately qemu’s emulation of the VMware backdoor is very half-assed. There’s no way to disable it except to patch qemu:
diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c index eaf3e61..ca1c422 100644 --- a/hw/i386/pc_piix.c +++ b/hw/i386/pc_piix.c @@ -204,7 +204,7 @@ static void pc_init1(QEMUMachineInitArgs *args, pc_vga_init(isa_bus, pci_enabled ? pci_bus : NULL); /* init basic PC hardware */ - pc_basic_device_init(isa_bus, gsi, &rtc_state, &floppy, xen_enabled(), + pc_basic_device_init(isa_bus, gsi, &rtc_state, &floppy, 1, 0x4); pc_nic_init(isa_bus, pci_bus);
It would be nice if this was configurable in qemu. This is now being fixed upstream.
Secondly I had to turn off MSR emulation. This is, unfortunately, a machine-wide setting:
# echo 1 > /sys/module/kvm/parameters/ignore_msrs # cat /sys/module/kvm/parameters/ignore_msrs Y
Thirdly I had to give the ESXi virtual machine an IDE disk and an
network card. Note also that ESXi requires ≥ 2 vCPUs and at least 2 GB of RAM.