Guest post: Converting VMware guests to libvirt/KVM guests

This is a guest post by Marko Myllynen.

VMware guests can be converted as Libvirt/QEMU/KVM guests without any need for proprietary VMware tools. There are two ways to do the conversion, use the one appropriate for your environment.

Converting On-Disk VMware Guests to Libvirt Guests

This method is suitable if you have locally the VMDK image file (e.g., the Virtual XP image from local Service Desk).

First, install needed packages:

# yum install qemu-img libguestfs-tools libguestfs-winsupport virsh

[RWMJ notes: libguestfs-winsupport package is only needed on RHEL 6 hosts]

Then, you’ll need to acquire the Windows registry entries described in Microsoft KB article 314082 and adjust them for use with virt-win-reg (that is, replace all CurrentControlSet instances with ControlSet001 and make sure the file is properly encoded – see the virt-win-reg(1) manual page for details). The resulting file is also available here. The steps below expect that the entries are available in the file named ./mergeide.reg.

Also, you’ll need to create a domain XML definition file for your new libvirt guest. Please download this file as ./virtual-xp.xml and adjust the following fields as needed:

• name
• uuid
• memory / currentMemory
• arch
• mac address

Note: you *must* set the full path of the virtual image inside the XML (look for the source file definition).

Finally, the actual conversation can be done in three steps:

# qemu-img convert virtual-xp.vmdk virtual-xp.img
# virt-win-reg --merge virtual-xp.img mergeide.reg
# virsh --connect qemu:///system define virtual-xp.xml

After this you should see your new XP VM in your virt-manager display.

Converting VMware Guests from ESX Server to Libvirt Guests

This method is suitable if your VMware guests are on an ESX server. Please install virt-v2v and see the manual page virt-v2v(1) for details.

---

Thanks Marko for this excellent tip!

Tip: Get the hostname of a guest

Because different operating systems store the hostname in different places, you have to know in advance what sort of OS your guest is (perhaps using virt-inspector). Perhaps we should add the hostname to virt-inspector.

This works for Fedora guests:

# virt-cat F13x64 /etc/sysconfig/network | \
  grep HOSTNAME= | \
  awk -F= '{print $2}'
f13x64.home.annexia.org

This is for Debian/Ubuntu guests:

# virt-cat Debian5x64 /etc/hostname
debian5x64.home.annexia.org

For Windows guests:

# virt-win-reg Win7x32 \
  'HKEY_LOCAL_MACHINE\System\ControlSet001\Services\Tcpip\Parameters' \
  Hostname
win7x32

I’m not completely clear how to get the DNS domain name from Windows. According to this article you should just replace “Hostname” with “Domain” in the above command, but for me that yields just an empty string.

Windows SAM and hivex

On Windows, the file C:\windows\system32\config\SAM contains the users and passwords known to the local machine. hivex can process this file to reveal the usernames and password (hashes):

$ virt-win-reg WinGuest HKLM\\SAM > sam.reg

For each local user you’ll see a key like this:

[HKEY_LOCAL_MACHINE\SAM\SAM\Domains\Account\Users\Names\rjones]
@=hex(3e9):

With typical technical brilliance Microsoft developers have written a zero-length key with the type field (0x3e9) overloaded as a key to use in another part of the registry:

[HKEY_LOCAL_MACHINE\SAM\SAM\Domains\Account\Users\000003E9]
"F"=hex(3):omitted...
"UserPasswordHint"=str(3):"usual"
"V"=hex(3):omitted...

(Apparently the number 0x3e9 is called the “RID” in Microsoft parlance).

My password hint is the “usual”. The “F” key is a dumped C structure containing the last login date amongst other things. The “V” key is another C structure containing my full name, home directory, the password hash and a bunch of other stuff.

With a bit of effort it looks like you could read and even modify these entries.

Tip: Install a service in a Windows VM

Previously I discussed how to get a script to run the first time a user logs in. This tip goes further and demonstrates how to install a service into a Windows VM using guestfish, virt-win-reg and a new open source program written by my colleague Yuval Kashtan called RHSrvAny¹.

First, compile RHSrvAny from source. You can do this using our completely free Fedora Windows cross-compiler stack. Just:

# yum install mingw32-gcc

Clone the RHSrvAny git repo and compile it:

$ mingw32-configure
$ make

Second we’ll copy the files we need into the Windows guest. Note: The Windows VM must be shut off.

# guestfish -i Windows7x64
Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.

Type: 'help' for a list of commands
      'man' to read the manual
      'quit' to quit the shell

><fs> upload RHSrvAny/rhsrvany.exe /rhsrvany.exe
><fs> upload test.exe /test.exe
><fs> exit

“test.exe” is a little program I wrote which writes the date into C:\TEST.LOG but you can also use the batch file from the last tip or any JScript or VBScript you happen to have (via cscript.exe).

Third we need to add some Windows Registry keys to tell Windows about the new service:

# cat service.reg
[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\services\RHSrvAny]
"Type"=dword:00000010
"Start"=dword:00000002
"ErrorControl"=dword:00000001
"ImagePath"="c:\\rhsrvany.exe"
"DisplayName"="RHSrvAny"
"ObjectName"="LocalSystem"

[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\services\RHSrvAny\Parameters]
"CommandLine"="c:\\test.exe"
"PWD"="c:\\Temp"
# virt-win-reg --merge Windows7x64 service.reg

The magic numbers in the registry entries let you do things like boot with the service disabled. See this MSDN article.

Edit: See Yuval’s comment about alternatives to using "ObjectName"="LocalSystem".

Now boot your Windows guest, and observe the log file to prove that test.exe was run, and/or look at the list of services in the control panel.

><fs> cat /TEST.LOG
Hello
Thu Apr 29 18:39:13 2010

¹ Actually you could install any service, but I’m using RHSrvAny because it can turn ordinary Windows programs and scripts into services. It takes care of the Windows “Service Control Protocol” for us.

Tip: Get a Windows VM to run a batch file at boot

With the virt-win-reg tool built on top of libguestfs and hivex it’s now relatively straightforward to modify a Windows virtual machine so that it runs a batch file, script or program at next boot.

Note: The Windows VM must be shut down before you attempt this.

The plan is that we upload the batch script to some place in the VM, and then add a “RunOnce” key in the Windows Registry (explained in this MSDN article and this article). First let’s just take a look at what’s in the key. In most cases it will be empty:

# virt-win-reg Windows7x64 \
    'HKLM\Software\Microsoft\Windows\CurrentVersion\RunOnce'
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnce]

Now we’ll prepare our batch file and upload it:

# cat test.bat
ECHO HELLO > C:\TEST.LOG
TIME /T >> C:\TEST.LOG
# guestfish -i Windows7x64

Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.

Type: 'help' for a list of commands
      'man' to read the manual
      'quit' to quit the shell

><fs> upload test.bat /test.bat
><fs> ^D

And finally we modify the RunOnce registry key:

# virt-win-reg --merge Windows7x64
[HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\RunOnce]
"Test"="c:\\test.bat"
^D

One potential gotcha: You must be running hivex ≥ 1.2.2.

Now you can boot your Windows guest and check that the script runs after the user logs in. Look for the file C:\TEST.LOG:

><fs> cat /TEST.LOG
HELLO 
09:21

Because we’re using the RunOnce key, the script will run just one time. If you want it to run every time, use the Run key.

Now, how do we make the script work without the user needing to log in? (Clue: The answer is not RunServicesOnce — this does not work in Windows 7). What’s surprising (coming from a Linux background) is the huge amount of incomplete, contradictory and simply false information contained in MSDN about this topic.

Edit the Windows Registry in your VMs from the host

Update: Want to test this out? There are packages for RHEL / CentOS 5 here.

For a while we have shipped a tool virt-win-reg which lets you read keys out from the Registry. The top feature people have asked for is the ability to make changes in the Registry, and we have now implemented this (tracking bug 575738).

With the virtual machine shutdown (live merges are not supported), you prepare a text file describing the changes in “.REG” format. For example:

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\RWMJ]
@="RWMJ"
"Key1"=dword:ff123456
"Key2"="This is key 2"

and simply merge that into the Registry:

virt-win-reg --merge Windows7x32 /tmp/updates.reg

After booting Windows, we can see the new subkey has appeared:

Now I change my updates file slightly:

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\RWMJ]
"Key1"=-

(which means “delete Key1″). I have to shut down Windows, run the same virt-win-reg command, and start Windows again.

At this point Windows decides to punish me by demanding “activation” (I believe this event is entirely separate from the Registry change). Red Hat pays lots of money each year to Microsoft for genuine MSDN licenses so we can do this sort of interop testing, to improve the utility of Windows, and this Windows installation is fully licensed. Nevertheless, there is a bug in this version of Windows which means it can never be activated because activation runs before the network setup. So I’m punished with a black desktop and a stern warning. This makes me hate Windows just a little bit more than before …

Anyway, the key has been removed:

Finally I put this into an updates file:

-[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\RWMJ]

And after shutting down Windows, running virt-win-reg and rebooting Windows, we see that the Registry key has been removed entirely:

Tip: virt-win-reg: CurrentControlSet in Windows Registry

I was asked today why this command doesn’t work (this Registry key would be visible if you were inside the Windows guest):

$ virt-win-reg Win2003x32 \
  '\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Keyboard Layout\DosKeybCodes'
hivexget: \CurrentControlSet: CurrentControlSet: path element not found

It’s because CurrentControlSet (and several other “Current*” keys) are synthetic. They don’t exist in the underlying Registry “hive” (file), but are created by Windows when it is running to refer to the currently selected setting for the local user (This isn’t quite correct. For the specifics refer to the Microsoft KB article and this stackoverflow posting).

Instead you have to refer to one of the possible selections. Usually ControlSet001 will work, so:

$ virt-win-reg Win2003x32 \
  '\HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Keyboard Layout\DosKeybCodes'
"00000402"="bg"
"00000404"="ch"
"00000405"="cz"
[etc]

virt-win-reg: get at the Windows Registry in your Windows guests

This is now in libguestfs git and will appear in version 1.0.75 in Fedora shortly.

 $ virt-win-reg MyWinGuest \
   '\HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion' \
   ProductName
 Microsoft Windows Server 2003

 $ virt-win-reg MyWinGuest \
   '\HKEY_LOCAL_MACHINE\System\ControlSet001\Control' \
   SystemBootDevice
 multi(0)disk(0)rdisk(0)partition(1)

 $ virt-win-reg MyWinGuest \
   '\HKEY_LOCAL_MACHINE\System\ControlSet001\Control'
 "CurrentUser"="USERNAME"
 "WaitToKillServiceTimeout"="20000"
 "SystemStartOptions"="NOEXECUTE=OPTOUT  FASTDETECT"
 "SystemBootDevice"="multi(0)disk(0)rdisk(0)partition(1)"

hivexget: Get values from a Windows Registry hive

$ hivexget system '\ControlSet001\Control'
"PreshutdownOrder"=hex(7):77,00,75,00,61[...]
"WaitToKillServiceTimeout"="12000"
"CurrentUser"="USERNAME"
"BootDriverFlags"=dword:00000000"
"ServiceControlManagerExtension"=str(2):"%systemroot%\\system32\\scext.dll"
"SystemStartOptions"=" NOEXECUTE=OPTIN"
"SystemBootDevice"="multi(0)disk(0)rdisk(0)partition(2)"
"FirmwareBootDevice"="multi(0)disk(0)rdisk(0)partition(1)"
$ hivexget system '\ControlSet001\Control' SystemBootDevice
multi(0)disk(0)rdisk(0)partition(2)

There is also a tool (hivexml) to convert the registry hive into an XML file.

These are low-level tools at the moment. These are the basis for writing a nice, usable, high-level virt-win-reg program for grabbing values out of a Windows guest.

libhivex: Windows Registry hive extractor library

Several people managing Windows virtual machines have told me that libguestfs/virt-cat isn’t enough for them. They’d like to be able to get at Windows Registry entries in the guest.

A typical example is the imaginary [as of now] virt-win-reg command that lets you interrogate the Registry in a guest:

$ virt-win-reg MyWinGuest '\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion'
"ProductName"="Microsoft Windows Server 2003"
"RegisteredOwner"="Richard Jones"
"CurrentVersion"="5.2"
"CurrentBuildNumber"="3790"
[etc]

Right now you can only do this indirectly, by laboriously downloading the registry hives and decoding them with tools such as reged. I outlined how to do that here but there’s no doubt that it should be made a lot easier.

The first step is a more reliable way to query registry files themselves. The files come from foreign, buggy, possibly malicious guests, and so code that touches them must be written carefully and conservatively to avoid security problems.

Another problem is that the tools in this area tend to convert the binary, proprietary “hive” format into a regedit-compatible text format. The problem is that regedit itself is no easier to parse. What we would like is a more compatible format — XML or a library.

There are several existing tools to do this. The best is certainly Petter Nordahl’s chntpw utility which we’ve been carrying in Fedora for a while. Unfortunately Petter hasn’t been answering our queries about issues in the code and we are concerned that the code isn’t cautious enough to deal with an onslaught of malicious registry files. Another is the BSD-licensed dumphive program written in Pascal.

To address our concerns I have spent the last three days writing a simpler version of Petter’s program called libhivex. This library and associated programs are able to extract the contents of Windows Registry “hive” files, and make this available through a simple C API or as XML. The library is written very defensively and should deal with malicious files. The scope of the library is also being kept intentionally small: we won’t use it to modify these files ever, just to extract data from them.

I hope to publish a patch series for this soon for libguestfs, followed by some useful command line tools to let sysadmins get data from their Windows virtual machines.

Got a suggestion for a useful libguestfs-related tool? Let me know in the comments.