Tag Archives: windows

Use hivex from Python to read and write Windows Registry “hive” files

I added Python bindings to hivex today.

Here is an example using Python, libguestfs and hivex to download the user preferences registry from a Windows virtual machine and print out the Internet Explorer start page for a particular user. When you run it, it should print out something like:

User rjones's IE home page is http://go.microsoft.com/fwlink/?LinkId=69157

This example shows downloading and printing values, but libguestfs and hivex can also be used to make changes (but not to live guests).

#!/usr/bin/python

import guestfs
import hivex

# The name of a Windows virtual machine on this host.  This
# example script makes some assumptions about the registry
# location and contents which only apply on Windows Vista
# and later versions.
windows_domain = "Win7x32"

# Username on the Windows VM.
username = "rjones"

# Use libguestfs to download the HKEY_CURRENT_USER hive.
g = guestfs.GuestFS ()
g.add_domain (windows_domain, readonly=1)
g.launch ()

roots = g.inspect_os ()
root = roots[0]
g.mount_ro (root, "/")

path = "/users/%s/ntuser.dat" % username
path = g.case_sensitive_path (path)
g.download (path, "/tmp/ntuser.dat")

# Use hivex to pull out a registry key.
h = hivex.Hivex ("/tmp/ntuser.dat")

key = h.root ()
key = h.node_get_child (key, "Software")
key = h.node_get_child (key, "Microsoft")
key = h.node_get_child (key, "Internet Explorer")
key = h.node_get_child (key, "Main")

val = h.node_get_value (key, "Start Page")
start_page = h.value_value (val)
#print start_page

# The registry key is encoded as UTF-16LE, so reencode it.
start_page = start_page[1].decode ('utf-16le').encode ('utf-8')

print "User %s's IE home page is %s" % (username, start_page)

7 Comments

Filed under Uncategorized

New home for (RH) SrvAny

http://github.com/beekhof/mingw32-srvany (thanks Andrew Beekhof for organizing this). Previously …

Leave a comment

Filed under Uncategorized

Tip: Get the hostname of a guest

Because different operating systems store the hostname in different places, you have to know in advance what sort of OS your guest is (perhaps using virt-inspector). Perhaps we should add the hostname to virt-inspector.

This works for Fedora guests:

# virt-cat F13x64 /etc/sysconfig/network | \
  grep HOSTNAME= | \
  awk -F= '{print $2}'
f13x64.home.annexia.org

This is for Debian/Ubuntu guests:

# virt-cat Debian5x64 /etc/hostname
debian5x64.home.annexia.org

For Windows guests:

# virt-win-reg Win7x32 \
  'HKEY_LOCAL_MACHINE\System\ControlSet001\Services\Tcpip\Parameters' \
  Hostname
win7x32

I’m not completely clear how to get the DNS domain name from Windows. According to this article you should just replace “Hostname” with “Domain” in the above command, but for me that yields just an empty string.

4 Comments

Filed under Uncategorized

Windows SAM and hivex

On Windows, the file C:\windows\system32\config\SAM contains the users and passwords known to the local machine. hivex can process this file to reveal the usernames and password (hashes):

$ virt-win-reg WinGuest HKLM\\SAM > sam.reg

For each local user you’ll see a key like this:

[HKEY_LOCAL_MACHINE\SAM\SAM\Domains\Account\Users\Names\rjones]
@=hex(3e9):

With typical technical brilliance Microsoft developers have written a zero-length key with the type field (0x3e9) overloaded as a key to use in another part of the registry:

[HKEY_LOCAL_MACHINE\SAM\SAM\Domains\Account\Users\000003E9]
"F"=hex(3):omitted...
"UserPasswordHint"=str(3):"usual"
"V"=hex(3):omitted...

(Apparently the number 0x3e9 is called the “RID” in Microsoft parlance).

My password hint is the “usual”. The “F” key is a dumped C structure containing the last login date amongst other things. The “V” key is another C structure containing my full name, home directory, the password hash and a bunch of other stuff.

With a bit of effort it looks like you could read and even modify these entries.

1 Comment

Filed under Uncategorized

Tip: Install a device driver in a Windows VM

Previously we looked at how to install a service in a Windows VM. You can use that technique or the RunOnce tip to install some device drivers too.

But what if Windows needs the device driver in order to boot? This is the problem we faced with converting old Xen and VMWare guests to use KVM. You can’t install viostor (the virtio disk driver) which KVM needs either on the source Xen/VMWare hypervisors (because those don’t use the virtio standard) or on the destination KVM hypervisor (because Windows needs to be able to see the disk first in order to be able to boot).

Nevertheless we can modify the Windows VM off line using libguestfs to install the virtio device driver and allow it to boot.

(Note: virt-v2v will do this for you. This article is for those interested in how it works).

There are three different aspects to installing a device driver in Windows. Two of these are Windows Registry changes, and one is to install the .SYS file (the device driver itself).

So first we make the two Registry changes. Device drivers are a bit like services under Windows, so the first change looks like installing a service in a Windows guest. The second Registry change adds viostor to the “critical device database”, a map of PCI addresses to device drivers used by Windows at boot time:

# virt-win-reg --merge Windows7x64

;
; Add the viostor service
;

[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\viostor]
"Group"="SCSI miniport"
"ImagePath"=hex(2):73,00,79,00,73,00,74,00,65,00,6d,00,33,00,32,00,5c,00,64,\
  00,72,00,69,00,76,00,65,00,72,00,73,00,5c,00,76,00,69,00,6f,00,73,00,74,00,6f,\
  00,72,00,2e,00,73,00,79,00,73,00,00,00
"ErrorControl"=dword:00000001
"Start"=dword:00000000
"Type"=dword:00000001
"Tag"=dword:00000040

[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\viostor\Parameters]
"BusType"=dword:00000001

[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\viostor\Parameters\MaxTransferSize]
"ParamDesc"="Maximum Transfer Size"
"type"="enum"
"default"="0"

[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\viostor\Parameters\MaxTransferSize\enum]
"0"="64  KB"
"1"="128 KB"
"2"="256 KB"

[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\viostor\Parameters\PnpInterface]
"5"=dword:00000001

[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\viostor\Enum]
"0"="PCI\\VEN_1AF4&DEV_1001&SUBSYS_00021AF4&REV_00\\3&13c0b0c5&2&20"
"Count"=dword:00000001
"NextInstance"=dword:00000001

;
; Add viostor to the critical device database
;

[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\CriticalDeviceDatabase\PCI#VEN_1AF4&DEV_1001&SUBSYS_00000000]
"ClassGUID"="{4D36E97B-E325-11CE-BFC1-08002BE10318}"
"Service"="viostor"

[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\CriticalDeviceDatabase\PCI#VEN_1AF4&DEV_1001&SUBSYS_00020000]
"ClassGUID"="{4D36E97B-E325-11CE-BFC1-08002BE10318}"
"Service"="viostor"

[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\CriticalDeviceDatabase\PCI#VEN_1AF4&DEV_1001&SUBSYS_00021AF4]
"ClassGUID"="{4D36E97B-E325-11CE-BFC1-08002BE10318}"
"Service"="viostor"

Comparatively speaking, the second step of uploading viostor.sys to the right place in the image is simple:

# guestfish -i Windows7x64
><fs> upload viostor.sys /Windows/System32/drivers/viostor.sys

After that, the Windows guest can be booted on KVM using virtio. In virt-v2v we then reinstall the viostor driver (along with other drivers like the virtio network driver) so that we can be sure they are all installed correctly.

8 Comments

Filed under Uncategorized

Tip: Install a service in a Windows VM

Previously I discussed how to get a script to run the first time a user logs in. This tip goes further and demonstrates how to install a service into a Windows VM using guestfish, virt-win-reg and a new open source program written by my colleague Yuval Kashtan called RHSrvAny1.

First, compile RHSrvAny from source. You can do this using our completely free Fedora Windows cross-compiler stack. Just:

# yum install mingw32-gcc

Clone the RHSrvAny git repo and compile it:

$ mingw32-configure
$ make

Second we’ll copy the files we need into the Windows guest. Note: The Windows VM must be shut off.

# guestfish -i Windows7x64
Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.

Type: 'help' for a list of commands
      'man' to read the manual
      'quit' to quit the shell

><fs> upload RHSrvAny/rhsrvany.exe /rhsrvany.exe
><fs> upload test.exe /test.exe
><fs> exit

“test.exe” is a little program I wrote which writes the date into C:\TEST.LOG but you can also use the batch file from the last tip or any JScript or VBScript you happen to have (via cscript.exe).

Third we need to add some Windows Registry keys to tell Windows about the new service:

# cat service.reg
[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\services\RHSrvAny]
"Type"=dword:00000010
"Start"=dword:00000002
"ErrorControl"=dword:00000001
"ImagePath"="c:\\rhsrvany.exe"
"DisplayName"="RHSrvAny"
"ObjectName"="LocalSystem"

[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\services\RHSrvAny\Parameters]
"CommandLine"="c:\\test.exe"
"PWD"="c:\\Temp"
# virt-win-reg --merge Windows7x64 service.reg

The magic numbers in the registry entries let you do things like boot with the service disabled. See this MSDN article.

Edit: See Yuval’s comment about alternatives to using "ObjectName"="LocalSystem".

Now boot your Windows guest, and observe the log file to prove that test.exe was run, and/or look at the list of services in the control panel.

><fs> cat /TEST.LOG
Hello
Thu Apr 29 18:39:13 2010

1 Actually you could install any service, but I’m using RHSrvAny because it can turn ordinary Windows programs and scripts into services. It takes care of the Windows “Service Control Protocol” for us.

8 Comments

Filed under Uncategorized

Tip: Get a Windows VM to run a batch file at boot

With the virt-win-reg tool built on top of libguestfs and hivex it’s now relatively straightforward to modify a Windows virtual machine so that it runs a batch file, script or program at next boot.

Note: The Windows VM must be shut down before you attempt this.

The plan is that we upload the batch script to some place in the VM, and then add a “RunOnce” key in the Windows Registry (explained in this MSDN article and this article). First let’s just take a look at what’s in the key. In most cases it will be empty:

# virt-win-reg Windows7x64 \
    'HKLM\Software\Microsoft\Windows\CurrentVersion\RunOnce'
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnce]

Now we’ll prepare our batch file and upload it:

# cat test.bat
ECHO HELLO > C:\TEST.LOG
TIME /T >> C:\TEST.LOG
# guestfish -i Windows7x64

Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.

Type: 'help' for a list of commands
      'man' to read the manual
      'quit' to quit the shell

><fs> upload test.bat /test.bat
><fs> ^D

And finally we modify the RunOnce registry key:

# virt-win-reg --merge Windows7x64
[HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\RunOnce]
"Test"="c:\\test.bat"
^D

One potential gotcha: You must be running hivex ≥ 1.2.2.

Now you can boot your Windows guest and check that the script runs after the user logs in. Look for the file C:\TEST.LOG:

><fs> cat /TEST.LOG
HELLO 
09:21

Because we’re using the RunOnce key, the script will run just one time. If you want it to run every time, use the Run key.

Now, how do we make the script work without the user needing to log in? (Clue: The answer is not RunServicesOnce — this does not work in Windows 7). What’s surprising (coming from a Linux background) is the huge amount of incomplete, contradictory and simply false information contained in MSDN about this topic.

8 Comments

Filed under Uncategorized

Tip: Use guestfish ‘du’ to get disk usage for a directory

Virt-df lets you see disk usage of your virtual machines by partition.

What if you want to see the usage for a part of the directory hierarchy, eg. everything under /home or in /usr/share/doc? There is no “virt-du” command, but you can use the “du” command in guestfish:

$ echo du /usr/share/doc | guestfish --ro -i Fedora12
282304

The number printed is in kilobytes.

This also works, with a small modification, for Windows VMs:

$ echo du 'win:C:\windows\system32' | guestfish --ro -i Win2003x32
990022

2 Comments

Filed under Uncategorized

Use hivex to unpack a Windows Boot Configuration Data (BCD) hive

Thanks to “TJ” for tipping me off about another use of the Registry “hive” format in recent versions of Windows.

There are scant details available, but if you have a version of Windows Vista or later, then the boot loader is no longer configured through a plain text file (“BOOT.INI”) but via a binary blob. Microsoft provides a tool called “BCDEDIT.EXE” that you are supposed to use to edit this, but the blob is a hive so you can use hivex to display or modify it.

We first use guestfish to download the blob:

$ guestfish --ro -a /dev/vg_trick/Windows7x64 -m /dev/sda1

Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.

Type: 'help' for help with commands
      'quit' to quit the shell

><fs> ll /
total 392
drwxrwxrwx  1 root root   4096 Dec 15 04:48 .
dr-xr-xr-x 20 root root      0 Mar 30 13:30 ..
-rwxrwxrwx  1 root root   8192 Dec 15 12:47 BOOTSECT.BAK
drwxrwxrwx  1 root root   4096 Dec 15 12:47 Boot
drwxrwxrwx  1 root root      0 Dec 15 04:48 System Volume Information
-rwxrwxrwx  1 root root 383562 Jul 13  2009 bootmgr
><fs> ll /Boot/
total 596
drwxrwxrwx 1 root root   4096 Dec 15 12:47 .
drwxrwxrwx 1 root root   4096 Dec 15 04:48 ..
-rwxrwxrwx 1 root root  24576 Mar 25 12:25 BCD
-rwxrwxrwx 1 root root  21504 Mar 25 12:25 BCD.LOG
-rwxrwxrwx 2 root root      0 Dec 15 12:47 BCD.LOG1
-rwxrwxrwx 2 root root      0 Dec 15 12:47 BCD.LOG2
-rwxrwxrwx 1 root root  65536 Dec 15 12:47 BOOTSTAT.DAT
[snipped]
><fs> download /Boot/BCD /tmp/BCD
><fs> ^D

Then we can dump the contents out using hivexregedit. (We could also browse the contents with hivexsh).

$ hivexregedit --export /tmp/BCD '\' > /tmp/BCD.reg

In typical Microsoft style, the contents themselves are obscure, consisting of plenty of subkeys that look like this:

[\Objects\{1afa9c49-16ab-4a5c-901b-212802da9460}\Elements\14000006]
"Element"=hex(7):7b,00,37,00,65,00,61,00,32,00,65,00,31,00,61,00,63,\
  00,2d,00,32,00,65,00,36,00,31,00,2d,00,34,00,37,00,32,00,38,00,2d,00,\
  61,00,61,00,61,00,33,00,2d,00,38,00,39,00,36,00,64,00,39,00,64,00,30,\
  00,61,00,39,00,66,00,30,00,65,00,7d,00,00,00,00,00

(Note that “type 7″ is a list of strings, and the whole thing is encoded in UTF-16LE, so this requires some further work to parse).

There’s scope here to extend virt-inspector to understand this stuff, or even to write a BCDEDIT-style tool to modify the way Window VMs boot. Apparently the current BCDEDIT tool is half-arsed, so here’s another opportunity to beat Microsoft’s own tooling.

2 Comments

Filed under Uncategorized

Edit the Windows Registry in your VMs from the host

Update: Want to test this out? There are packages for RHEL / CentOS 5 here.

For a while we have shipped a tool virt-win-reg which lets you read keys out from the Registry. The top feature people have asked for is the ability to make changes in the Registry, and we have now implemented this (tracking bug 575738).

With the virtual machine shutdown (live merges are not supported), you prepare a text file describing the changes in “.REG” format. For example:

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\RWMJ]
@="RWMJ"
"Key1"=dword:ff123456
"Key2"="This is key 2"

and simply merge that into the Registry:

virt-win-reg --merge Windows7x32 /tmp/updates.reg

After booting Windows, we can see the new subkey has appeared:

Now I change my updates file slightly:

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\RWMJ]
"Key1"=-

(which means “delete Key1″). I have to shut down Windows, run the same virt-win-reg command, and start Windows again.

At this point Windows decides to punish me by demanding “activation” (I believe this event is entirely separate from the Registry change). Red Hat pays lots of money each year to Microsoft for genuine MSDN licenses so we can do this sort of interop testing, to improve the utility of Windows, and this Windows installation is fully licensed. Nevertheless, there is a bug in this version of Windows which means it can never be activated because activation runs before the network setup. So I’m punished with a black desktop and a stern warning. This makes me hate Windows just a little bit more than before …

Anyway, the key has been removed:

Finally I put this into an updates file:

-[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\RWMJ]

And after shutting down Windows, running virt-win-reg and rebooting Windows, we see that the Registry key has been removed entirely:

Leave a comment

Filed under Uncategorized