Tip: Enable minidumps in a Windows guest

You can use virt-win-reg to enable minidumps in Windows guests. Quite easily as it happens.

First prepare a file crashcontrol.reg containing:

; NB: This assumes CurrentControlSet == ControlSet001
; See "CurrentControlSet etc." in virt-win-reg(1)

[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\CrashControl]
"AutoReboot"=dword:00000000
"CrashDumpEnabled"=dword:00000003
"DumpFile"=str(2):"%SystemRoot%\MEMORY.DMP"
"LogEvent"=dword:00000001
"MinidumpDir"=str(2):"%SystemRoot%\Minidump"
"MinidumpsCount"=dword:00000032
"Overwrite"=dword:00000001

The key fields are AutoReboot, which you probably want to set to 0 to stop the guest from automatically rebooting when it gets a BSOD, and CrashDumpEnabled for which you can read the docs here.

Then import this into the guest (which must not be running):

$ virt-win-reg --merge GuestName crashcontrol.reg

Ruby bindings for Hivex

Hivex is a library for reading and writing Windows Registry “hive” files. New in version 1.3.0 and Fedora 16 is the ability to access the library from Ruby.

As an example, first grab some hive files from a Windows virtual machine. The simplest way is using virt-copy-out:

# virt-copy-out -a win.img \
    'win:c:\windows\system32\config' .
# ls config/
...
SOFTWARE
SYSTEM
...

Using the following Ruby script you can extract and display registry keys from the hive files:

#!/usr/bin/ruby

require 'hivex'

h = Hivex::open("config/SOFTWARE", {})

# Use this instead if you want to make changes:
# h = Hivex::open("config/SOFTWARE", { :write => 1 })

root = h.root()
node = h.node_get_child(root, "Microsoft")
if node.nil? then
  puts "no HKLM\\SOFTWARE\\Microsoft node: Probably not the correct hive"
end

node = h.node_get_child(node, "Windows NT")
node = h.node_get_child(node, "CurrentVersion")
val = h.node_get_value(node, "ProductName")

hash = h.value_value(val)
puts "Windows product name:", hash[:value]

Tip: Using a backing file to record file and registry changes, addendum

Part 1, part 2, and part 3

Inspired by the tricky and slow method to pull out file metadata that I showed in part 2 I added some features to virt-ls to make this much easier. These features are not yet in virt-ls. You will either have to apply this patch series or wait for libguestfs ≥ 1.11.9.

With the forthcoming virt-ls -lR option you will be able to extract the file metadata from a virtual machine easily. The output format is designed so that simple grep patterns can be used to detect interesting things in the output.

For example to display the names of all setuid and setgid files in the VM:

# virt-ls -lR -d guest / | grep '^- [42]'
- 4755      12544 /bin/cgexec -
- 4755      32448 /bin/fusermount -
- 4755      78648 /bin/mount -
- 4755      43160 /bin/ping -
- 4755      47888 /bin/ping6 -
- 4755      34904 /bin/su -
- 4755      50432 /bin/umount -
[...]

To display all public writable directories:

# virt-ls -lR -d guest / | grep '^d ...7'
d 1777      12288 /tmp -
d 1777       4096 /tmp/.ICE-unix -
d 1777       4096 /tmp/.X11-unix -
d 1777       4096 /var/tmp -

To display files larger than 10MB in home directories:

# virt-ls -lR -d guest /home | awk '$3 >= 10*1024*1024'

Find regular files modified in the last 24 hours:

# virt-ls -lR -d guest --time-days / |
    grep '^-' |
    awk '$6 < 1'
[...]
- 0600        138   0   0   0 /home/rjones/.Xauthority
- 0600         69   0   0   0 /root/.xauthsdYvWC
- 0444         11   0   0   0 /tmp/.X0-lock
[...]

Also filesystem comparisons are made much simpler. So to display changes in files between a snapshot and the latest version of a VM you would simply do:

# virt-ls -lR -a snapshot.img / --uids --time-t --checksum > old
# virt-ls -lR -a current.img / --uids --time-t --checksum > new
# diff -u old new | less

Tip: Using a backing file to record file and registry changes, part 3

In the first part I showed you that you can use libguestfs directly on QEMU snapshots and backing files, and this can be used to forensically look at changes made to virtual machines, eg. by installing new software. In the second part yesterday I showed you how to look for files that have changed.

Today we’ll look at differences in the Windows registry.

Although the Windows registry is stored in hive files, because these files are complex binary structures it makes sense to examine them at a higher level using a library like hivex, or the high level tool I wrote virt-win-reg.

Using virt-win-reg we can examine each hive that virt-win-reg supports separately, and use “diff” to list differences:

$ virt-win-reg backing.qcow2 'HKLM\SYSTEM' > system.without-chrome
$ virt-win-reg win7.qcow2 'HKLM\SYSTEM' > system.with-chrome
$ virt-win-reg backing.qcow2 'HKLM\SOFTWARE' > software.without-chrome
$ virt-win-reg win7.qcow2 'HKLM\SOFTWARE' > software.with-chrome

There were no significant changes to the HKEY_LOCAL_MACHINE\SYSTEM hive, but Chrome made many additions to the SOFTWARE hive, for example:

+[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\ChromeHTML]
+@=str(1):"Chrome HTML Document"
+"URL Protocol"=str(1):""
+
+[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\ChromeHTML\DefaultIcon]
+@=str(1):"C:\Users\rjones\AppData\Local\Google\Chrome\Application\chrome.exe,0"
+
+[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\ChromeHTML\shell]
+
+[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\ChromeHTML\shell\open]
+
+[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\ChromeHTML\shell\open\command]
+@=str(1):"\"C:\Users\rjones\AppData\Local\Google\Chrome\Application\chrome.exe\" -- \"%1\""

Note that you can make the output more readable by using the --unsafe-printable-strings option (but note that it’s called “unsafe” for a reason).

Chrome appears to have modified the list of CA certificates — should I be worried?

There is an addendum to this coming tomorrow morning.

Tip: Using a backing file to record file and registry changes, part 2

In the first part I showed you that you can use libguestfs directly on QEMU snapshots and backing files, and this can be used to forensically look at changes made to virtual machines, eg. by installing new software.

Given the backing file and changes from yesterday, we can also look at differences in file content and permissions on files — in other words, what existing files did the Google Chrome installer change in the Windows guest?

A simple and easy way to do this is to instruct guestfish to export the whole filesystem as a tarball, which we’ll use on the host to compare file permissions and sizes:

$ guestfish --ro -i -a backing.qcow2 tar-out / - |
    tar tvvf - | sort -k6 > files.without-chrome
$ guestfish --ro -i -a win7.qcow2 tar-out / - |
    tar tvvf - | sort -k6 > files.with-chrome
$ diff -u files.without-chrome files.with-chrome | less
[... 1,677 lines of output ...]
--rwxrwxrwx root/root    1157779 2011-06-02 11:06 ./Windows/WindowsUpdate.log
+-rwxrwxrwx root/root    1221259 2011-06-03 10:04 ./Windows/WindowsUpdate.log

What did I find out? As well as adding its own files to AppData/Local, Chrome also made changes to all of the registry hives, and as a result of installing new files, many internal Windows indexes were updated.

This technique will miss several changes which might be important to you: it will miss files that have had their content changed, but the file size and modification date didn’t change. And it will omit changes to extended attributes such as file forks, NT symbolic links and so on. If you want to list those changes too, you have to get down and dirty in the libguestfs API. This posting would be a good place to start.

Tomorrow we’ll look at changes in the Windows registry.

Tip: Using a backing file to record file and registry changes, part 1

Gary asked if it is possible to examine a KVM snapshot or backing file and perhaps list out the files and so on that had changed between the backing file and the current image.

It’s possible to use libguestfs to examine the changes, and in this three part series I’ll show you how.

I want to examine the file and Windows registry changes that happen when I install Google Chrome for Windows.

I first set up a Windows guest with a backing file, and I made sure the backing file was committed just before Chrome was downloaded and installed:

$ ll win7.qcow2 backing.qcow2 
-rw-r--r--. 1 qemu qemu 10099228672 Jun  3 10:40 backing.qcow2
-rw-r--r--. 1 root root    60555264 Jun  3 10:40 win7.qcow2

Then I installed Chrome in the guest, and as you can see the win7.qcow2 file (containing just changes) is much larger while the backing file has stayed the same:

$ ll win7.qcow2 backing.qcow2 
-rw-r--r--. 1 qemu qemu 10099228672 Jun  3 10:40 backing.qcow2
-rw-r--r--. 1 root root   682164224 Jun  3 11:08 win7.qcow2

Getting a list of files that have been added or removed by installing Chrome is easy. Note that this does not show files that have been modified (we’ll get to that in the next part). Note #2 because of a bug in WordPress, you have to type “backslash zero” where it says “NUL” below.

$ guestfish --ro -i -a win7.qcow2 find0 / - |
    tr 'NUL' '\n' | sort > files.with-chrome
$ guestfish --ro -i -a backing.qcow2 find0 / - |
    tr 'NUL' '\n' | sort > files.without-chrome
$ diff -u files.without-chrome files.with-chrome |
    less
[...]
+Users/rjones/AppData/Local/Google
+Users/rjones/AppData/Local/Google/Chrome
+Users/rjones/AppData/Local/Google/Chrome/Application
+Users/rjones/AppData/Local/Google/Chrome/Application/11.0.696.71
+Users/rjones/AppData/Local/Google/Chrome/Application/11.0.696.71/avcodec-52.dll
[...]
 Users/rjones/Desktop
 Users/rjones/Desktop/desktop.ini
+Users/rjones/Desktop/Google Chrome.lnk
 Users/rjones/Documents
 Users/rjones/Documents/desktop.ini

Google Chrome doesn’t (or can’t?) install anything under Program Files, instead preferring to install itself completely within AppData/Local in the user’s home directory.

In the next part I’ll show you how to find out when file contents, size or permissions have changed, and in the third part, we’ll look at Windows registry changes.

virt-win-reg can now read and write HKEY_USERS (Windows user preferences)

In libguestfs 1.11.7 (and some further patches in git) ≥ 1.11.8 you can now use virt-win-reg to list out and update Windows user preferences stored in HKEY_USERS keys.

There are two ways to use this. You can either specify a User SID like:

virt-win-reg Windows 'HKEY_USERS\S-1-5-19\Software\Policies'

or (as a libguestfs extension) you can specify a local user name directly:

virt-win-reg Windows 'HKEY_USERS\rjones\Control Panel'

Tip: Change the background image in a Windows VM

Thanks to Tom Horsley who worked out how to do this for Windows XP guests (the technique is probably different for other versions of Windows).

Here is Tom’s script and here are more of his KVM tips.

hivex 1.2.5 released

The latest version of hivex — the library for extracting and modifying Windows Registry hive files has been released. You can get the source from here.

I spent a lot of time examining real hive files from Windows machines and running the library under the awesome valgrind tool, and found one or two places where a corrupt hive file could cause hivex to read uninitialized memory. It’s not clear to me if these are security issues — I think they are not — but everyone is advised to upgrade to this version anyway.

hivex would be a great candidate for fuzz testing if anyone wants to try that.

Tip: Code for getting DHCP address from a virtual machine disk image

Previously (1) and previously (2) I showed there are many different ways to get the IP address from a virtual machine.

The example below shows one way to use libguestfs and hivex from a C program (virt-dhcp-address) to get the DHCP address that a virtual machine has picked up.