Tip: Convert a Windows DVD ISO to a bootable USB key using guestfish

Situation: You have a Windows DVD (or ISO), but like any sane person in 2013 you don’t have a DVD drive on the computer. You want to convert the Windows DVD into a bootable USB key. There are many recipes for this online, but they all require another Windows machine and of course cannot be automated.

However with guestfish (and the always brilliant SYSLINUX doing most of the heavy lifting), this script will unpack the ISO and turn it into a bootable USB key.

Notes:

1. I am not going to support this script. You will need to read the script, look up the commands in the guestfish man page, and understand what it does. Any requests for help will be deleted unread.
2. You need to edit the USB key device before using the script.
3. You need libguestfs ≥ 1.21 with SYSLINUX support compiled in.

#!/bin/bash -

guestfish <<'EOF'
trace on

add-ro en_microsoft_hyper-v_server_2012_x64_dvd_915600.iso

# NB: The next line MUST be changed to your USB drive.
# ANYTHING ON THIS DRIVE WILL BE OVERWRITTEN WITHOUT WARNING.
add /dev/sdX

run

# Inside the appliance, /dev/sda = DVD, /dev/sdb = USB.
# THESE ARE NOT RELATED TO HOST DISK NAMES.

echo "Partitioning the USB disk ..."
part-init /dev/sdb mbr
part-add /dev/sdb p 63 -1
part-set-mbr-id /dev/sdb 1 0xb
part-set-bootable /dev/sdb 1 true
mkfs vfat /dev/sdb1

echo "Copying the contents of the DVD to the USB key ..."
mkmountpoint /cd
mkmountpoint /usb
mount /dev/sda /cd
mount /dev/sdb1 /usb
# XXX We should add cp-r command XXX
debug sh "cp -rP /sysroot/cd/* /sysroot/usb"
#glob cp-a /cd/* /usb
umount /cd
umount /usb
rmmountpoint /cd
rmmountpoint /usb

echo "Making the USB key bootable using SYSLINUX ..."
syslinux /dev/sdb1
mount /dev/sdb1 /
upload /usr/share/syslinux/chain.c32 /chain.c32
write /syslinux.cfg "DEFAULT windows\n\nLABEL windows\nCOM32 chain.c32\nAPPEND fs ntldr=/bootmgr\n"
umount /dev/sdb1
upload /usr/share/syslinux/mbr.bin /dev/sdb

echo "Finished."

EOF

A quick note about 64 bit Fedora Windows cross-compiler

I don’t often talk about the Fedora Windows cross-compiler project which I guess I helped to start back in 2008.

However last week some legal uncertainty about the 64 bit compiler was lifted and Erik van Pienbroek and Kalev Lember have been doing an amazing job getting a full 32 + 64 bit cross-compiler chain into Fedora 17.

The upshot of this is incredible: From a single code-base, you can build, on Fedora, a Linux binary, 32- and 64- bit Windows binaries, and (with a tiny bit of proprietary code) Mac OS X binaries. This includes cross-platform graphical programs (using Gtk), and over a hundred libraries for graphics, sound, games, networking, databases and much more. It’s not just plain binaries either, mingw-w64 includes a credible DDK replacement so you can build Windows device drivers too.

Android is the new Windows …

You want to know if Android or iPhone is going to “win” in the end? It’s obviously going to be Android, because it’s the new Windows.

Exhibit (a): My Google Nexus S phone: requires rebooting every 3 days otherwise it just stops being able to receive/send calls and/or texts. Data over 3g still works during these outages.

Exhibit (b): My new Samsung Galaxy tab, a Christmas present of sorts from Red Hat: Fresh out of the box(!), this cannot connect to my wifi (absolutely everything else can connect fine to this wifi). Support forums are full of people reporting the same thing, and everyone suggesting you reboot/reinstall/press some magic key sequence/etc. Where have I heard that story before?

libguestfs 1.12.6 for Debian

Thanks to the tireless work of Hilko Bengen, libguestfs 1.12.6 is now available as an official Debian package.

Also, you can compile hivex on Mac OS X and Windows, thanks to Alex Nelson and Gillen Daniel respectively.

Tip: Using a backing file to record file and registry changes, addendum

Part 1, part 2, and part 3

Inspired by the tricky and slow method to pull out file metadata that I showed in part 2 I added some features to virt-ls to make this much easier. These features are not yet in virt-ls. You will either have to apply this patch series or wait for libguestfs ≥ 1.11.9.

With the forthcoming virt-ls -lR option you will be able to extract the file metadata from a virtual machine easily. The output format is designed so that simple grep patterns can be used to detect interesting things in the output.

For example to display the names of all setuid and setgid files in the VM:

# virt-ls -lR -d guest / | grep '^- [42]'
- 4755      12544 /bin/cgexec -
- 4755      32448 /bin/fusermount -
- 4755      78648 /bin/mount -
- 4755      43160 /bin/ping -
- 4755      47888 /bin/ping6 -
- 4755      34904 /bin/su -
- 4755      50432 /bin/umount -
[...]

To display all public writable directories:

# virt-ls -lR -d guest / | grep '^d ...7'
d 1777      12288 /tmp -
d 1777       4096 /tmp/.ICE-unix -
d 1777       4096 /tmp/.X11-unix -
d 1777       4096 /var/tmp -

To display files larger than 10MB in home directories:

# virt-ls -lR -d guest /home | awk '$3 >= 10*1024*1024'

Find regular files modified in the last 24 hours:

# virt-ls -lR -d guest --time-days / |
    grep '^-' |
    awk '$6 < 1'
[...]
- 0600        138   0   0   0 /home/rjones/.Xauthority
- 0600         69   0   0   0 /root/.xauthsdYvWC
- 0444         11   0   0   0 /tmp/.X0-lock
[...]

Also filesystem comparisons are made much simpler. So to display changes in files between a snapshot and the latest version of a VM you would simply do:

# virt-ls -lR -a snapshot.img / --uids --time-t --checksum > old
# virt-ls -lR -a current.img / --uids --time-t --checksum > new
# diff -u old new | less

Tip: Using a backing file to record file and registry changes, part 3

In the first part I showed you that you can use libguestfs directly on QEMU snapshots and backing files, and this can be used to forensically look at changes made to virtual machines, eg. by installing new software. In the second part yesterday I showed you how to look for files that have changed.

Today we’ll look at differences in the Windows registry.

Although the Windows registry is stored in hive files, because these files are complex binary structures it makes sense to examine them at a higher level using a library like hivex, or the high level tool I wrote virt-win-reg.

Using virt-win-reg we can examine each hive that virt-win-reg supports separately, and use “diff” to list differences:

$ virt-win-reg backing.qcow2 'HKLM\SYSTEM' > system.without-chrome
$ virt-win-reg win7.qcow2 'HKLM\SYSTEM' > system.with-chrome
$ virt-win-reg backing.qcow2 'HKLM\SOFTWARE' > software.without-chrome
$ virt-win-reg win7.qcow2 'HKLM\SOFTWARE' > software.with-chrome

There were no significant changes to the HKEY_LOCAL_MACHINE\SYSTEM hive, but Chrome made many additions to the SOFTWARE hive, for example:

+[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\ChromeHTML]
+@=str(1):"Chrome HTML Document"
+"URL Protocol"=str(1):""
+
+[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\ChromeHTML\DefaultIcon]
+@=str(1):"C:\Users\rjones\AppData\Local\Google\Chrome\Application\chrome.exe,0"
+
+[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\ChromeHTML\shell]
+
+[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\ChromeHTML\shell\open]
+
+[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\ChromeHTML\shell\open\command]
+@=str(1):"\"C:\Users\rjones\AppData\Local\Google\Chrome\Application\chrome.exe\" -- \"%1\""

Note that you can make the output more readable by using the --unsafe-printable-strings option (but note that it’s called “unsafe” for a reason).

Chrome appears to have modified the list of CA certificates — should I be worried?

There is an addendum to this coming tomorrow morning.

Tip: Using a backing file to record file and registry changes, part 2

In the first part I showed you that you can use libguestfs directly on QEMU snapshots and backing files, and this can be used to forensically look at changes made to virtual machines, eg. by installing new software.

Given the backing file and changes from yesterday, we can also look at differences in file content and permissions on files — in other words, what existing files did the Google Chrome installer change in the Windows guest?

A simple and easy way to do this is to instruct guestfish to export the whole filesystem as a tarball, which we’ll use on the host to compare file permissions and sizes:

$ guestfish --ro -i -a backing.qcow2 tar-out / - |
    tar tvvf - | sort -k6 > files.without-chrome
$ guestfish --ro -i -a win7.qcow2 tar-out / - |
    tar tvvf - | sort -k6 > files.with-chrome
$ diff -u files.without-chrome files.with-chrome | less
[... 1,677 lines of output ...]
--rwxrwxrwx root/root    1157779 2011-06-02 11:06 ./Windows/WindowsUpdate.log
+-rwxrwxrwx root/root    1221259 2011-06-03 10:04 ./Windows/WindowsUpdate.log

What did I find out? As well as adding its own files to AppData/Local, Chrome also made changes to all of the registry hives, and as a result of installing new files, many internal Windows indexes were updated.

This technique will miss several changes which might be important to you: it will miss files that have had their content changed, but the file size and modification date didn’t change. And it will omit changes to extended attributes such as file forks, NT symbolic links and so on. If you want to list those changes too, you have to get down and dirty in the libguestfs API. This posting would be a good place to start.

Tomorrow we’ll look at changes in the Windows registry.

Tip: Using a backing file to record file and registry changes, part 1

Gary asked if it is possible to examine a KVM snapshot or backing file and perhaps list out the files and so on that had changed between the backing file and the current image.

It’s possible to use libguestfs to examine the changes, and in this three part series I’ll show you how.

I want to examine the file and Windows registry changes that happen when I install Google Chrome for Windows.

I first set up a Windows guest with a backing file, and I made sure the backing file was committed just before Chrome was downloaded and installed:

$ ll win7.qcow2 backing.qcow2 
-rw-r--r--. 1 qemu qemu 10099228672 Jun  3 10:40 backing.qcow2
-rw-r--r--. 1 root root    60555264 Jun  3 10:40 win7.qcow2

Then I installed Chrome in the guest, and as you can see the win7.qcow2 file (containing just changes) is much larger while the backing file has stayed the same:

$ ll win7.qcow2 backing.qcow2 
-rw-r--r--. 1 qemu qemu 10099228672 Jun  3 10:40 backing.qcow2
-rw-r--r--. 1 root root   682164224 Jun  3 11:08 win7.qcow2

Getting a list of files that have been added or removed by installing Chrome is easy. Note that this does not show files that have been modified (we’ll get to that in the next part). Note #2 because of a bug in WordPress, you have to type “backslash zero” where it says “NUL” below.

$ guestfish --ro -i -a win7.qcow2 find0 / - |
    tr 'NUL' '\n' | sort > files.with-chrome
$ guestfish --ro -i -a backing.qcow2 find0 / - |
    tr 'NUL' '\n' | sort > files.without-chrome
$ diff -u files.without-chrome files.with-chrome |
    less
[...]
+Users/rjones/AppData/Local/Google
+Users/rjones/AppData/Local/Google/Chrome
+Users/rjones/AppData/Local/Google/Chrome/Application
+Users/rjones/AppData/Local/Google/Chrome/Application/11.0.696.71
+Users/rjones/AppData/Local/Google/Chrome/Application/11.0.696.71/avcodec-52.dll
[...]
 Users/rjones/Desktop
 Users/rjones/Desktop/desktop.ini
+Users/rjones/Desktop/Google Chrome.lnk
 Users/rjones/Documents
 Users/rjones/Documents/desktop.ini

Google Chrome doesn’t (or can’t?) install anything under Program Files, instead preferring to install itself completely within AppData/Local in the user’s home directory.

In the next part I’ll show you how to find out when file contents, size or permissions have changed, and in the third part, we’ll look at Windows registry changes.

Fedora packages for decoding the Windows Event Log

EvtxParser package (requires perl-Data-Hexify).

See previously …

virt-win-reg can now read and write HKEY_USERS (Windows user preferences)

In libguestfs 1.11.7 (and some further patches in git) ≥ 1.11.8 you can now use virt-win-reg to list out and update Windows user preferences stored in HKEY_USERS keys.

There are two ways to use this. You can either specify a User SID like:

virt-win-reg Windows 'HKEY_USERS\S-1-5-19\Software\Policies'

or (as a libguestfs extension) you can specify a local user name directly:

virt-win-reg Windows 'HKEY_USERS\rjones\Control Panel'