Tip: Convert a Windows DVD ISO to a bootable USB key using guestfish

Situation: You have a Windows DVD (or ISO), but like any sane person in 2013 you don’t have a DVD drive on the computer. You want to convert the Windows DVD into a bootable USB key. There are many recipes for this online, but they all require another Windows machine and of course cannot be automated.

However with guestfish (and the always brilliant SYSLINUX doing most of the heavy lifting), this script will unpack the ISO and turn it into a bootable USB key.

Notes:

1. I am not going to support this script. You will need to read the script, look up the commands in the guestfish man page, and understand what it does. Any requests for help will be deleted unread.
2. You need to edit the USB key device before using the script.
3. You need libguestfs ≥ 1.21 with SYSLINUX support compiled in.

#!/bin/bash -

guestfish <<'EOF'
trace on

add-ro en_microsoft_hyper-v_server_2012_x64_dvd_915600.iso

# NB: The next line MUST be changed to your USB drive.
# ANYTHING ON THIS DRIVE WILL BE OVERWRITTEN WITHOUT WARNING.
add /dev/sdX

run

# Inside the appliance, /dev/sda = DVD, /dev/sdb = USB.
# THESE ARE NOT RELATED TO HOST DISK NAMES.

echo "Partitioning the USB disk ..."
part-init /dev/sdb mbr
part-add /dev/sdb p 63 -1
part-set-mbr-id /dev/sdb 1 0xb
part-set-bootable /dev/sdb 1 true
mkfs vfat /dev/sdb1

echo "Copying the contents of the DVD to the USB key ..."
mkmountpoint /cd
mkmountpoint /usb
mount /dev/sda /cd
mount /dev/sdb1 /usb
# XXX We should add cp-r command XXX
debug sh "cp -rP /sysroot/cd/* /sysroot/usb"
#glob cp-a /cd/* /usb
umount /cd
umount /usb
rmmountpoint /cd
rmmountpoint /usb

echo "Making the USB key bootable using SYSLINUX ..."
syslinux /dev/sdb1
mount /dev/sdb1 /
upload /usr/share/syslinux/chain.c32 /chain.c32
write /syslinux.cfg "DEFAULT windows\n\nLABEL windows\nCOM32 chain.c32\nAPPEND fs ntldr=/bootmgr\n"
umount /dev/sdb1
upload /usr/share/syslinux/mbr.bin /dev/sdb

echo "Finished."

EOF

Using rsync with libguestfs

Since libguestfs 1.20 it has been possible to use rsync to upload or download files to a disk image incrementally. This is one way to do backups, but note that it won’t work on live guests unless you take a snapshot.

rsync involves using a network connection into or out of the appliance, and is therefore a lot more involved to set up. The script below shows one way to do this, by running an rsync daemon on the host, and letting the libguestfs appliance connect to it.

The script runs rsync inside the appliance, copying /home from the attached disk image out to /tmp/backup on the host. If the operation is repeated, then only incrementally changed files will be copied out. (To incrementally delete files on the target, add the deletedest:true flag).

Note you will need to open port 2999 on your host’s firewall for this to work.

#!/bin/bash -

set -x

# The target directory.
mkdir -p /tmp/backup

# Create the daemon.
rm -f /tmp/rsyncd.pid

cat <<EOF > /tmp/rsyncd.conf
port = 2999
pid file = /tmp/rsyncd.pid

[backup]
  path = /tmp/backup
  use chroot = false
  read only = false
EOF

rsync --daemon --config=/tmp/rsyncd.conf

# Run guestfish and attach to the guest.
guestfish --ro --network -a /dev/fedora/f19rawhidex32 -i <<EOF
trace on
rsync-out /home rsync://rjones@192.168.122.1:2999/backup archive:true
EOF

# Kill the rsync daemon.
kill `cat /tmp/rsyncd.pid`

Tip: Test changes while continuing to work

This script below lets you test changes while continuing to work on code. Let’s say that your test suite takes quite a while to run (hello, libguestfs). You can do:

$ test-change make check

The script copies the whole current directory into a temporary directory and runs the check in there. You still have to open a new terminal to run the tests, but the tests can go ahead while you continue working.

#!/bin/bash -
# Copy current directory to a temporary,
# then run the test command on that copy,
# and report the results.
# by Richard W.M. Jones <rjones@redhat.com>
#
# Usage (from current directory):
#   test-change command [args ...]
# eg:
#   test-change make check

echo "Copying original directory; wait a moment ..."
d=`mktemp -d`

trap "rm -rf $d" EXIT INT TERM QUIT

cp -a . $d

cd $d

echo "Original directory copied, starting test."
echo "You can carry on working now."
sleep 1

# Run the test command.
"$@"

Please don’t do this (v2v and p2v requests that are wrong)

Using v2p to get around Oracle support contracts

Problem: Oracle won’t support the database in a virtualized environment. If you report a bug, they’ll ask you to reproduce it on a supported (ie. physical) machine.

Wrong solution: We’ll run Oracle in a VM. When we run into trouble, we’ll use a V2P tool to convert the virtual machine to a physical machine!

Why this is wrong: Conversion involves copying the disks, ripping out device drivers, adding new device drivers, fiddling with configuration files, doing resize ops on filesystems, and reinstalling the boot loader. These are (a) slow, (b) very intrusive, and (c) liable to break. This is all a recipe for turning a small disaster (ie. my database is down) into a very big disaster (my database is still down and the hairy support “solution” took 6 hours and didn’t work).

Good solution: Oracle are probably right that you shouldn’t try to run your database virtualized. But assuming you want to ignore that advice, put your database and its files onto a separate SAN LUN. When you need support, detach the LUN from the virtual machine and reattach it to a physical machine. This operation should be instantaneous and doesn’t involve any modification of the data.

Using p2v and v2p to test upgrades

Problem: It’s not easy to test an upgrade on a production physical machine.

Wrong solution: Virtual machines let you snapshot, test your upgrade on the snapshot, and if it’s bad you just throw away the snapshot. Therefore to test our upgrade, we’ll convert the physical machine to virtual (P2V), do the test, and if it works we’ll convert it back to a physical machine (V2P)!

Why this is wrong: Conversion involves a slow disk copy and a very intrusive set of modifications to the configuration. P2V followed by V2P is not a symmetric operation that leaves you with an identical machine. More than likely it’ll simply break the machine, and if it doesn’t, then drivers could be less than optimal after the conversion. Plus (unlike with virtualized environments) your physical machine is a one-of-a-kind system, and if you break it with a hairy set of P2V and V2P operations you can’t just roll back to a previous snapshot.

Good solution: Virtualize your workloads! If you don’t want to do that, use a filesystem like btrfs/ZFS that lets you do cheap snapshots, or use the snapshot feature of your SAN. In any case, always arrange your production environment so that you have a staging mirror on which to do tests before you deploy anything to production, and have a tested back-out plan.

Using multiple v2v steps

Problem: We don’t have a conversion tool that can do (eg.) Citrix Xen to KVM in one step.

Wrong solution: We found something on the web that can do Citrix to VMware, and Red Hat have a great tool for doing VMware to KVM, so we’ll just run one after the other!

Why this is wrong: Conversion involves a large set of intrusive changes on the guest such as installing device drivers for the particular target hypervisor. Doing this in two steps means you go through two rounds of intrusive changes to your guest, and it’s unlikely that anyone has tested both together. Most likely it’ll break, or leave your guest with conflicting device drivers.

Good solution: Sorry, but at the moment there isn’t a good solution, but that doesn’t mean you should use the bad solution. It could be your best bet is to reinstall the guest from scratch on the target VM.

Build a guest library

I have this exported over NFS to all my local machines:

$ ls -lh /mnt/media/guest-library/
total 12G
-rw-r--r--. 1 rjones rjones 971M Apr 16 23:38 debian5x64.img.xz
-rw-r--r--. 1 rjones rjones 250M Jul 26 23:09 f16x32.img.xz
-rw-r--r--. 1 rjones rjones 809M Jul 26 20:29 f17x64.img.xz
-rw-r--r--. 1 rjones rjones  59M Jul  6 22:19 freedos11.img.xz
-rw-r--r--. 1 rjones rjones 379M Jul  7 20:11 opensuse1113x64.img.xz
-rw-r--r--. 1 rjones rjones 349M Jul  6 22:31 plan9.img.xz
-rw-rw-r--. 1 rjones rjones  126 Apr 16 17:22 README
-rw-r--r--. 1 rjones rjones 384M Apr 18 00:19 rhel3x64.img.xz
-rw-r--r--. 1 rjones rjones 471M Apr 18 00:15 rhel4x64.img.xz
-rw-r--r--. 1 rjones rjones 848M Apr 16 17:02 rhel5epelx64.img.xz
-rw-r--r--. 1 rjones rjones 768M Jul  7 07:21 sl60x64.img.xz
-rw-r--r--. 1 rjones rjones 148M May 23 16:08 ubuntu1204ppc.img.xz
-rw-r--r--. 1 rjones rjones 1.1G Apr 16 17:09 win2003.img.xz
-rw-r--r--. 1 rjones rjones 3.4G Jun  1 10:46 win8-preview.img.xz
-rw-r--r--. 1 rjones rjones 2.1G Apr 16 20:28 winxp.img.xz

Having a library of images which are (relatively) small like this is great for testing. If someone gives me a bug and it happens to affect one of the guests I have saved, then I can just unpack that guest, virt-resize it to the right size and off I go, and that’s usually a lot quicker than installing a new guest, particularly when you’ve got only slow ADSL.

Adding a new guest is easy using virt-sparsify and xz. Note I’ve set the temporary directory (TMPDIR) to somewhere with plenty of space.

$ sudo TMPDIR=$(pwd) virt-sparsify /dev/vg_pin/F18Rawhidex32 f18rawhidex32.img
Create overlay file to protect source disk ...
Examine source disk ...
 100% ⟦▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓⟧ 00:00
Fill free space in /dev/vda1 with zero ...
 100% ⟦▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓⟧ --:--
Fill free space in /dev/vg_f18rawhidex32/lv_root with zero ...
 100% ⟦▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓⟧ --:--
Clearing Linux swap on /dev/vg_f18rawhidex32/lv_swap ...
 100% ⟦▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓⟧ 00:00
Copy to destination and make sparse ...

Sparsify operation completed with no errors.  Before deleting the old 
disk, carefully check that the target disk boots and works correctly.

$ ll -h f18rawhidex32.img
-rw-r--r--. 1 root root 10G Jul 27 14:42 f18rawhidex32.img
$ du -sh f18rawhidex32.img
3.0G	f18rawhidex32.img
$ xz -T 0 --best f18rawhidex32.img
$ ll -h f18rawhidex32.img.xz
-rw-r--r--. 1 rjones rjones 667M Jul 27 14:42 f18rawhidex32.img.xz

Tip: gdb break if first function argument has a value

I wanted to find out who was closing stdout in my program (ie. who is calling close(1);).

This seems to work in gdb:

(gdb) break close if $rdi == 1

The explanation is that on x86-64, register %rdi is used to pass the first argument to a function. In gdb conditionals, register names are prefixed by $. Hence the condition checks if the first argument to the function is 1.

Using virt-sparsify and xz, we can really compress VMs for storage

17G	debian5x64.img.orig

The original image has about 4.7 GB of data, plus a large swap partition, according to virt-df:

$ virt-df -a debian5x64.img.orig -h
Filesystem                        Size       Used  Available  Use%
debian5x64.img.orig:/dev/sda1     322M        66M       239M   21%
debian5x64.img.orig:/dev/debian5x64.home.annexia.org/home
                                  3.4G       359M       2.9G   11%
debian5x64.img.orig:/dev/debian5x64.home.annexia.org/root
                                  320M       301M       2.4M   95%
debian5x64.img.orig:/dev/debian5x64.home.annexia.org/tmp
                                  300M       8.2M       276M    3%
debian5x64.img.orig:/dev/debian5x64.home.annexia.org/usr
                                  3.4G       2.0G       1.2G   60%
debian5x64.img.orig:/dev/debian5x64.home.annexia.org/var
                                  2.6G       2.0G       536M   76%

Using virt-sparsify, all unused space in the image is made sparse. The most recent version can sparsify swap partitions too:

4.6G	debian5x64.img

xz --best -T 0 reduces the final image to under a gigabyte:

971M	debian5x64.img.xz

Tip: Detecting guest activity: three methods

(1) Is the guest generating disk and network activity (a.k.a are the lights flashing)?

I don’t have nice code for this, but you can see the technique that Oz uses here. Oz uses the libvirt monitoring APIs to look for disk and network activity, and signals when it hasn’t seen any after a certain timeout period.

(2) When was the last time a user logged in?

If this is the sort of “activity” you’re after, then you can use virt-cat on Linux, or virt-win-reg [sorry, no actual example yet] on Windows.

(3) What was the last file updated in a guest?

Finally, if you’re interested in the newest file updated in a guest, see this technique which will work for any Linux or Windows guest.

Are there other kinds of “activity” that it’s interesting to find from guests?

Tip: Debugging the early boot process with qemu and gdb

Update: A much easier way is to use gdbserver.

Start qemu with the following parameters:

$ qemu-system-x86_64 -s -S -m 512 -hda winxp.img

And connect with gdb like this:

$ gdb
(gdb) target remote localhost:1234
(gdb) set architecture i8086
(gdb) break *0x7c00
(gdb) cont

This will breakpoint at 0x7c00, which is when the boot sector has been loaded into memory by the BIOS and control is passed to the boot sector.

You can use ordinary gdb commands to disassemble and debug the guest.

Tip: Call libguestfs from JRuby

You can call libguestfs from JRuby.

JRuby only recently got support for Ruby C extensions, and support is currently quite wobbly. This excellent blog post summarizes the situation.

There are two ways to get access to libguestfs from JRuby. One way would be via the usual Ruby C extension, but as I said above, that’s wobbly and in fact does not work. The second way, which is what we’ll use, is to use the native Java module for libguestfs.

One thing to note is that the Ruby extension and the Java-via-Ruby extension are similar but not quite the same. In fact I found you can copy and paste most Ruby libguestfs code directly, and have it run, but some things need alteration.

Here is the example JRuby libguestfs program:

include Java

Dir["/usr/share/java/libguestfs-*.jar"].each { |jar| require jar }

guestfs = com.redhat.et.libguestfs

g = guestfs.GuestFS().new
version = g.version

printf("libguestfs %d.%d.%d%s\n",
       version.major, version.minor, version.release, version.extra)

# Create an empty, raw format 500 MB disk.
File.open("test.img", "w") {
  |f| f.seek(500*1024*1024); f.write("")
}

# Note that the key must be a string ("format") and
# not an atom (:format).
optargs = java.util.HashMap.new( {"format" => "raw"} )
g.add_drive_opts("test.img", optargs)
g.launch()

g.pvcreate("/dev/sda")
g.vgcreate("VG", ["/dev/sda"]);
g.lvcreate("LV1", "VG", 200);
g.lvcreate("LV2", "VG", 200);

puts "Created test.img"
puts "Now try running: virt-filesystems -a test.img --all --long -h"

You can run it by saving the above to test.rb, and then doing (on Fedora Rawhide or upcoming Fedora 16):

# yum install jruby libguestfs-java-devel
$ jruby test.rb