Tip: Creating throwaway appliances with febootstrap

This is an update to a previous posting, but using the new febootstrap 3.x cross-distro toolchain to make building supermin appliances even simpler.

Firstly we create a supermin appliance containing a few packages and their dependencies. Note that I’m not minimizing this appliance so it’s a bit bigger (3.1MB!) than the ones we would generate in reality:

$ mkdir supermin.d
$ febootstrap --names 'bash' 'coreutils' -o supermin.d
febootstrap: warning: some host files are unreadable by non-root
febootstrap: warning: get your distro to fix these files:
	/sbin/unix_update
	/usr/libexec/pt_chown
	/usr/sbin/build-locale-archive
	/usr/sbin/glibc_post_upgrade.x86_64
	/usr/sbin/tzdata-update
$ ls -lh supermin.d/
total 3.1M
-rw-rw-r--. 1 rjones rjones 2.7M Dec 10 18:23 base.img
-rw-rw-r--. 1 rjones rjones 462K Dec 10 18:23 hostfiles

The purpose of these two files is explained in the febootstrap documentation.

This won’t boot without a /init script, and we can easily provide one:

$ cat init
#!/bin/bash -
echo Welcome to my world
bash -i
$ chmod +x init
$ echo init | cpio -o -H newc --quiet > supermin.d/init.img

The 3 files in supermin.d/ are a supermin appliance, and could be packaged up in a Fedora, Debian or Ubuntu package.

When you actually want to come and launch this appliance, you use febootstrap-supermin-helper to reconstruct the appliance:

$ febootstrap-supermin-helper -f ext2 supermin.d x86_64 \
    kernel initrd root

(Note that “kernel”, “initrd” and “root” are output files in that command)

You’d usually arrange for those files to be cached, since febootstrap-supermin-helper takes a few seconds to run (8 seconds on my laptop) and by caching it you can get reconstruction time down to a fraction of a second.

Now to boot, run qemu or qemu-kvm like this:

$ qemu-kvm -kernel kernel -initrd initrd -hda root

After a few seconds you’ll get to the shell:

Ubuntu packages for libguestfs

You can download Ubuntu packages (*.deb files) for libguestfs. Don’t forget to read the README file.

Previously: Debian packages.

libguestfs binaries for Debian and Ubuntu

Over the weekend I rewrote the way supermin appliance building works to make it much more lightweight and mostly distro-independent. As a result, libguestfs now builds from source reasonably simply on both Debian and Ubuntu.

I’m also able to make binaries available for Debian and Ubuntu (10.10), but you must read the README file first.

If these do/don’t work for you, please let me know.

Building libguestfs from source with the binary appliance

Inspired by this comment and reply I wrote a short tutorial which explains how to build libguestfs from source, but using the binary appliance that we ship.

http://libguestfs.org/download/binaries/REUSING-THE-APPLIANCE.txt

This is essentially a very easy way to build libguestfs if it hasn’t been ported to your Linux distro, or if you have to rely on some appliance builder which is frequently not working.

Notes: compiling libguestfs from source on Ubuntu 10.04

These are just notes that I made while compiling libguestfs on Ubuntu.

Note (1): Interested in maintaining libguestfs on Ubuntu? We really need a dedicated, skilled packager to help us out.

Note (2): More information is available on the libguestfs homepage, and for detailed help please send a message to the mailing list.

Start with the tarball. I am using libguestfs-1.5.12.tar.gz from here.

You need to install a boatload of dependencies. Luckily, they are all (except one) in Ubuntu repositories already, so they are just an “apt-get” and a moderate wait away:

$ sudo apt-get install \
  libfindlib-ocaml-dev ocaml-native-compilers \
  libxml-light-ocaml-dev \
  debootstrap debirf \
  libpcre3-dev libaugeas-dev libmagic-dev \
  libxml2-dev libvirt-dev \
  kvm \
  libreadline-dev \
  gcc make \
  po4a

You need libhivex0 and libhivex-dev too. Ubuntu doesn’t have these, but these two packages from Debian squeeze worked fine for me: libhivex0, libhivex-dev.

Unpack the tarball and configure like this:

$ ./configure \
  --with-repo=lucid \
  --with-mirror=http://gb.archive.ubuntu.com/ubuntu/

If you’re using a different version of Ubuntu from me, change “lucid” to whatever is appropriate. It needs to exactly match your host version.

Obviously ./configure needs to get all the way through. If there is a missing dependency, install it too. If all goes well it will print a summary of what it is going to compile which will look something like this:

Thank you for downloading libguestfs 1.5.12

This is how we have configured the optional components for you today:

Daemon .............................. yes
Appliance ........................... yes
QEMU ................................ /usr/bin/qemu-system-x86_64
OCaml bindings ...................... yes
Perl bindings ....................... yes
Python bindings ..................... no
Ruby bindings ....................... no
Java bindings ....................... no
Haskell bindings .................... no
PHP bindings ........................ no
virt-inspector ...................... no
virt-* tools ........................ no
supermin appliance .................. no
FUSE filesystem ..................... no

If you want more of those features to be enabled, you’ll need to track down the missing optional dependencies by reading back over the full configure output.

Build:

$ make

During the build, libguestfs will build a small Ubuntu-based appliance using debirf. It needs network access to fetch the .debs for this (using squid can help a lot). It’s also possible to use a local repository in which case network access won’t be required, although it doesn’t work by magic — you still have to supply the .debs from somewhere.

You might need to adjust appliance/packagelist.in and/or files in appliance/debian/modules/ if a package is missing or if appliance building fails. On Ubuntu 10.04 I had to comment out the “find” statement where it was trying to delete unused kernel modules (bug in debirf I suspect) and comment out the apt-get remove command.

Building the appliance is generally the most error-prone part of the whole process because it depends on infrequently tested parts of Debian such as debirf, debootstrap, fakeroot and packages themselves. If you have no patience for this you can grab the pre-built binary appliance from our binary distribution (also use ./configure --disable-appliance).

Now once you’re through the build, check that things work. You can do a quick “does it work at all” check:

$ make quickcheck

(Obviously if this fails, you need to go back and take a hard look at the build).

Finally run the full test suite:

$ make check

[Fix the bugs and submit patches!]

You can now try running guestfish etc.

libguestfs binaries for all Linux distros

I uploaded some distro-independent Linux/x86-64 binaries of libguestfs, guestfish, guestmount and the virt-* tools. Be sure to read the README file first.

These are a little experimental and I’d welcome feedback. I got them to work fine on Debian 5.0 after upgrading glibc and Perl, but YMMV.

Update: OpenSUSE 11.3 working.

The version of KVM supplied is too old (doesn’t support virtio-serial) so I had to compile qemu from git and drop the following qemu wrapper in place:

#!/bin/sh -
qemudir=/home/rjones/d/qemu
exec $qemudir/x86_64-softmmu/qemu-system-x86_64 -L $qemudir/pc-bios "$@"

The next problem which had me confused for a very long time was that qemu kept aborting while allocating memory. After trying things like adding swap, playing with overcommit settings and so on, it turned out that SUSE uses some really silly, and very small default ulimits for virtual memory. You have to do:

ulimit -Hv unlimited
ulimit -Sv unlimited
ulimit -Hm unlimited
ulimit -Sm unlimited

Update: Ubuntu 10.04 working.

As with SUSE, the version of KVM shipped by Ubuntu is too old to support virtio-serial. All I had to do was replace it with qemu from git and the same qemu wrapper above.

After that guestfish works fine.

If you need the Perl tools, then you have to upgrade Perl to 5.12.1, install hivex, and link libpcre.so.0 -> libpcre.so.3.

Tip: find out when filesystems get full with virt-df (working version!)

Thanks go to “S” for pointing out the mistake in my last posting. It’s easy to fix though.

When virt-df is run with the –csv option, it outputs a Comma-Separated Values format spreadsheet. CSV is not totally straightforward to parse and so you should leave it up to a specialized tool. Of course I recommend my own tool csvtool which is available in Fedora, Debian and Ubuntu.

The basic plan is to use csvtool to drop the first row (the column headings row), and pull out columns 1, 2 and 6 from the rest of the output. Those columns are respectively the virtual machine name, the filesystem, and the %usage:

$ virt-df --csv | csvtool drop 1 - | csvtool col 1-2,6 -
Debian5x64,/dev/debian5x64/home,35.6%
Debian5x64,/dev/debian5x64/root,60.9%
Debian5x64,/dev/debian5x64/tmp,3.3%
Debian5x64,/dev/debian5x64/usr,40.5%
Debian5x64,/dev/debian5x64/var,15.2%
Debian5x64,/dev/vda1,20.3%
RHEL620100329n0x64,/dev/vda1,7.4%
RHEL620100329n0x64,/dev/vg_rhel620100329n0/lv_root,16.7%
Ubuntu910x64,/dev/vda1,42.5%
[...snipped...]

Adding the previous awk test gives the desired output, all filesystems with ≥ 60% usage:

$ virt-df --csv |
    csvtool drop 1 - |
    csvtool col 1-2,6 - |
    csvtool readable - |
    awk 'strtonum(substr($3,0,length($3)-1))>=60 {print}'
Debian5x64         /dev/debian5x64/root            60.9% 
F13Rawhidex64      /dev/vg_f13rawhide/lv_root      66.0% 
Windows7x32        /dev/vda2                       74.6% 
Windows7x64        /dev/vda2                       90.7% 

This can be placed in a script or cron job.

Ubuntu 9.10 packages for libguestfs

By popular demand, I’ve built Ubuntu packages for the latest libguestfs, guestfish, virt-inspector, virt-cat and virt-df:

http://www.annexia.org/tmp/debian/

These are experimental. If they break, you get to keep both pieces.

One point in particular is there is no Perl Sys::Virt package in Ubuntu, which means you have to supply this yourself, else virt-inspector etc. won’t work.

libguestfs on Ubuntu – works!

libguestfs works on Ubuntu 9.10. You need to apply the five non-finalized patches here, and read the instructions in the first posting carefully.

(Thanks to Wolfgang Richter for prompting me to get this working, Guido Gunther for patch review and Jameson Rollins for fixing debirf for us)

Update

Packages are available …

Prebuilt distributions part 3

[This is the final article in a three part series. Read part 1 and part 2.]

In this part I was going to discuss the different installation options, and I’ll give a quick introduction to each at the end. But you know? What I really need is this tool:

+--------------------------------------------------------+
| Rich's Amazing Bug Environment Reproducer Tool!        |
+--------------------------------------------------------+
| Bugzilla number: # [545116] [ Go ]                     |
|                                                        |
| Preselecting Fedora Rawhide image.  If this is wrong,  |
| choose the right image from the list below:            |
|                                                        |
| [X] Fedora Rawhide (x86-64)                            |
| [ ] Fedora 12                                          |
| [ ] Fedora 11                                          |
| [ ] Ubuntu 9.11                                        |
| [ ] Ubuntu 9.04                                        |
| [ ] Debian 6.0                                         |
| [ ] Debian 5.0    [Download more distributions]        |
|                                                        |
| [X] x86-64   [ ] i386                                  |
|                                                        |
| [Create  virtual machine now]                          |
+--------------------------------------------------------+

The Create virtual machine now button must take no more than 60 seconds to deliver the virtual machine, plus maybe another 60 seconds to boot it and give me the login prompt. I can then go in, configure it and reproduce the bug, and snapshot it, so I can go back and test fixes later. (I’d like the snapshot workflow to go through the user interface too, and ponies).

The only approach that doesn’t take ages to install is to have the distributions prebuilt as small disk images locally (as described already in part 2).

Note stuff I don’t want to have to specify: storage, download URLs, VM names, etc etc.

Why doesn’t this exist already, or perhaps it does?

I’d also like to suggest that the resizable disk image should become yet another distribution format that Linux distros provide. Many do already, albeit buried inside a live CD.

---

As promised, my quick summary of installation methods:

virt-install

virt-install is a great command line tool for installing new VMs. I have these two commands pretty much permanently in my command history:

# lvcreate -n F13Rawhidex64 -L 10G vg_trick
# virt-install -v -n F13Rawhidex64  --accelerate \
    -r 512 -f /dev/vg_trick/F13Rawhidex64 \
    --cdrom Fedora-12-x86_64-DVD.iso

The commands above lead to a manual installation, but you can also specify a kickstart file, although the syntax for doing this is not obvious and not documented in the manual page.

Nevertheless, even an install from a CD-ROM on the local drive (as in the example above) is a slow process, taking anything up to an hour.

kickstart

Kickstart is the method used to automate Anaconda installs, ie. installs of all Fedora, Red Hat and derived distributions. It’s a little unwieldy because you have to write a separate kickstart file, then host that on a web server (so the VM can find it), then add a special magic kernel parameter when booting the VM: ks=http://local.example.com/kickstart.ks

I found in the past that it’s better to keep a collection of kickstart files around and iterate them slowly so you can find out what works and what doesn’t.

Also, kickstarts are automatic, but just as slow as manual installation.

debootstrap

Debian’s debootstrap (and the Fedora equivalent I wrote called febootstrap) is, as far as I know, the standard way to install a new Debian instance from an existing Debian instance. Unlike our VM installs above, debootstrap creates a “chroot” directory, and what you do with that directory is then up to you. Usually you would debootstrap into some mounted device. debootstrap is very simple to use and driven completely from the command line.

It’s not particularly fast. Because it downloads, unpacks and installs each package, it can take an hour or more to run.

ubuntu-vm-builder

This is Ubuntu’s wrapper around debootstrap and libvirt/qemu.

cobbler

Cobbler is a network install server which lets you pxeboot and configure a network of machines (and virtual machines) from a central server. It has lots of wonderful features — go and read the web site.