libguestfs is 5 years old today

As we’re hard at work on the 1.26 release of libguestfs which is going to be a blockbuster, with massive improvements to virt-builder, a rewritten and much more flexible and robust appliance, a new guest customization tool, loads of new virt-sysprep features, rewritten virt-make-fs, and lots more .. it’s worth mentioning that libguestfs is 5 years old today.

The first commit was:

commit 28d760
Author: rjones <rjones>
Date:   Tue Mar 3 08:58:37 2009 +0000

    Build environment set up for libguestfs.

Creating a cloud-init config disk for non-cloud boots

There are lots of cloud disk images floating around. They are designed to run in clouds where there is a boot-time network service called cloud-init available that provides initial configuration. If that’s not present, or you’re just trying to boot these images in KVM/libvirt directly without any cloud, then things can go wrong.

Luckily it’s fairly easy to create a config disk (aka “seed disk”) which you attach to the guest and then let cloud-init in the guest get its configuration from there. No cloud, or even network, required.

I’m going to use a tool called virt-make-fs to make the config disk, as it’s easy to use and doesn’t require root. There are other tools around, eg. make-seed-disk which do a similar job. (NB: You might hit this bug in virt-make-fs, which should be fixed in the latest version).

I’m also using a cloud image downloaded from the Fedora project, but any cloud image should work.

First I create my cloud-init metadata. This consists of two files. meta-data contains host and network configuration:

instance-id: iid-123456
local-hostname: cloudy

user-data contains other custom configuration (note #cloud-config is
not a comment, it’s a directive to tell cloud-init the format of the file):

#cloud-config
password: 123456
runcmd:
 - [ useradd, -m, -p, "", rjones ]
 - [ chage, -d, 0, rjones ]

(The idea behind this split is probably not obvious, but apparently it’s because the meta-data is meant to be supplied by the Cloud, and the user-data is meant to be supplied by the Cloud’s customer. In this case, no cloud, so we’re going to supply both!)

I put these two files into a directory, and run virt-make-fs to create the config disk:

$ ls
meta-data  user-data
$ virt-make-fs --type=msdos --label=cidata . /tmp/seed.img
$ virt-filesystems -a /tmp/seed.img --all --long -h
Name      Type        VFS   Label   MBR  Size  Parent
/dev/sda  filesystem  vfat  cidata  -    286K  -
/dev/sda  device      -     -       -    286K  -

Now I need to pass some kernel options when booting the Fedora cloud image, and the only way to do that is if I boot from an external kernel & initrd. This is not as complicated as it sounds, and virt-builder has an option to get the kernel and initrd that I’m going to need:

$ virt-builder --get-kernel Fedora-cloud.raw
download: /boot/vmlinuz-3.9.5-301.fc19.x86_64 -> ./vmlinuz-3.9.5-301.fc19.x86_64
download: /boot/initramfs-3.9.5-301.fc19.x86_64.img -> ./initramfs-3.9.5-301.fc19.x86_64.img

Finally I’m going to boot the guest using KVM (you could also use libvirt with a little extra effort):

$ qemu-kvm -m 1024 \
    -drive file=Fedora-cloud.raw,if=virtio \
    -drive file=seed.img,if=virtio \
    -kernel ./vmlinuz-3.9.5-301.fc19.x86_64 \
    -initrd ./initramfs-3.9.5-301.fc19.x86_64.img \
    -append 'root=/dev/vda1 ro ds=nocloud-net'

You’ll be able to log in either as fedora/123456 or rjones (no password), and you should see that the hostname has been set to cloudy.

virt-format: Erase and make blank disks

There is a new tool in libguestfs ≥ 1.15.17: virt-format lets you erase disks and make blank disks.

Usage is quite simple:

$ virt-format -a disk.img

(Note that erases any data on disk.img!)

This works for any format of disk, eg. qcow2, or you can run it on host partitions, LVs, USB storage etc. By default it just creates a partition, but using other options you can make empty filesystems and logical volumes.

This is a simple tool that doesn’t cover everything you might want to do. For more complex requirements, see virt-make-fs or guestfish.

Tip: Pack files into a new disk image

Note: This requires libguestfs ≥ 1.4

If files/ is a directory containing some files, you can create a new disk image populated with those files using the command below (virt-make-fs can also be used for this):

$ guestfish -N fs:ext2:400M -m /dev/sda1 tar-in <(tar -C files/ -cf - .) /

Explanation:

1. guestfish -N fs:ext2:400M creates a prepared disk image which is 400MB in size with a single partition formatted as ext2. The new disk image is called test1.img.
2. -m /dev/sda1 mounts the new filesystem when guestfish starts up.
3. tar-in [...] / uploads and unpacks the tar file into the root directory of the mounted disk image.
4. <(tar -C files/ -cf - .) is a bash process substitution which runs the tar command and supplies the output (ie. the tar file) as the input on the command line.

This is a quick way to create “appliances” using febootstrap and libguestfs, although you should note that I don’t think these appliances would really work, I just use them for testing our virtualization management tools, like the ability to detect and manage disk images:

$ febootstrap -i fedora-release -i bash -i setup -i coreutils fedora-13 f13
$ echo '/dev/sda1 / ext2 defaults 1 1' > fstab
$ febootstrap-install f13 fstab /etc 0644 root.root
$ febootstrap-minimize f13
$ guestfish -N fs:ext2:40M -m /dev/sda1 tar-in <(tar -C f13 -cf - .) /
$ guestfish --ro -i -a test1.img

Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.

Type: 'help' for a list of commands
      'man' to read the manual
      'quit' to quit the shell

Operating system: Fedora release 13 (Goddard)
/dev/vda1 mounted on /
><fs> cat /etc/fstab
/dev/sda1 / ext2 defaults 1 1

New libguestfs tool: virt-make-fs

We posted a new tool today which you can use to make filesystems containing some prepopulated data.

Previously you’ve been able to create ISO and other read-only filesystems using programs like mkisofs and mksquashfs.

This program lets you do the same thing, but for any filesystem, for example an ext3 filesystem or NTFS.

Unlike ISO filesystems, ordinary filesystems don’t “just fit” around the files inside them. And so the main difficulty has been to estimate the amount of space you have to allocate in order to fit your files, with the least wasted space. This program automates this with a simple estimator function which should work for most cases and we hope to improve over time.

Usage is simple (virt-make-fs man page). Start with either a tarball or a directory full of files to add, and just do:

virt-make-fs [--type=fstype] /inputdir output.img