libguestfs: Access and modify virtual machine disk images

Time to reveal what we’re doing with qemu, febootstrap and minimal Fedora images.

A constant problem with virtualization management is how to “see inside” the guest virtual machine. You can measure things like how much CPU it’s using or how much network traffic it generates, but people often ask how much disk space a guest is using (not allocated, but actually using)? Did the guest boot up correctly? Can you install virtio drivers in that guest? Can you add a disk to a guest and change its /etc/fstab at the same time? Can you migrate a guest from Xen to KVM, uninstall the Xen PV kernel and add the right KVM drivers?

These operations require us to fish around inside the guest’s disk image, perhaps just reading the image and printing out simple stats as we do already with virt-df, but perhaps modifying the guest’s disk quite substantially, such as installing drivers or running programs.

Now step aside for a minute and think about virtual machine disk images. From the host, they look like files or partitions, but they contain a partition table, partitions, perhaps LVM, a variety of filesystems like ext3 or NTFS or btrfs, and the whole lot might be wrapped in a tricky container such as qcow2. For virt-df we actually wrote new code that can decode a lot of this, but it’s a massive effort to keep up with changes in the formats.

What we need to use is the Linux kernel code directly.

The way I’m going to do this is to boot a Linux kernel. A small, you might say “minimal” Linux distro, with a bit of userspace. The whole thing runs inside a qemu container, and we talk from a small library to the userspace inside qemu, giving it instructions like “edit this file”, “run this program”, “install this device driver”.

The beauty of this is that Linux already has the drivers to handle numerous different filesystems, LVM, partition types and so on. The qemu process can decode qcow2 and a variety of other containers. We can even run an NFS server inside the guest and mount filesystems on the host. The whole thing can build and run as non-root, because you don’t need to be root to run either febootstrap or qemu.

We hide the complexity inside a library, libguestfs.

Nothing much is coded at the moment. I have documented what the API will probably look like, and written a lot of code that I might throw away. However the fundamental components are there, such as febootstrap and rpmdepsize, and I’ve done some work upstream with making fakechroot, yum and rpm work together.

I have a hard deadline to release both a working libguestfs and some migration tools by April 8th (just 1 week away). Ideas are welcome, but I’m probably not going to really be open to contributions before next week, when I should have something working to show.