Tip: Configure guest with filesystem directly on a host device

Xen lets you assign a host block device as a guest partition, synthesizing the partition table for you. So for example, the host /dev/VG/LV might appear in the guest as /dev/xvda1 with a filesystem directly on it. This means from the host you just see a filesystem, which you can create directly, mount etc. Some people like this and others like me have guests already configured like this.

It is possible to configure a libvirt/KVM host this way, and this post describes how. It is somewhat more manual than using virt-install. In fact I will assume that you either already have guests configured this way, or you know how to prepare a guest filesystem (eg. using debootstrap) directly onto a device like this.

There are two points about such a configuration on KVM.

Number one: the kernel and initrd of the guest live “outside” the guest, on the host. This can be beneficial, eg. if you want all your guests to have a common kernel which you will manage, compile and upgrade centrally. Number two is that KVM cannot synthesize the partition table like Xen, so inside the guest it’s going to see a filesystem on /dev/vda directly, with no partition table. Linux will work just fine. Windows wouldn’t, but that doesn’t matter because Windows cannot be configured like this anyway.

Decide where to put your external kernel and initrd on the host, and then configure libvirt like this:

<domain type='kvm'>
  <name>guest</name>
  ...
  <os>
    <type arch='x86_64' machine='pc'>hvm</type>
    <kernel>/usr/local/etc/virt/vmlinuz</kernel>
    <initrd>/usr/local/etc/virt/initrd</initrd>
    <cmdline>ro root=/dev/vda</cmdline>
    <boot dev='hd'/>
  </os>
  ...
  <devices>
    <disk type='block' device='disk'>
      <source dev='/dev/VG/LV'/>
      <target dev='vda' bus='virtio'/>
    </disk>

The kernel, initrd, cmdline elements are what libvirt calls “direct kernel boot”.

Secondly of course you need your guest filesystem directly on the device or logical volume. Its fstab should be set up accordingly, plus any other configuration files it needs. (guestfish can of course safely create, view and edit guest filesystems which are configured in this manner).

In fact I use several production guests configured like this, for historical reasons (they came from a Xen server). However I find in general this a more clumsy way to organize guests. It might scale better at very high-end configurations, if you wrote a lot of custom tools, but the vast majority of users don’t have systems on such a scale.

There is also a security issue: although you can mount a guest filesystem directly on the host this way, it’s likely that you shouldn’t. Only this week I found a kernel-crasher, possibly exploitable, in the minix filesystem driver. Use libguestfs to put a safe barrier between your guests and your host.