Tip: Configure guest with filesystem directly on a host device

Xen lets you assign a host block device as a guest partition, synthesizing the partition table for you. So for example, the host /dev/VG/LV might appear in the guest as /dev/xvda1 with a filesystem directly on it. This means from the host you just see a filesystem, which you can create directly, mount etc. Some people like this and others like me have guests already configured like this.

It is possible to configure a libvirt/KVM host this way, and this post describes how. It is somewhat more manual than using virt-install. In fact I will assume that you either already have guests configured this way, or you know how to prepare a guest filesystem (eg. using debootstrap) directly onto a device like this.

There are two points about such a configuration on KVM.

Number one: the kernel and initrd of the guest live “outside” the guest, on the host. This can be beneficial, eg. if you want all your guests to have a common kernel which you will manage, compile and upgrade centrally. Number two is that KVM cannot synthesize the partition table like Xen, so inside the guest it’s going to see a filesystem on /dev/vda directly, with no partition table. Linux will work just fine. Windows wouldn’t, but that doesn’t matter because Windows cannot be configured like this anyway.

Decide where to put your external kernel and initrd on the host, and then configure libvirt like this:

<domain type='kvm'>
  <name>guest</name>
  ...
  <os>
    <type arch='x86_64' machine='pc'>hvm</type>
    <kernel>/usr/local/etc/virt/vmlinuz</kernel>
    <initrd>/usr/local/etc/virt/initrd</initrd>
    <cmdline>ro root=/dev/vda</cmdline>
    <boot dev='hd'/>
  </os>
  ...
  <devices>
    <disk type='block' device='disk'>
      <source dev='/dev/VG/LV'/>
      <target dev='vda' bus='virtio'/>
    </disk>

The kernel, initrd, cmdline elements are what libvirt calls “direct kernel boot”.

Secondly of course you need your guest filesystem directly on the device or logical volume. Its fstab should be set up accordingly, plus any other configuration files it needs. (guestfish can of course safely create, view and edit guest filesystems which are configured in this manner).

In fact I use several production guests configured like this, for historical reasons (they came from a Xen server). However I find in general this a more clumsy way to organize guests. It might scale better at very high-end configurations, if you wrote a lot of custom tools, but the vast majority of users don’t have systems on such a scale.

There is also a security issue: although you can mount a guest filesystem directly on the host this way, it’s likely that you shouldn’t. Only this week I found a kernel-crasher, possibly exploitable, in the minix filesystem driver. Use libguestfs to put a safe barrier between your guests and your host.

4 Comments

Filed under Uncategorized

4 responses to “Tip: Configure guest with filesystem directly on a host device

  1. frankiii

    Love it. Thanks for the write-up.🙂

  2. Gary

    Sorry to ask this on your blog. I have posted this around and no one seems to know, but I figure you will. Can guestfs be used to mount a virtual machine disk image rw as a regular mounted file system? I couldn’t tell from the documentation if it was possible or not. I could use a regular partition in KVM for Windows and mount it when the guest is powered off, but it would be much handier to do be able to do this to an image file.

  3. Pingback: Technique for synthesizing a partition table on a “naked” filesystem « Richard WM Jones

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s