Create a partitioned device from a collection of filesystems

Xen has a feature where it can export virtual partitions directly to virtual machines. You can configure a Xen VM like this example:

disk = ['phy:raidvg/devroot,hda1,w','phy:raidvg/devswap,hda2,w']

Notice that host device /dev/raidvg/devroot is mapped to a partition inside the guest (/dev/hda1), and on the host this device directly contains a filesystem:

host# file - < /dev/raidvg/devroot
/dev/stdin: Linux rev 1.0 ext3 filesystem data, UUID=... (needs journal recovery)

Inside the guest, it sees /dev/hda1, /dev/hda2, but no /dev/hda device or partition table.

This is actually a nice feature of Xen because resizing filesystems directly is much easier than resizing a partitioned block device. You can just make the host device bigger (lvresize -L sizeG /dev/raidvg/devroot), reboot the guest so it sees the increased device size, then resize the filesystem (resize2fs — this can even be done live if you want to make the filesystem bigger).

Imagine if we’d been dealing with a KVM partitioned block device instead:

+-+---------------------+------------+
|M| hda1                | hda2       |
|B| (root filesystem)   | (swap)     |
|R|                     |            |
+-+---------------------+------------+

Resizing this is much more painful. You first have to extend the host block device:

+-+---------------------+------------+-------+
|M| hda1                | hda2       | space |
|B| (root filesystem)   | (swap)     |       |
|R|                     |            |       |
+-+---------------------+------------+-------+

Now what do you do? Easiest is probably to create a third (hda3) partition in that extra space. If you didn’t have the foresight to use LVM, then this means your root filesystem cannot be extended — you can only create another extra filesystem (say for /var) and copy files over. This is very inflexible.

Instead you could recalculate the MBR and move (ie. copy block by block) hda2 up. (Imagine it wasn’t swap space since you can just throw that away and recreate it, but some valuable files). Recalculating the MBR is generally error-prone because partitions have strange limitations and alignment requirements.

One day I intend to write a program to do these kinds of complex resizing operations …

Anyhow, this wasn’t even what this rambling blog entry was about. It is a companion to last week’s tip about extracting filesystems from disk images. Can we do the opposite, ie. create a partitioned device from a collection of Xen filesystems?

Answer, yes we can, with guestfish.

I’m starting in fact with the filesystem and swap devices copied from my Xen server, and I need to know their exact sizes in 1024-byte-blocks first:

$ ls --block-size=1024 -l devroot devswap
-rw-rw-r--. 1 rjones rjones 3145728 2010-03-17 14:18 devroot
-rw-rw-r--. 1 rjones rjones 1048576 2010-03-17 14:19 devswap

I’m going to put this into a 5G disk image, giving me space to expand the root filesystem to fit. Inexplicably I’ve decided to keep the swap partition content even though in reality I would just throw it away and recreate the swap partition (imagine there’s some important filesystem content in there instead). I want devswap to precisely fit at the end of the new disk image.

Let’s create the disk image and find out how big it is in sectors:

$ rm -f disk.img
$ truncate -s 5G disk.img
$ guestfish -a disk.img -a devroot -a devswap
><fs> run
><fs> blockdev-getsz /dev/vda
10485760  # size in 512 byte sectors

Now I need to do some back of the envelope calculations to work out how I will size and place each partition. (This is a huge pain in the neck — I had to do several runs to get the numbers to come out right …)

><fs> part-init /dev/vda mbr
# numbers below are in units of 512 byte sectors:
><fs> part-add /dev/vda primary 64 8388607
><fs> part-add /dev/vda primary 8388608 -1
><fs> sfdisk-l /dev/vda

Disk /dev/vda: 10402 cylinders, 16 heads, 63 sectors/track
Units = cylinders of 516096 bytes, blocks of 1024 bytes, counting from 0

   Device Boot Start     End   #cyls    #blocks   Id  System
/dev/vda1          0+   8322-   8322-   4194272   83  Linux
/dev/vda2       8322+  10402-   2081-   1048576   83  Linux
/dev/vda3          0       -       0          0    0  Empty
/dev/vda4          0       -       0          0    0  Empty

Notice the number of (1024-byte) blocks for devswap is exactly the correct size: 1048576.

The sfdisk-l command is also telling me that my partitions aren’t aligned on “cylinders” which I don’t care about. But the swap partition should be aligned for the underlying device because sector 8388608 == 8192 * 1024.

Once the hard bit is out of the way, I can now copy across my filesystems. Notice I added devroot and devswap as devices (the -a option to guestfish). They appear in the guest as /dev/vdb and /dev/vdc respectively and I can just dd them to the right places:

><fs> dd /dev/vdb /dev/vda1
><fs> dd /dev/vdc /dev/vda2

and resize the root filesystem to fit the space available:

><fs> e2fsck-f /dev/vda1
><fs> resize2fs /dev/vda1

Now I have a single partitioned device, suitable for use with KVM (mind you, not bootable because it still contains a Xen paravirt kernel):

$ virt-list-filesystems -al disk.img
/dev/sda1 ext3
/dev/sda2 swap

As you can see there is much scope for automation …