Caseless virtualization cluster, part 3

virt-builder can be used to build baremetal images on (eg) USB keys, so that’s what I’m using to create the bootable keys for my virtualization cluster:

$ virt-builder fedora-20 --update -o /dev/sdX
[   1.0] Downloading: http://libguestfs.org/download/builder/fedora-20.xz
[   1.0] Planning how to build this image
[   1.0] Uncompressing
[  13.0] Resizing (using virt-resize) to expand the disk to 14.9G
[1497.0] Opening the new disk
[1501.0] Setting a random seed
[1501.0] Updating core packages
[1693.0] Setting passwords
Setting random password of root to pNooenUMhHz8n6iX
[1693.0] Finishing off
Output: /dev/sdc
Output size: 14.9G
Output format: raw
Total usable space: 13.9G
Free space: 12.9G (92%)

One small fact that will save you a lot of hair-pulling: This motherboard will not boot from the blue USB 3.0 ports! You have to put the USB key into one of the regular (black) USB ports.

As you can see from the timings (left hand column of virt-builer output above), writing to cheap consumer USB keys is slooowwwww. If we were using even hard disks, virt-builder would have built that image in under 60 seconds.

The OS has to be modified to avoid writes as far as possible. I’m using NFS for home directories, logging remotely with rsyslog, and exploring where to store the VM disk images. (And, yes, /tmp is on tmpfs here — it makes sense in this application)

I did some simple experiments using my development machine as an NFS (v4) server. The two machines are connected through a consumer gigabit ethernet switch. The NFS server has 32 GB of RAM and SSDs and of course the virtualization cluster is “diskless” (just a USB key to boot). Performance is pretty good:

Reads: 115 MBytes/sec
Writes: 62 MBytes/sec

One final (anticipated) problem with a caseless system is that it generates large amounts of radio frequency interference. You can show this simply by putting a transistor radio next to the machine. I have a plan to build a metal case which should reduce this.

1 Comment

Filed under Uncategorized

One response to “Caseless virtualization cluster, part 3

  1. Pingback: Caseless virtualization cluster, part 4 | Richard WM Jones

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s