Using virt-builder to build packages

I spent yesterday seeing if virt-builder could be used to safely build or test packages in custom virtual machines. It’s a bit complicated, but the answer is yes.

Building a libvirt RPM takes 6 minutes, versus about 2 minutes when run locally. Most of the overhead is fixed, ie. it would be the same overhead whatever package you were building. Virt-builder takes just over 2 minutes to run.

The whole script is at the bottom, but I’ll just talk about the virt-builder command itself. Here is the command with my comments added:

I’m running virt-builder from git because I added a few features to make this easier:

$d/run $d/builder/virt-builder

$guest_type is the base guest, eg. fedora-19:

  $guest_type

Select the size and temporary output file:

  --size 20G
  --output /tmp/$guest_type.img

Don’t bother relabelling as we’re not going to use SELinux:

  --delete /.autorelabel

Create a /home/build directory which contains the SRPM, the build script (a small wrapper around rpmbuild), and a configuration file that has the name of the SRPM file for the build script. --mkdir and --write options are new features which are not in any released version of virt-builder yet:

  --mkdir /home/build
  --upload $srpmdir/$srpm:/home/build
  --upload $build_script:/home/build
  --write "/home/build/config:srpm=$srpm"

Install the basic build system and the build dependencies of the SRPM:

  --install /usr/bin/yum-builddep,\
     /usr/bin/rpmbuild,\
     @buildsys-build,\
     @development-tools
  --run-command "yum-builddep -y /home/build/$srpm"

When the guest boots, we create a builder user account, run rpmbuild, and then power off the machine:

  --firstboot-command 'useradd -m -p "" builder'
  --firstboot-command 'chown builder.builder /home/build'
  --firstboot-command 'su - builder -c /home/build/build-it.sh'
  --firstboot-command 'poweroff'

The entire build script runs virt-builder as above, then runs qemu to boot the VM, then copies out the resulting RPMs.

#!/bin/bash -

set -e
set -x

# Use a homebrew compile of libguestfs from git.
d=$HOME/d/libguestfs

# This is the Fedora platform we want to build on.
guest_type=fedora-19

# The SRPM we want to build.
srpmdir=/home/rjones/d/fedora/libvirt/f19
srpm=libvirt-1.0.5.6-3.fc19.src.rpm

# The build script.
build_script=/tmp/build-it.sh
# Because virt-builder copies the build script permissions too.
chmod +x $build_script

# How much guest memory we need for the build:
export LIBGUESTFS_MEMSIZE=4096

# Run virt-builder.
# Use a long build path to work around RHBZ#757089.
$d/run $d/builder/virt-builder \
  $guest_type \
  --size 20G \
  --output /tmp/$guest_type.img \
  --delete /.autorelabel \
  --mkdir /home/build \
  --upload $srpmdir/$srpm:/home/build \
  --upload $build_script:/home/build \
  --write "/home/build/config:srpm=$srpm" \
  --install /usr/bin/yum-builddep,/usr/bin/rpmbuild,@buildsys-build,@development-tools \
  --run-command "yum-builddep -y /home/build/$srpm" \
  --firstboot-command 'useradd -m -p "" builder' \
  --firstboot-command 'chown builder.builder /home/build' \
  --firstboot-command 'su - builder -c /home/build/build-it.sh' \
  --firstboot-command 'poweroff'

# Run qemu directly.  Could also use virt-install --import here.
qemu-system-x86_64 \
  -nodefconfig \
  -nodefaults \
  -nographic \
  -machine accel=kvm:tcg \
  -cpu host \
  -m 2048 \
  -smp 4 \
  -net user \
  -serial stdio \
  -drive file=/tmp/$guest_type.img,format=raw,if=virtio,cache=unsafe

# The build ran OK if this contains the magic string (see $build_script).
virt-cat -a /tmp/$guest_type.img /root/virt-sysprep-firstboot.log

# Copy out the SRPMs & RPMs.
rm -rf /tmp/result
mkdir /tmp/result
virt-copy-out -a /tmp/$guest_type.img /home/build/RPMS /home/build/SRPMS \
  /tmp/result

# Leave the guest around so you can examine the /home/build dir if you want.
# Or you could delete it.
#rm /tmp/$guest_type.img

The build-it script is just a small wrapper around rpmbuild:

#!/bin/bash -

# This runs inside the throwaway guest at first boot.

set -e
set -x

cd /home/build
source config

# Build from SRPM.
rpmbuild --define '_topdir /home/build' --rebuild /home/build/$srpm

# If we get this far, everything built successfully.
# This string is detected in the guest afterwards.
echo '=== BUILD FINISHED OK ==='