Notes: compiling libguestfs from source on Ubuntu 10.04

These are just notes that I made while compiling libguestfs on Ubuntu.

Note (1): Interested in maintaining libguestfs on Ubuntu? We really need a dedicated, skilled packager to help us out.

Note (2): More information is available on the libguestfs homepage, and for detailed help please send a message to the mailing list.

Start with the tarball. I am using libguestfs-1.5.12.tar.gz from here.

You need to install a boatload of dependencies. Luckily, they are all (except one) in Ubuntu repositories already, so they are just an “apt-get” and a moderate wait away:

$ sudo apt-get install \
  libfindlib-ocaml-dev ocaml-native-compilers \
  libxml-light-ocaml-dev \
  debootstrap debirf \
  libpcre3-dev libaugeas-dev libmagic-dev \
  libxml2-dev libvirt-dev \
  kvm \
  libreadline-dev \
  gcc make \
  po4a

You need libhivex0 and libhivex-dev too. Ubuntu doesn’t have these, but these two packages from Debian squeeze worked fine for me: libhivex0, libhivex-dev.

Unpack the tarball and configure like this:

$ ./configure \
  --with-repo=lucid \
  --with-mirror=http://gb.archive.ubuntu.com/ubuntu/

If you’re using a different version of Ubuntu from me, change “lucid” to whatever is appropriate. It needs to exactly match your host version.

Obviously ./configure needs to get all the way through. If there is a missing dependency, install it too. If all goes well it will print a summary of what it is going to compile which will look something like this:

Thank you for downloading libguestfs 1.5.12

This is how we have configured the optional components for you today:

Daemon .............................. yes
Appliance ........................... yes
QEMU ................................ /usr/bin/qemu-system-x86_64
OCaml bindings ...................... yes
Perl bindings ....................... yes
Python bindings ..................... no
Ruby bindings ....................... no
Java bindings ....................... no
Haskell bindings .................... no
PHP bindings ........................ no
virt-inspector ...................... no
virt-* tools ........................ no
supermin appliance .................. no
FUSE filesystem ..................... no

If you want more of those features to be enabled, you’ll need to track down the missing optional dependencies by reading back over the full configure output.

Build:

$ make

During the build, libguestfs will build a small Ubuntu-based appliance using debirf. It needs network access to fetch the .debs for this (using squid can help a lot). It’s also possible to use a local repository in which case network access won’t be required, although it doesn’t work by magic — you still have to supply the .debs from somewhere.

You might need to adjust appliance/packagelist.in and/or files in appliance/debian/modules/ if a package is missing or if appliance building fails. On Ubuntu 10.04 I had to comment out the “find” statement where it was trying to delete unused kernel modules (bug in debirf I suspect) and comment out the apt-get remove command.

Building the appliance is generally the most error-prone part of the whole process because it depends on infrequently tested parts of Debian such as debirf, debootstrap, fakeroot and packages themselves. If you have no patience for this you can grab the pre-built binary appliance from our binary distribution (also use ./configure --disable-appliance).

Now once you’re through the build, check that things work. You can do a quick “does it work at all” check:

$ make quickcheck

(Obviously if this fails, you need to go back and take a hard look at the build).

Finally run the full test suite:

$ make check

[Fix the bugs and submit patches!]

You can now try running guestfish etc.

5 Comments

Filed under Uncategorized

5 responses to “Notes: compiling libguestfs from source on Ubuntu 10.04

  1. Michael

    “During the build, libguestfs will build a small Ubuntu-based appliance using debirf. It needs network access to fetch the .debs for this (using squid can help a lot). It’s also possible to use a local repository in which case network access won’t be required, although it doesn’t work by magic — you still have to supply the .debs from somewhere.”

    Can you explain this a little more? I have been stuck on make for over 20 minutes downloading and unpacking all sorts of stuff that doesn’t seem to make sense, eg Plymouth?

  2. Michael

    Another question – how do you use the binary appliance?

    • rich

      Sorry for the delayed reply — I’ve been on holiday.

      We do indeed build a small appliance, because that’s how libguestfs works, ie. how we are able to reuse all the Linux code to be able to read every partition scheme and filesystem type under the sun.

      On Fedora we use a program called febootstrap to do this. On Debian/Ubuntu we use debirf + debootstrap to do the same. Both febootstrap and debirf install packages in a chroot environment (without needing root). They work completely differently, one is based on rpm + yum, the other on dpkg + apt-get, but the outcome is the same on both.

      Unfortunately debirf (or debootstrap) is kinda buggy, or at least the version that ships in Ubuntu is often buggy, and generally requires some level of tweaking to make it work at all. We mainly concentrate on getting febootstrap working, where we control everything, and that’s how we reliably produce regular builds for Fedora and the binary appliance. It would be great to have someone take a good look at debirf and make it robust.

      In the meantime, and to answer your other question, to use the binary appliance download it and pull out the two files in usr/local/lib/guestfs:

      rjones@ubuntu1004x64:~/tmp/usr/local/lib/guestfs$ ll
      total 43784
      drwxr-xr-x 2 rjones rjones     4096 2010-09-11 14:19 ./
      drwxr-xr-x 8 rjones rjones     4096 2010-09-11 14:19 ../
      -rw-r--r-- 1 rjones rjones 40854484 2010-09-11 14:19 initramfs.fedora-12.x86_64.img
      -rw-r--r-- 1 rjones rjones  3967824 2010-09-11 14:19 vmlinuz.fedora-12.x86_64
      

      Copy these two files into the appliance/ subdirectory of your build. Then rebuild without building the appliance or daemon (since you’ve got a binary appliance, you don’t need to build either the daemon or the appliance which contains it):

      ./configure --with-repo=fedora-12 --disable-appliance --disable-daemon
      make clean
      make
      

      (Note that the repo name there is meaningless except that it must match the filename of the binary appliance. It just changes the name that libguestfs looks for. Encoding the repo name in the appliance filename was a stupid idea in hindsight).

      Then you can try out guestfish etc.

      ./fish/guestfish [...]
      ./tools/run-locally df disk.img
      

      make quickcheck and make check are also a good idea.

      Note also that the library/appliance protocol changes occasionally, so you should make sure that the versions are similar or the same, and watch for changes to the protocol which are announced on the mailing list.

      Three other issues you should note about the binary appliance:

      1. it’s much slower (because of a big qemu/KVM performance regression in Linux 2.6.35 that we can’t workaround when supplying the binary appliance)
      2. it requires a recent version of qemu which supports virtio-serial; if you have problems, compile the latest qemu and/or KVM from source
      3. if you get a hang after “No volume groups found” then you’ve hit #630583, a bug in Linux 2.6.36.
  3. Pingback: Building libguestfs from source with the binary appliance « Richard WM Jones

    • Michael

      Thanks for this/these tutorial(s).

      Ubuntu 10.04 does not ship with virtio-serial, or I at least get “The virtio-serial device has not been found” when building with the binary application.

      Does the comment about compiling qemu from source apply to this post as well, or just when using the binary application? I think if it is the former you ought to mention it in this post.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s