RHEL 6.1 beta – libguestfs changes

RHEL 6.1 beta is available for customers on RHN.

The version of libguestfs in RHEL 6.1 is 1.7.17, but with 77 patches on top of the tarball it is essentially the same as the upstream stable branch 1.8.

New features are too numerous to mention, but the main thing is greatly improved performance. It’s 4x or 5x faster than the RHEL 6.0 version.

libguestfs 1.9.10 for RHEL 5.6

Here are some libguestfs packages for Red Hat Enterprise Linux 5.6:

http://libguestfs.org/download/binaries/rhel5-packages/

Read the README file before attempting to use them.

Updated libguestfs RHEL 6.1 preview packages

Updated packages are here now based on libguestfs 1.7.17.

Changes ahead for libguestfs RHEL 6.1 package

I previously said that libguestfs in RHEL 6.1 would be based on the recent upstream 1.6 release.

This plans have had to change slightly. It looks like we’ll rebase to 1.7.16 (a development version).

The reason is simply that to get into the next release of RHEV we had to remove the Perl dependencies on a number of key programs, because the tiny RHEV-H hypervisor [PDF] doesn’t have space to include Perl. Several programs like virt-inspector and virt-df had to be rewritten in C. We could backport all of the changes but they amount to nearly every change since 1.6 anyway.

What I do have to do is to meticulously check each C program precisely matches the old Perl version, in terms of output, command line arguments and so on, so that scripts written against RHEL 6.0 won’t break. But that’s what you pay Red Hat for.

Preview packages will be available here.

Red Hat Enterprise Linux 6

Looks like RHEL 6 is out.

Of course I’ve been using it for a while and it’s great.

This version has libguestfs-1.2.7-1.24.el6 (with backports) from back around April/May (when we started to stabilize and QA everything). Have fun using it, and when RHEL 6.1 ships it will come with libguestfs 1.6 which is much faster and more full-featured.

libguestfs roadmap for 1.6, RHEL 6.1

We have mostly finalized the list of bugs and features that we will finish before the new stable branch of libguestfs (1.6) is created. You can view the list of 1.6 bugs and features here. Although I can’t promise exact dates, we’re hoping to have this out in November.

libguestfs is also due to be rebased for RHEL 6.1, and that will (probably) be based on upstream version 1.6. The new version is much faster (about 4x or 5x faster) than the version we are shipping in RHEL 6.0, and has a very large number of new features, but is backwards compatible with RHEL 6.0 so no code or script changes will be required.

Tip: Run Xen as a KVM guest

(Thanks to Dan for pointing out that this is possible)

You can run legacy Xen hypervisor as a KVM guest, which is useful for testing and development. Since the Xen HV won’t have access to the hardware (in particular, to hardware virt) you can only run Xen paravirt guests this way, which in reality means only Linux XenPV guests. [It is supposed to be possible, if you have AMD hardware supporting nested SVN, to get Xen fullvirt guests going, but I did not try this.]

Preparation:

1. Install a RHEL 5 guest first. This will later be changed into a “Xen host” guest.
2. You will definitely appreciate having libguestfs around since you can do virt-edit RHEL5Xen /boot/grub/grub.conf to adjust the boot configuration iteratively.

Boot the RHEL 5 guest ordinarily, and install the appropriate kernel-xen and xen packages, and edit grub.conf to enable the Xen hypervisor:

default=0
[...]
title Red Hat Enterprise Linux Server (2.6.18-194.17.1.el5xen)
        root (hd0,0)
        kernel /xen.gz-2.6.18-194.17.1.el5 noapic
        module /vmlinuz-2.6.18-194.17.1.el5xen ro root=/dev/VolGroup00/LogVol00
        module /initrd-2.6.18-194.17.1.el5xen.img

I added the noapic option, and removed rhgb quiet so I could see what was going on.

Furthermore you need to switch the guest to using emulated devices (IDE, rtl8139) instead of virtio, if it’s not already. Shut down the guest and do:

# virsh edit RHEL5Xen

and change <target dev=’vda’ bus=’virtio’/> to <target dev=’sda’ bus=’ide’/> and remove any <address> element in the same <disk> block. Similarly if there is a network card, make it an emulated rtl8139 instead of virtio.

If during boot you see the error:

request_module: runaway loop modprobe binfmt-464c

then this means you’ve got the wrong kernel-xen package installed, usually that you’ve got the 32 bit kernel-xen.i686 with a 64 bit userspace.

As I said before, I really appreciated guestfish / virt-edit to let me interactively edit the grub configuration and go back and forwards between Xen and ordinary kernel until I got it right.

But finally it did boot, and it was running the Xen hypervisor and dom0:

# /etc/init.d/xend start
Starting xend:                                             [  OK  ]
# /usr/sbin/xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0      864     1 r-----     48.3
# uname -a
Linux rhel5xenx64.home.annexia.org 2.6.18-194.17.1.el5xen #1 SMP Mon Sep 20 07:20:39 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux

Update #1

If you have libvirt running inside this guest, it will try to register a bridged network with the address 192.168.122.x which is the exact same address as your host, and that causes confusion. Typical symptoms are that you can’t ssh into the Xen guest from the host, and that networking inside the Xen guest seems broken in strange ways. The solution is simple. In the Xen guest type:

# virsh net-edit default

and change “122″ everywhere to something else, eg. “123″.

Then restart the default network:

# virsh net-destroy default
# virsh net-start default

and everything will work again.

Update #2

Because this KVM guest is running a Xen hypervisor and guests, you need to give it a bit more memory. I bumped mine up to 2G, allowing me to comfortably install and run one (nested) guest:

# /usr/sbin/xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0     1224     1 r-----    103.8
RHEL5xenguest                              1      767     1 -b----     33.0

libguestfs versions

A useful table (original posting and explanation of version numbers):

Rawhide	1.5.x (currently 1.5.0)
Fedora 13	1.4.x (currently 1.4.1)
Fedora 12	1.2.x (currently 1.2.10)
EPEL 5	1.2.x (currently 1.2.10)
RHEL 6.0	1.2.7 + backported fixes and features needed for V2V
RHEL 6.1	undecided, probably rebase to latest 1.2.x or 1.4.x
RHEL 5.5 with RHEV-M subscription	1.2.7 + backported fixes and features needed for V2V

Tip: mock-build Rawhide packages on RHEL 5

The Fedora build system Koji runs on RHEL 5 Xen and builds everything on top of that using mock. This can lead to some rather difficult to debug problems where your package builds and tests OK for you on your local Rawhide machine, but fails in Koji. The reason it can fail in Koji is because it is running on the RHEL 5 Linux kernel (2.6.18). Your program, or any program you depend on during the build, might make assumptions about system calls that work for a Rawhide kernel, but fail for a RHEL 5 kernel.

Reproducing these bugs is difficult. Hopefully this posting should be a good start.

Koji is doing roughly the equivalent of this command (on a RHEL 5 host):

mock -r fedora-rawhide-x86_64 --rebuild your.src.rpm

That command doesn’t work straightaway. There are some things you have to install and upgrade first before that works:

1. Install RHEL 5 (or use CentOS or another no-cost alternative).
2. Install EPEL.
3. Install or update yum, python-hashlib, mock.
4. Use /usr/sbin/vigr to add yourself to the “mock” group.
5. The version of RPM from RHEL 5 is too old to understand the new xz-based compression format used by Rawhide RPMs. You have to build the Fedora 12 RPM (NB: Fedora 13 RPM definitely doesn’t work because it requires Python 2.6). The Fedora 12 specfile is a starting point, but it won’t work directly. There are some small changes you have to make, and a single patch to the source code, but hopefully those will be obvious. Update: Here for a short time is a scratch build of the Fedora 12 RPM made to work on RHEL 5.4. Once you’ve built the new rpm RPM (!), install it.

At this point you can use the mock command above to test-build SRPMs using the unusual RHEL 5 kernel / Rawhide userspace combination.