Upstream qemu can access a variety of remote disks, like NBD and Ceph. This feature is exposed in libguestfs so you can easily mount remote storage.
However in RHEL 7 many of these drivers are disabled, because they’re not stable enough to support. I was asked exactly how this works, and this post is my answer — as it’s not as simple as it sounds.
There are (at least) five separate layers involved:
|qemu code||What block drivers are compiled into qemu, and which ones are compiled out completely.|
|qemu block driver r/o whitelist||A whitelist of drivers that qemu allows you to use read-only.|
|qemu block driver r/w whitelist||A whitelist of drivers that qemu allows you to use for read and write.|
|libvirt||What libvirt enables (not covered in this discussion).|
|libguestfs||In RHEL we patch out some qemu remote storage types using a custom patch.|
Starting at the bottom of the stack, in RHEL we use
./configure --disable-* flags to disable a few features: Ceph is disabled on
9pfs is disabled everywhere. This means the qemu binary won’t even contain code for those features.
If you run
qemu-img --help in RHEL 7, you’ll see the drivers which are compiled into the binary:
$ rpm -qf /usr/bin/qemu-img qemu-img-1.5.3-92.el7.x86_64 $ qemu-img --help [...] Supported formats: vvfat vpc vmdk vhdx vdi ssh sheepdog rbd raw host_cdrom host_floppy host_device file qed qcow2 qcow parallels nbd iscsi gluster dmg tftp ftps ftp https http cloop bochs blkverify blkdebug
Although you can use all of those in
qemu-img, not all of those drivers work in
qemu (the hypervisor). qemu implements two whitelists. The RHEL 7
qemu-kvm.spec file looks like this:
./configure [...] --block-drv-rw-whitelist=qcow2,raw,file,host_device,blkdebug,nbd,iscsi,gluster,rbd \ --block-drv-ro-whitelist=vmdk,vhdx,vpc,ssh,https
--block-drv-rw-whitelist parameter configures the drivers for which full read and write access is permitted and supported in RHEL 7. It’s quite a short list!
Even shorter is the
--block-drv-ro-whitelist parameter — drivers for which only read-only access is allowed. You can’t use qemu to open these files for write. You can use these as (read-only) backing files, but you can’t commit to those backing files.
In practice what happens is you get an error if you try to use non-whitelisted block drivers:
$ /usr/libexec/qemu-kvm -drive file=test.vpc qemu-kvm: -drive file=test.vpc: could not open disk image test.vpc: Driver 'vpc' can only be used for read-only devices $ /usr/libexec/qemu-kvm -drive file=test.qcow1 qemu-kvm: -drive file=test.qcow1: could not open disk image test.qcow1: Driver 'qcow' is not whitelisted
Note that’s a qcow v1 (ancient format) file, not modern qcow2.
Side note: Only
qemu (the hypervisor) enforces the whitelist. Tools like
qemu-img ignore it.
At the top of the stack, libguestfs has a patch which removes support for many remote protocols. Currently (RHEL 7.2/7.3) we disable: ftp, ftps, http, https, tftp, gluster, iscsi, sheepdog, ssh. That leaves only: local file, rbd (Ceph) and NBD enabled.
virt-v2v uses a mixture of libguestfs and
qemu-img to convert VMware and Xen guests to run on KVM. To access VMware we need to use
https and to access Xen we use
ssh. Both of those drivers are disabled in libguestfs, and only available read-only in the qemu whitelist. However that’s sufficient for virt-v2v, since all it does is add the https or ssh driver as a read-only backing file. (If you are interested in finding out more about how virt-v2v works, then I gave a talk about it at the KVM Forum which is available online).
In summary — it’s complicated.
5 responses to “virt-v2v, libguestfs and qemu remote drivers in RHEL 7”
I installed libguestfs on centos 7.1, but virt-v2v is not there. You know where I can install it from? Also, will it allow taking a hyper-v windows 2008vm and load it as a openstack glance image suitable for instance creation?
See this thread. Hyper-V: maybe, but it’s not been recently tested. Openstack glance: yes.
Actually for Hyper-V you’re going to need virt-p2v too.
Thanks for the document.
In teh future – please include the path of the file you are referring to, e.g. qemu-kvm.spec . i know this maybe be relevant to a distro but it ease the searching of the file as i am doing now on centos 7
The qemu-kvm.spec file is not linked anywhere directly as far as I know, but it’s included in the qemu-kvm source RPM provided by the CentOS project.