Upstream qemu can access a variety of remote disks, like NBD and Ceph. This feature is exposed in libguestfs so you can easily mount remote storage.
However in RHEL 7 many of these drivers are disabled, because they’re not stable enough to support. I was asked exactly how this works, and this post is my answer — as it’s not as simple as it sounds.
There are (at least) five separate layers involved:
qemu code | What block drivers are compiled into qemu, and which ones are compiled out completely. |
---|---|
qemu block driver r/o whitelist | A whitelist of drivers that qemu allows you to use read-only. |
qemu block driver r/w whitelist | A whitelist of drivers that qemu allows you to use for read and write. |
libvirt | What libvirt enables (not covered in this discussion). |
libguestfs | In RHEL we patch out some qemu remote storage types using a custom patch. |
Starting at the bottom of the stack, in RHEL we use ./configure --disable-*
flags to disable a few features: Ceph is disabled on !x86_64
and 9pfs
is disabled everywhere. This means the qemu binary won’t even contain code for those features.
If you run qemu-img --help
in RHEL 7, you’ll see the drivers which are compiled into the binary:
$ rpm -qf /usr/bin/qemu-img qemu-img-1.5.3-92.el7.x86_64 $ qemu-img --help [...] Supported formats: vvfat vpc vmdk vhdx vdi ssh sheepdog rbd raw host_cdrom host_floppy host_device file qed qcow2 qcow parallels nbd iscsi gluster dmg tftp ftps ftp https http cloop bochs blkverify blkdebug
Although you can use all of those in qemu-img
, not all of those drivers work in qemu
(the hypervisor). qemu implements two whitelists. The RHEL 7 qemu-kvm.spec
file looks like this:
./configure [...] --block-drv-rw-whitelist=qcow2,raw,file,host_device,blkdebug,nbd,iscsi,gluster,rbd \ --block-drv-ro-whitelist=vmdk,vhdx,vpc,ssh,https
The --block-drv-rw-whitelist
parameter configures the drivers for which full read and write access is permitted and supported in RHEL 7. It’s quite a short list!
Even shorter is the --block-drv-ro-whitelist
parameter — drivers for which only read-only access is allowed. You can’t use qemu to open these files for write. You can use these as (read-only) backing files, but you can’t commit to those backing files.
In practice what happens is you get an error if you try to use non-whitelisted block drivers:
$ /usr/libexec/qemu-kvm -drive file=test.vpc qemu-kvm: -drive file=test.vpc: could not open disk image test.vpc: Driver 'vpc' can only be used for read-only devices $ /usr/libexec/qemu-kvm -drive file=test.qcow1 qemu-kvm: -drive file=test.qcow1: could not open disk image test.qcow1: Driver 'qcow' is not whitelisted
Note that’s a qcow v1 (ancient format) file, not modern qcow2.
Side note: Only qemu
(the hypervisor) enforces the whitelist. Tools like qemu-img
ignore it.
At the top of the stack, libguestfs has a patch which removes support for many remote protocols. Currently (RHEL 7.2/7.3) we disable: ftp, ftps, http, https, tftp, gluster, iscsi, sheepdog, ssh. That leaves only: local file, rbd (Ceph) and NBD enabled.
virt-v2v uses a mixture of libguestfs and qemu-img
to convert VMware and Xen guests to run on KVM. To access VMware we need to use https
and to access Xen we use ssh
. Both of those drivers are disabled in libguestfs, and only available read-only in the qemu whitelist. However that’s sufficient for virt-v2v, since all it does is add the https or ssh driver as a read-only backing file. (If you are interested in finding out more about how virt-v2v works, then I gave a talk about it at the KVM Forum which is available online).
In summary — it’s complicated.
I installed libguestfs on centos 7.1, but virt-v2v is not there. You know where I can install it from? Also, will it allow taking a hyper-v windows 2008vm and load it as a openstack glance image suitable for instance creation?
See this thread. Hyper-V: maybe, but it’s not been recently tested. Openstack glance: yes.
Actually for Hyper-V you’re going to need virt-p2v too.
Thanks for the document.
In teh future – please include the path of the file you are referring to, e.g. qemu-kvm.spec . i know this maybe be relevant to a distro but it ease the searching of the file as i am doing now on centos 7
The qemu-kvm.spec file is not linked anywhere directly as far as I know, but it’s included in the qemu-kvm source RPM provided by the CentOS project.