Upstream qemu can access a variety of remote disks, like NBD and Ceph. This feature is exposed in libguestfs so you can easily mount remote storage.
However in RHEL 7 many of these drivers are disabled, because they’re not stable enough to support. I was asked exactly how this works, and this post is my answer — as it’s not as simple as it sounds.
There are (at least) five separate layers involved:
qemu code |
What block drivers are compiled into qemu, and which ones are compiled out completely. |
qemu block driver r/o whitelist |
A whitelist of drivers that qemu allows you to use read-only. |
qemu block driver r/w whitelist |
A whitelist of drivers that qemu allows you to use for read and write. |
libvirt |
What libvirt enables (not covered in this discussion). |
libguestfs |
In RHEL we patch out some qemu remote storage types using a custom patch. |
Starting at the bottom of the stack, in RHEL we use ./configure --disable-*
flags to disable a few features: Ceph is disabled on !x86_64
and 9pfs
is disabled everywhere. This means the qemu binary won’t even contain code for those features.
If you run qemu-img --help
in RHEL 7, you’ll see the drivers which are compiled into the binary:
$ rpm -qf /usr/bin/qemu-img
qemu-img-1.5.3-92.el7.x86_64
$ qemu-img --help
[...]
Supported formats: vvfat vpc vmdk vhdx vdi ssh
sheepdog rbd raw host_cdrom host_floppy host_device
file qed qcow2 qcow parallels nbd iscsi gluster dmg
tftp ftps ftp https http cloop bochs blkverify blkdebug
Although you can use all of those in qemu-img
, not all of those drivers work in qemu
(the hypervisor). qemu implements two whitelists. The RHEL 7 qemu-kvm.spec
file looks like this:
./configure [...]
--block-drv-rw-whitelist=qcow2,raw,file,host_device,blkdebug,nbd,iscsi,gluster,rbd \
--block-drv-ro-whitelist=vmdk,vhdx,vpc,ssh,https
The --block-drv-rw-whitelist
parameter configures the drivers for which full read and write access is permitted and supported in RHEL 7. It’s quite a short list!
Even shorter is the --block-drv-ro-whitelist
parameter — drivers for which only read-only access is allowed. You can’t use qemu to open these files for write. You can use these as (read-only) backing files, but you can’t commit to those backing files.
In practice what happens is you get an error if you try to use non-whitelisted block drivers:
$ /usr/libexec/qemu-kvm -drive file=test.vpc
qemu-kvm: -drive file=test.vpc: could not open disk image
test.vpc: Driver 'vpc' can only be used for read-only devices
$ /usr/libexec/qemu-kvm -drive file=test.qcow1
qemu-kvm: -drive file=test.qcow1: could not open disk
image test.qcow1: Driver 'qcow' is not whitelisted
Note that’s a qcow v1 (ancient format) file, not modern qcow2.
Side note: Only qemu
(the hypervisor) enforces the whitelist. Tools like qemu-img
ignore it.
At the top of the stack, libguestfs has a patch which removes support for many remote protocols. Currently (RHEL 7.2/7.3) we disable: ftp, ftps, http, https, tftp, gluster, iscsi, sheepdog, ssh. That leaves only: local file, rbd (Ceph) and NBD enabled.
virt-v2v uses a mixture of libguestfs and qemu-img
to convert VMware and Xen guests to run on KVM. To access VMware we need to use https
and to access Xen we use ssh
. Both of those drivers are disabled in libguestfs, and only available read-only in the qemu whitelist. However that’s sufficient for virt-v2v, since all it does is add the https or ssh driver as a read-only backing file. (If you are interested in finding out more about how virt-v2v works, then I gave a talk about it at the KVM Forum which is available online).
In summary — it’s complicated.