This seems to be completely undocumented which is why I’m writing this … It is possible to boot a Linux guest (Fedora in this case) from a live CD on a website without downloading it. I’m using our favourite flexible NBD server, nbdkit and virt-install.
First of all we’ll run nbdkit and attach it to the Fedora 29 live workstation ISO. To make this work more efficiently I’m going to place a couple of filters on top — one is the readahead (prefetch) filter recently added to nbdkit 1.12, and the other is the cache filter. In combination these filters should reduce the load on the website and improve local performance.
$ rm /tmp/socket
$ nbdkit -f -U /tmp/socket --filter=readahead --filter=cache \
curl https://download.fedoraproject.org/pub/fedora/linux/releases/29/Workstation/x86_64/iso/Fedora-Workstation-Live-x86_64-29-1.2.iso
I actually replaced that URL with a UK-based mirror to make the process a little faster.
Now comes the undocumented virt-install command:
$ virt-install --name test --ram 2048 \
--disk /var/tmp/disk.img,size=10
--disk device=cdrom,source_protocol=nbd,source_host_transport=unix,source_host_socket=/tmp/socket \
--os-variant fedora29
After a bit of grinding that should boot into Fedora 29, and you never (not explicitly at least) had to download the ISO.
To be fair qemu does also have a curl driver which virt-install could use, but nbdkit is better with the filters and plugins system giving you ultimate flexibility — check out my video about it.
Richard WM Jones 
Does this monolithically download the entire ISO before it boots or is it opportunistic? I’d expect the latter because of the nested imaging being used in the compose process. The random access file system we care about is on root.img which is nested on a squashfs.img with xz compression and that’s a (nested) file on the ISO 9660 formatted image that is the ISO itself. To make heads or tails out of out of any single file, it’d probably have to download the whole image first. If so, that could be an opportunity for an optimized image, i.e. skip the nested root.img, and use squashfs directly for random seeks and reads.
Hi Rich. It looks like this no longer works well.
First, the virt-install step (which has some missing \ in the code sample) now requires an install method. Otherwise you get
An install method must be specified
(–location URL, –cdrom CD/ISO, –pxe, –import, –boot hd|cdrom|…)
I tried –boot cdrom, but despite being given as an example, it just gives me the same message.
If I try –import, then I get:
ERROR internal error: process exited while connecting to monitor: 2023-03-01T16:56:36.768751Z qemu-system-x86_64: -blockdev {“driver”:”nbd”,”server”:{“type”:”unix”,”path”:”/tmp/socket”},”node-name”:”libvirt-1-storage”,”auto-read-only”:true,”discard”:”unmap”}: Failed to connect to ‘/tmp/socket’: Permission denied
Yup, I can reproduce both those findings. Oh well!
The first one is a virt-install bug. In theory this should work (but does not because libvirt doesn’t parse the NBD URI correctly):
The second one is an SELinux thing. You can get around this by adding the
nbdkit --selinux-labelflag + chcon command, see here.BTW we rewrote the readahead filter since this article was written, and although it’s better in general, it’s currently incompatible with the curl plugin. You’ll see a harmless warning about this.
SELinux does not seem to be the problem. After a few rounds trying your solution (part of the problem being that chcon seems to not be allowed to operate on a socket in /tmp, I got some SELinux denials for that), I finally used the big hammer of “setenforce 0”, and I still get the “Permission denied”. I’ll keep looking and keep you posted.
So apparently, the permission problem is a combination of:
– Running nbdkit as root in my case, whereas the virt-install qemu runs as qemu, meaning you need a chmod or chown on the socket to make sure the qemu process can access it.
– SELinux label for the socket, which can be addressed as indicated by the link Rich gave above.
– chcon silently failing if the socket is under /tmp/. You know it failed because there is an AVC denial in /var/log/audit/audit.log, but the command itself does not tell you it failed. Workaround is to put the socket file under /run/libvirt/
With all three, then it works.