FROM fedora RUN dnf install -y libguestfs libguestfs-tools-c virt-v2v \ libvirt-daemon libvirt-daemon-config-network # https://bugzilla.redhat.com/show_bug.cgi?id=1045069 RUN useradd -ms /bin/bash v2v USER v2v WORKDIR /home/v2v # This is required for virt-v2v because neither systemd nor # root libvirtd runs, and therefore there is no virbr0, and # therefore virt-v2v cannot set up the network through libvirt. ENV LIBGUESTFS_BACKEND direct
Tag Archives: virt-tools
As usual I’ve placed the proposed RHEL 7.5 libguestfs packages in a public repository so you can try them out.
Thanks to Pino Toscano for doing the packaging work.
Debian 9 (“Stretch”) was released last week and now it’s available in virt-builder, the fast way to build virtual machine disk images:
$ virt-builder -l | grep debian debian-6 x86_64 Debian 6 (Squeeze) debian-7 sparc64 Debian 7 (Wheezy) (sparc64) debian-7 x86_64 Debian 7 (Wheezy) debian-8 x86_64 Debian 8 (Jessie) debian-9 x86_64 Debian 9 (stretch) $ virt-builder debian-9 \ --root-password password:123456 [ 0.5] Downloading: http://libguestfs.org/download/builder/debian-9.xz [ 1.2] Planning how to build this image [ 1.2] Uncompressing [ 5.5] Opening the new disk [ 15.4] Setting a random seed virt-builder: warning: random seed could not be set for this type of guest [ 15.4] Setting passwords [ 16.7] Finishing off Output file: debian-9.img Output size: 6.0G Output format: raw Total usable space: 3.9G Free space: 3.1G (78%) $ qemu-system-x86_64 \ -machine accel=kvm:tcg -cpu host -m 2048 \ -drive file=debian-9.img,format=raw,if=virtio \ -serial stdio
libguestfs is a C library for creating and editing disk images. In the most common (but not the only) configuration, it uses KVM to sandbox access to disk images. The C library talks to a separate daemon running inside a KVM appliance, as in this Unicode-art diagram taken from the fine manual:
┌───────────────────┐ │ main program │ │ │ │ │ child process / appliance │ │ ┌──────────────────────────┐ │ │ │ qemu │ ├───────────────────┤ RPC │ ┌─────────────────┐ │ │ libguestfs ◀╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍╍▶ guestfsd │ │ │ │ │ ├─────────────────┤ │ └───────────────────┘ │ │ Linux kernel │ │ │ └────────┬────────┘ │ └───────────────│──────────┘ │ │ virtio-scsi ┌──────┴──────┐ │ Device or │ │ disk image │ └─────────────┘
The library has to be written in C because it needs to be linked to any main program. The daemon (
guestfsd in the diagram) is also written in C. But there’s not so much a specific reason for that, except that’s what we did historically.
The daemon is essentially a big pile of functions, most corresponding to a libguestfs API. Writing the daemon in C is painful to say the least. Because it’s a long-running process running in a memory-constrained environment, we have to be very careful about memory management, religiously checking every return from
strdup etc., making even the simplest task non-trivial and full of untested code paths.
So last week I modified libguestfs so you can now write APIs in OCaml if you want to. OCaml is a high level language that compiles down to object files, and it’s entirely possible to link the daemon from a mix of C object files and OCaml object files. Another advantage of OCaml is that you can call from C ↔ OCaml with relatively little glue code (although a disadvantage is that you still need to write that glue mostly by hand). Most simple calls turn into direct CALL instructions with just a simple bitshift required to convert between ints and bools on the C and OCaml sides. More complex calls passing strings and structures are not too difficult either.
OCaml also turns memory errors into a single exception, which unwinds the stack cleanly, so we don’t litter the code with memory handling. We can still run the mixed C/OCaml binary under valgrind.
Code gets quite a bit shorter. For example the case_sensitive_path API — all string handling and directory lookups — goes from 183 lines of C code to 56 lines of OCaml code (and much easier to understand too).
I’m reimplementing a few APIs in OCaml, but the plan is definitely not to convert them all. I think we’ll have C and OCaml APIs in the daemon for a very long time to come.
virt-inspector is a very convenient tool to examine a disk image and find out if it contains an operating system, what applications are installed and so on.
nbdkit xz file=win7.img.xz \ -U - \ --run 'virt-inspector --format=raw -a nbd://?socket=$unixsocket'
What’s happening here is we run nbdkit with the xz plugin, and tell it to serve NBD over a randomly named Unix domain socket (
We then run virt-inspector as a sub-process. This is called “captive nbdkit”. (Nbdkit is “captive” here, because it will exit as soon as virt-inspector exits, so there’s no need to clean anything up.)
$unixsocket variable expands to the name of the randomly generated Unix domain socket, forming a libguestfs NBD URL which allows virt-inspector to examine the raw uncompressed data exported by nbdkit.
The nbdkit xz plugin only uncompresses those blocks of the data which are actually accessed, so this is quite efficient.
$ ./run ./utils/boot-benchmark/boot-benchmark Warming up the libguestfs cache ... Running the tests ... test version: libguestfs 1.33.28 test passes: 10 host version: Linux moo.home.annexia.org 4.4.4-301.fc23.x86_64 #1 SMP Fri Mar 4 17:42:42 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux host CPU: Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz backend: direct [to change set $LIBGUESTFS_BACKEND] qemu: /home/rjones/d/qemu/x86_64-softmmu/qemu-system-x86_64 [to change set $LIBGUESTFS_HV] qemu version: QEMU emulator version 2.5.94, Copyright (c) 2003-2008 Fabrice Bellard smp: 1 [to change use --smp option] memsize: 500 [to change use --memsize option] append: [to change use --append option] Result: 575.9ms ±5.3ms
There are various tricks here:
- I’m using the (still!) not upstream qemu DMA patches.
- I’ve compiled my own very minimal guest Linux kernel.
- I’m using my nearly upstream
"crypto: Add a flag allowing the self-tests to be disabled at runtime."patch.
- I’ve got two sets of non-upstream libguestfs patches 1, 2
- I am not using libvirt, but if you do want to use libvirt, make sure you use the very latest version since it contains an important performance patch.
$ time LIBGUESTFS_BACKEND=direct LIBGUESTFS_HV=~/d/qemu/x86_64-softmmu/qemu-system-x86_64 guestfish -a /dev/null run real 0m0.966s user 0m0.623s sys 0m0.281s
However I had to patch qemu to enable DMA loading of the kernel and initrd.