Tag Archives: virt-resize

Writing a Planner to solve a tricky programming optimization problem

Suppose a monkey is in one corner of a room, a box is in another corner of a room, and a banana is hanging from the ceiling in the middle of the room. The monkey can’t reach the banana without standing on the box, but he first has to move the box under the banana. The problem of how to get a computer to work out that the monkey has to move the box first, then climb on the box second, was solved by Nils Nilsson’s STRIPS system in 1971. STRIPS is now an A.I. standard, and is used in game A.I. and elsewhere.

Suppose you have a disk image template that you want to uncompress, convert to another format, and resize. You can run xzcat, followed by qemu-img convert followed by virt-resize. But virt-resize can also do format conversion, so you don’t need to run qemu-img convert. Unless the user was happy with the original size, in which case qemu-img convert is faster than virt-resize. But what if the original template isn’t compressed and is already in the right format and size? You can just run cp.

How can a computer work out the right sequence of steps to convert the disk image most efficiently? Virt-builder has exactly this problem, and it solves it using a STRIPS-inspired planner.

The STRIPS planner in virt-builder is only 50 lines of code, was easy to write, finds the near optimal plan for almost any user input, and is a useful technique that can be applied to many programming problems. This article will explain how it works. I have changed some of the academic terms and simplified things to make this easier to understand.

First of all I’ll introduce tags on the original template. These define the state of that template:

Input tags: ✚xz ✚template ✚size=4G ✚format=raw

Secondly I’ll set up my goal state:

Goal tags: ❌xz ❌template ✚size=4G ✚format=qcow2

where means the tag MUST NOT exist in the final state.

I want my planner to find me the best path from my input state to my goal. As it can’t go straight from the input to the goal in one step, I have to tell the planner what transitions are possible, using a function:

transitions (input_tags) {
  if ✚xz then {
    you could run 'xzcat'
        which will ❌xz and ❌template;
  else /* no xz tag */ {
    you could run 'virt-resize'
       which will change ✚format and ✚size, and ❌template;
    you could run 'qemu-img convert'
       which will change ✚format, and ❌template;

  you could run 'cp'
      which will ❌template;

Notice that the transitions function returns a list of all possible transitions from the input state. It’s not judgemental about which one should be taken, although it won’t return impossible transitions (for example, running virt-resize is not possible on xz-compressed files). The actual transitions function also returns a weight for each transition, so that the planner can choose the least expensive plan if there are several plans possible.

The ✚template tag may appear a bit mysterious. It’s there to make sure that the planner always copies the original template, even if the original template already has the desired goal size and format. Since xzcat, virt-resize and qemu-img convert always copy the disk image, they drop the template tag (❌template).

The transitions function in virt-builder can be found here.

The planner does a breadth-first search over the tree of transitions, starting with the input state, finishing when it finds any branch that satisfies the output goals, or when it reaches a maximum depth in which case it gives up (and the user sees an error message).

The planner in virt-builder (50 lines of code) can be found here.

If the planner finds several paths that satisfy the goals, the planner chooses the one with the smallest weight. However my planner is not clever enough to look deeper in the tree to see if a longer path might have a smaller weight (it’s not very likely in virt-builder).

Also my planner is not smart enough to prune bogus paths. For example, if a path runs cp in adjacent steps, then that path should be pruned.

Nevertheless the planner always gets the right result, and it is considerably simpler than the original hand-written code. The old code had become unmaintainable and wasn’t even efficient: it sometimes made unnecessary copies in order to make the code simpler, wasting end-user time. Because of the ease of maintenance I was able to add new functionality: virt-builder can now run qemu-img resize to expand a disk by < 256 MB, a case where virt-resize doesn’t work (previously the user would have got an error).

Applying old academic techniques like this one doesn’t need to be hard and can help with real world problems. I hope this technique helps others with similar optimization problems.

Edit: The Hacker News discussion includes links to alternative solving tools.


Filed under Uncategorized

New in virt-builder 1.25.2

Note these virt-builder features are not available in the 1.24 branch, they are new features in development.

In no particular order:

  • --write can be used to write a literal file:
    virt-builder ... --write '/etc/motd:Obey me, users!'
  • --upload can now upload a file into an existing directory, instead of having to specify the target filename.
  • --mkdir can be used to create directories. It uses the equivalent of mkdir -p so it can create multi-level directories.
  • --no-sync can be used to avoid the sync on exit. There’s a long explanation in the virt-builder(1) man page (which I won’t reproduce here) about why this might or might not be a good idea for you.
  • pxzcat can be used, if available. I’m not sure yet whether we’ll package this, wait for parallel support in xz, or go for some custom C code in virt-builder.
  • virt-resize is avoided when possible. This can speed up builds.
  • Firstboot output is now also sent to the console, making it easy to monitor what’s going on with a virtual serial port.
  • You can use the -m or --smp options to specify the amount of memory or number of VCPUs to give to the appliance when it runs the --run or --install commands. If (especially) yum ran of out memory in a big install, then -m 2048 is your friend.

Leave a comment

Filed under Uncategorized

Virt-builder making VMs in around 3̶0̶ 16 seconds

The newest upstream virt-builder can now build simple VMs in around 30 seconds (this is on a machine with a hard drive, it’s faster if you have SSDs):

$ virt-builder ubuntu-12.04
[   0.0] Downloading: file:///home/rjones/d/libguestfs/builder/website/ubuntu-12.04.xz
[   1.0] Creating disk image: ubuntu-12.04.img
[   1.0] Uncompressing: file:///home/rjones/d/libguestfs/builder/website/ubuntu-12.04.xz
[   9.0] Opening the new disk
[  33.0] Setting a random seed
[  33.0] Random root password: z6T5zQiqIIy0yZfA [did you mean to use --root-password?]
[  33.0] Finishing off
Output: ubuntu-12.04.img
Total usable space: 2.9G
Free space: 2.1G (73%)

Even installing packages in the VM, it’s still taking under 1 minute:

$ virt-builder ubuntu-12.04 --install nmap
[   0.0] Downloading: file:///home/rjones/d/libguestfs/builder/website/ubuntu-12.04.xz
[   1.0] Creating disk image: ubuntu-12.04.img
[   1.0] Uncompressing: file:///home/rjones/d/libguestfs/builder/website/ubuntu-12.04.xz
[  10.0] Opening the new disk
[  34.0] Setting a random seed
[  34.0] Random root password: Mt3kxbEsBqumG6bR [did you mean to use --root-password?]
[  34.0] Installing packages: nmap
[  53.0] Finishing off
Output: ubuntu-12.04.img
Total usable space: 2.9G
Free space: 2.0G (70%)

The major changes are: It will use pxzcat (parallel xzcat) if available, and it will bypass the virt-resize step if you don’t specify the --size option (or if you make sure the templates are the right size to start with).


On a machine with an Intel SSD and parallel xzcat installed:

$ virt-builder ubuntu-12.04
[   0.0] Downloading: file:///home/rjones/d/libguestfs/builder/website/ubuntu-12.04.xz
[   1.0] Creating disk image: ubuntu-12.04.img
[   1.0] Uncompressing: file:///home/rjones/d/libguestfs/builder/website/ubuntu-12.04.xz
[  12.0] Opening the new disk
[  16.0] Setting a random seed
[  16.0] Random root password: JdIsqD5QB64yImYL [did you mean to use --root-password?]
[  16.0] Finishing off
Output: ubuntu-12.04.img
Total usable space: 2.9G
Free space: 2.1G (73%)

Leave a comment

Filed under Uncategorized

virt-resize from an NBD source

New in libguestfs ≥ 1.23.23 is the ability to use virt-resize from a remote source.

One use for this is to use nbdkit’s xz plugin as a source, to easily create new images from highly compressed templates. In the example below nbdkit transparently (and in RAM) uncompresses the disk image, serving it up over /tmp/sock (a socket):

$ nbdkit -r -f -U /tmp/sock \
    /usr/lib64/nbdkit/plugins/nbdkit-xz-plugin.so \

Virt-resize fetches the image from the socket and writes & resizes it to the destination file:

$ truncate -s 20G /tmp/output.img   # adjust target size
$ virt-resize 'nbd://?socket=/tmp/sock' /tmp/output.img
$ killall nbdkit; rm /tmp/sock

You can then use virt-sysprep to set the hostname and run configuration scripts in the output image.

Unfortunately you also have to temporarily disable SELinux for this to work, because SELinux prevents qemu from connecting to the NBD socket. Grrrr.

Leave a comment

Filed under Uncategorized

Cool new bash-completions of libguestfs tools

Starting in libguestfs ≥ 1.21.23-2, bash tab completions of guestfish, guestmount and virt-* tools have been rewritten and greatly improved.

Note you will need to install the libguestfs-bash-completion package to enable this feature.

You can now tab complete all long options on most tools:

$ virt-df --[tab]
--add             --domain          --human-readable  --uuid
--connect         --format          --inodes          --verbose
--csv             --help            --one-per-guest   --version
$ virt-resize --[tab]
--align-first          --help                 --no-extra-partition
--alignment            --ignore               --ntfsresize-force
--debug                --lvexpand             --output-format
--debug-gc             --lv-expand            --quiet
--delete               --LVexpand             --resize
--dryrun               --LV-expand            --resize-force
--dry-run              --machine-readable     --shrink
--expand               --no-copy-boot-loader  --version
--format               --no-expand-content    

Where appropriate, the -d option will now expand to the list of libvirt domains:

# virt-df -d [tab]
archlinux20121201x64  f19rawhidex32
f18x64                f19rawhidex64

Finally, guestfish commands are expanded on the command line:

$ guestfish add /tmp/disk : run : list-[tab]
list-9p              list-events          list-md-devices
list-devices         list-filesystems     list-partitions
list-disk-labels     list-ldm-partitions  
list-dm-devices      list-ldm-volumes     

To make this less intrusive, so you can really use it daily, I left the default readline expansions enabled. This means that filenames and so on can continue to be used in every position on the command line, and should mean that bash completions won’t try to be cleverer than the user.

Libguestfs bash completions are also demand-loaded now, so that if you’re not using them, they don’t consume any resources in the shell.

Leave a comment

Filed under Uncategorized

Build a guest library

I have this exported over NFS to all my local machines:

$ ls -lh /mnt/media/guest-library/
total 12G
-rw-r--r--. 1 rjones rjones 971M Apr 16 23:38 debian5x64.img.xz
-rw-r--r--. 1 rjones rjones 250M Jul 26 23:09 f16x32.img.xz
-rw-r--r--. 1 rjones rjones 809M Jul 26 20:29 f17x64.img.xz
-rw-r--r--. 1 rjones rjones  59M Jul  6 22:19 freedos11.img.xz
-rw-r--r--. 1 rjones rjones 379M Jul  7 20:11 opensuse1113x64.img.xz
-rw-r--r--. 1 rjones rjones 349M Jul  6 22:31 plan9.img.xz
-rw-rw-r--. 1 rjones rjones  126 Apr 16 17:22 README
-rw-r--r--. 1 rjones rjones 384M Apr 18 00:19 rhel3x64.img.xz
-rw-r--r--. 1 rjones rjones 471M Apr 18 00:15 rhel4x64.img.xz
-rw-r--r--. 1 rjones rjones 848M Apr 16 17:02 rhel5epelx64.img.xz
-rw-r--r--. 1 rjones rjones 768M Jul  7 07:21 sl60x64.img.xz
-rw-r--r--. 1 rjones rjones 148M May 23 16:08 ubuntu1204ppc.img.xz
-rw-r--r--. 1 rjones rjones 1.1G Apr 16 17:09 win2003.img.xz
-rw-r--r--. 1 rjones rjones 3.4G Jun  1 10:46 win8-preview.img.xz
-rw-r--r--. 1 rjones rjones 2.1G Apr 16 20:28 winxp.img.xz

Having a library of images which are (relatively) small like this is great for testing. If someone gives me a bug and it happens to affect one of the guests I have saved, then I can just unpack that guest, virt-resize it to the right size and off I go, and that’s usually a lot quicker than installing a new guest, particularly when you’ve got only slow ADSL.

Adding a new guest is easy using virt-sparsify and xz. Note I’ve set the temporary directory (TMPDIR) to somewhere with plenty of space.

$ sudo TMPDIR=$(pwd) virt-sparsify /dev/vg_pin/F18Rawhidex32 f18rawhidex32.img
Create overlay file to protect source disk ...
Examine source disk ...
 100% ⟦▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓⟧ 00:00
Fill free space in /dev/vda1 with zero ...
 100% ⟦▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓⟧ --:--
Fill free space in /dev/vg_f18rawhidex32/lv_root with zero ...
 100% ⟦▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓⟧ --:--
Clearing Linux swap on /dev/vg_f18rawhidex32/lv_swap ...
 100% ⟦▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓⟧ 00:00
Copy to destination and make sparse ...

Sparsify operation completed with no errors.  Before deleting the old 
disk, carefully check that the target disk boots and works correctly.

$ ll -h f18rawhidex32.img
-rw-r--r--. 1 root root 10G Jul 27 14:42 f18rawhidex32.img
$ du -sh f18rawhidex32.img
3.0G	f18rawhidex32.img
$ xz -T 0 --best f18rawhidex32.img
$ ll -h f18rawhidex32.img.xz
-rw-r--r--. 1 rjones rjones 667M Jul 27 14:42 f18rawhidex32.img.xz


Filed under Uncategorized

New tool: virt-sysprep

New in libguestfs 1.13.19 is a tool called virt-sysprep which makes it easier to clone virtual machines.

The new tool isn’t a complete clone tool, but you can easily combine it with other tools like dd, the old virt-clone, virt-resize and virt-sparsify to clone virtual machines.


Filed under Uncategorized

New tool for making VMs sparse (aka thin provisioning)

New in libguestfs 1.13.17 is a tool for making virtual machines or disk images sparse, also known as thin provisioning.

This tool restores sparseness lost after you’ve created and deleted files. It works for virtually any guest OS, and supports almost any filesystem.

It can also do some clever things like finding sparseness in the unused parts of LVM physical volumes, or converting between formats (eg. raw to thin-provisioned qcow2).

It’s also (like virt-resize) a temporary tool: hopefully one day KVM and guests will be able to resize and trim themselves. That day isn’t here yet.

virt-sparsify manual page


Filed under Uncategorized

Arrrgghh writing GUI programs is tedious

Last week I thought it would be a good idea to write a small GUI front end to virt-resize.

After two days, I nearly have the first tab (out of four) working.

Granted, maybe the first tab is the hardest one:

The job of the first tab is to ask the user for the source (disk image or libvirt guest). It then fires off a background job to open that guest and inspect it for operating systems. Based on the outcome of that (opened OK, found OSes, no OSes found, etc) it has to update and enable the other tabs.

Also the user can preselect a guest on the command line. We also have to deal with connecting to libvirt asynchronously to read the list of guests (and this might also fail in a number of ways).

So far, 1600 lines of code, and the first tab is by no means complete.

One part of the problem is there’s a certain “impedance mismatch” between functional programming in OCaml and writing Gtk. Gtk is heavily mutable and object based. OCaml prefers (but does not require) immutability, and objects in OCaml are obscure and not widely used, and the Gtk bindings are written in a very hard-core object OCaml style.

Another part is just that it’s tedious. It would be tedious if I was doing this in C or Python too. You’ve got asynchronous actions going off here and there which update bits of state. Every control or input could potentially affect every other control or output, resulting in a kind of O(n2) mess of wiring up signals and callbacks.

Is there an easier way? I don’t know …


Filed under Uncategorized

virt-resize machine readable output

As I planned earlier I have added machine readable output to virt-resize.

One huge frustration we have had with qemu for a long time is that you cannot query a qemu binary for its “capabilities” (eg. does this binary support virtio? does this binary support particular -drive options?).

virt-resize now supports this kind of querying, thus with any virt-resize ≥ 1.12.5 you can do:

$ virt-resize --machine-readable

The simple list of facts is designed to be easy to parse. Does this virt-resize work on 32-bit platforms (old versions written in Perl didn’t)? Answer: yes because of the 32bitok fact. Does this binary support btrfs guests? In this case yes.

Over time as we add more features to virt-resize we will add facts to this list, so that consumers of virt-resize can tell what the binary supports.

This is a simple plan. Qemu developers, take note!

The second thing that this option does is to make the output of virt-resize more regular and easier to consume. At the moment this means that progress bars in virt-resize will revert to a simple format:

$ virt-resize --machine-readable old.img new.img

so that programs can just match on ^\d+/\d+$ on stdout, and turn that into a progress bar. User messages go to stdout and errors to stderr. The output is designed to be both very lightweight to parse and easy to understand for humans (and in fact not that much different from the regular virt-resize output). We may make small enhancements in future.

Leave a comment

Filed under Uncategorized