virt-resize –shrink now works

(Shrinking is still tedious, but at least it now works. Expanding is much simpler).

Look at the original disk image and decide what scope there is for shrinking it. In this case we could reduce the size by as much as 5.5GB, but for the purpose of this test I will only shrink the image from 10GB to 7GB.

$ virt-df /tmp/disk.img
Filesystem                           1K-blocks       Used  Available  Use%
disk.img:/dev/sda1                      495844      29565     440679    6%
disk.img:/dev/vg_f13x64/lv_root        7804368    2077504    5647596   27%

We can expand guests automatically, but virt-resize is much more conservative about the complex business of shrinking guests. You have to first use guestfish on (a copy of) the original disk to manually shrink the content of the partitions you want to shrink. In this case I manually shrink the root filesystem, its LV, and the PV.

$ guestfish -a /tmp/disk.img

Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.

Type: 'help' for a list of commands
      'man' to read the manual
      'quit' to quit the shell

><fs> run
><fs> resize2fs-size /dev/vg_f13x64/lv_root 4G
libguestfs: error: resize2fs_size: resize2fs 1.41.12 (17-May-2010)
Please run 'e2fsck -f /dev/vg_f13x64/lv_root' first.

This known problem with resize2fs is documented.

><fs> e2fsck-f /dev/vg_f13x64/lv_root
><fs> resize2fs-size /dev/vg_f13x64/lv_root 4G
><fs> lvresize /dev/vg_f13x64/lv_root 4096
><fs> pvresize-size /dev/sda2 6G
libguestfs: error: pvresize_size: /dev/vda2:
/dev/vda2: cannot resize to 191 extents as later ones are allocated.

This is unexpected: pvresize does not “defrag”. Perhaps in a future version of pvresize, pvmove or libguestfs we will add this, but for now I have to workaround it by deleting the troublesome swap partition, doing the resize, then recreating the swap.

><fs> lvremove /dev/vg_f13x64/lv_swap
><fs> pvresize-size /dev/sda2 6G
><fs> lvcreate lv_swap vg_f13x64 512
><fs> mkswap /dev/vg_f13x64/lv_swap
><fs> exit

Now I can perform the resize operation itself using virt-resize. Note this is also a copy operation, since virt-resize never changes the source disk.

$ truncate -s 7G /tmp/disk2.img
$ virt-resize --shrink /dev/sda2 /tmp/disk.img /tmp/disk2.img
Summary of changes:
/dev/sda1: partition will be left alone
/dev/sda2: partition will be resized from 9.5G to 6.5G
Copying /dev/sda1 ...
[########################################################]
Copying /dev/sda2 ...
[########################################################]

After a test boot, the final guest worked(!)

New tool: virt-resize

Virt-resize lets you resize existing virtual machines (not live however).

$ rm -f /tmp/centos.img
$ truncate -s 12G /tmp/centos.img
$ virt-resize --resize sda1=200% --resize sda2=11.2G \
    /dev/vg_trick/CentOS5x32 /tmp/centos.img
Summary of changes:
/dev/sda1: partition will be resized from 101.9M to 203.9M
/dev/sda2: partition will be resized from 9.9G to 11.2G
There is a surplus of 644971316 bytes (615.1M).
An extra partition will be created for the surplus.
Copying /dev/sda1 ... done
Copying /dev/sda2 ... done

$ rm -f /tmp/centos.img
$ truncate -s 12G /tmp/centos.img
$ virt-resize --resize sda1=200% --expand sda2 \
    /dev/vg_trick/CentOS5x32 /tmp/centos.img 
Summary of changes:
/dev/sda1: partition will be resized from 101.9M to 203.9M
/dev/sda2: partition will be resized from 9.9G to 11.8G
Copying /dev/sda1 ... done
Copying /dev/sda2 ... done

After some discussion on the list we decided to start with a simple / primitive tool and work upwards. So virt-resize as implemented now will not resize filesystems and PVs. You have to do that as a separate step after running the tool (either running pvresize/resize2fs in the guest or using guestfish for offline changes.

The new tool’s man page after the cut.

Is ext2/3/4 faster? On LVM?

This question arose at work — is LVM a performance penalty compared to using straight partitions? To save you the trouble, the answer is “not really”. There is a very small penalty, but as with all benchmarks it does depend on what the benchmark measures versus what your real workload does. In any case, here is a small guestfish script you can use to compare the performance of various filesystems with or without LVM, with various operations. Whether you trust the results is up to you, but I would advise caution.

#!/bin/bash -

tmpfile=/tmp/test.img

for fs in ext2 ext3 ext4; do
    for lvm in off on; do
        rm -f $tmpfile
        if [ $lvm = "on" ]; then
            guestfish <<EOF
              sparse $tmpfile 1G
              run
              part-disk /dev/sda efi
              pvcreate /dev/sda1
              vgcreate VG /dev/sda1
              lvcreate LV VG 800
              mkfs $fs /dev/VG/LV
EOF
            dev=/dev/VG/LV
        else # no LVM
            guestfish <<EOF
              sparse $tmpfile 1G
              run
              part-disk /dev/sda efi
              mkfs $fs /dev/sda1
EOF
            dev=/dev/sda1
        fi
        echo "fs=$fs lvm=$lvm"
        sync
        guestfish -a $tmpfile -m $dev <<EOF
          time fallocate /file1 200000000
          time cp /file1 /file2
EOF
    done
done

fs=ext2 lvm=off
elapsed time: 2.74 seconds
elapsed time: 4.52 seconds
fs=ext2 lvm=on
elapsed time: 2.60 seconds
elapsed time: 4.24 seconds
fs=ext3 lvm=off
elapsed time: 2.62 seconds
elapsed time: 4.31 seconds
fs=ext3 lvm=on
elapsed time: 3.07 seconds
elapsed time: 4.79 seconds

# notice how ext4 is much faster at fallocate, because it
# uses extents

fs=ext4 lvm=off
elapsed time: 0.05 seconds
elapsed time: 3.54 seconds
fs=ext4 lvm=on
elapsed time: 0.05 seconds
elapsed time: 4.16 seconds

What’s new for libguestfs

The libguestfs team have been busy adding features over the past few weeks. Here is a quick round-up.

• Writing to Windows Registry files. We haven’t quite got this feature working yet, but we intend that you’ll be able to “shell into” a Windows Registry and change things:

   $ hivexsh SOFTWARE
    
    Welcome to hivexsh, the hivex interactive shell for examining
    Windows Registry binary hive files.
    
    Type: 'help' for help summary
          'quit' to quit the shell
    
    SOFTWARE\> ls
    ATI Technologies
    Classes
    Clients
    Intel
    Microsoft
    ODBC
    Policies
    RegisteredApplications
    Sonic
    Wow6432Node
    SOFTWARE\> cd \Microsoft\Windows\TabletPC\TabSetup
    SOFTWARE\Microsoft\Windows\TabletPC\TabSetup> ls
    SOFTWARE\Microsoft\Windows\TabletPC\TabSetup> lsval
    "TabletSetup"=dword:00000000"
    SOFTWARE\Microsoft\Windows\TabletPC\TabSetup> cd ..
    SOFTWARE\Microsoft\Windows\TabletPC> ls
    Snipping Tool
    TabSetup
  

• Rename LVs and VGs. (Guymon’s incredibly convoluted description of how to do this when you don’t have libguestfs inspired me to add lvrename and vgrename commands).
• C# bindings. This doesn’t mean you can use libguestfs on Windows, but it does mean you can use libguestfs from Mono/Linux.
• Indian language support. We received Punjabi, Telegu and Oriya translations all at once from the Fedora L10N project so now you can display guestfish in elegant cursive script. “Run a command from the guest filesystem” is “ਗਿਸਟ ਫਾਇਲ-ਸਿਸਟਮ ਤੋਂ ਕਮਾਂਡ ਚਲਾਓ” in Punjabi – how beautiful. The L10N guys work hard doing a pretty thankless job and don’t seem to get much credit for what they do, so thanks guys (and gals).

libguestfs 0.8, now with fewer bugs :-)

I released libguestfs 0.8 yesterday, the library for accessing virtual machines. This version fixes a large number of bugs, because I wrote a test suite which sends several hundred commands through the library and verifies the results. Took a couple of days to fix all the bugs this found …

Here are some interesting things you can do with guestfish.

1: Just find out what’s in a virtual machine:

$ guestfish -a RHEL52PV32.img run : list-devices
/dev/sda

$ guestfish -a RHEL52PV32.img run : list-partitions
/dev/sda1
/dev/sda2

$ guestfish -a RHEL52PV32.img run : lvs
/dev/VolGroup00/LogVol00
/dev/VolGroup00/LogVol01

2: Make virtual machine filesystems containing partitions, LVM, filesystems and whatever else you want:

$ guestfish
Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.

Type: 'help' for help with commands
      'quit' to quit the shell

><fs> alloc /tmp/test 500M
><fs> run
><fs> pvcreate /dev/sda
><fs> vgcreate VG /dev/sda
><fs> vgs
VG
><fs> lvcreate LV VG 250
><fs> lvs
/dev/VG/LV
><fs> mkfs ext3 /dev/VG/LV
><fs> mount /dev/VG/LV /
><fs> touch /myfile
><fs> ll /
total 13
drwxr-xr-x  3 root root  1024 Apr 12 17:19 .
drwxr-xr-x 18 root root     0 Apr 12 17:15 ..
drwx------  2 root root 12288 Apr 12 17:19 lost+found
-rw-r--r--  1 root root     0 Apr 12 17:19 myfile
><fs> sync
><fs> quit
$ ll /tmp/test
-rw-rw-r--. 1 rjones rjones 524288000 2009-04-12 22:19 /tmp/test
$ file /tmp/test
/tmp/test: LVM2 (Linux Logical Volume Manager) , UUID: bhL0DP0rySspzviDKHN7TZABBcfTWuo

3: Use Augeas to edit configuration files:

$ guestfish -a RHEL52PV32.img -m /dev/VolGroup00/LogVol00
  
  Welcome to guestfish, the libguestfs filesystem interactive shell for
  editing virtual machine filesystems.
  
  Type: 'help' for help with commands
        'quit' to quit the shell
  
><fs> aug-init / 0
><fs> aug-match /augeas/*
/augeas/root
/augeas/save
/augeas/files
><fs> aug-match /files//error
><fs> aug-match /augeas//error
><fs> aug-match /augeas/root/*
><fs> aug-match /files/etc/*
/files/etc/ldap.conf
/files/etc/aliases
/files/etc/yum.repos.d
/files/etc/yum.conf
/files/etc/sysconfig
[etc]
><fs> aug-match /files/etc/hosts/*
/files/etc/hosts/comment[1]
/files/etc/hosts/comment[2]
/files/etc/hosts/1
/files/etc/hosts/2
><fs> cat /etc/hosts               
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1      	       localhost.localdomain localhost
::1		               localhost6.localdomain6 localhost6
><fs> aug-get /files/etc/hosts/comment[1]
Do not remove the following line, or various programs
><fs> help aug-insert
aug-insert - insert a sibling Augeas node
       aug-insert   
  
      Create a new sibling "label" for "path", inserting it into the tree
      before or after "path" (depending on the boolean flag "before").
  
      "path" must match exactly one existing node in the tree, and "label"
      must be a label, ie. not contain "/", "*" or end with a bracketed index
      "[N]".
  
><fs> aug-insert /files/etc/hosts/comment[2] comment false
><fs> aug-match /files/etc/hosts/*
/files/etc/hosts/comment[1]
/files/etc/hosts/comment[2]
/files/etc/hosts/comment[3]
/files/etc/hosts/1
/files/etc/hosts/2
><fs> aug-set /files/etc/hosts/comment[3] HELLO
><fs> aug-save
><fs> cat /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
# HELLO
127.0.0.1		localhost.localdomain localhost
::1			localhost6.localdomain6 localhost6

There are C, C++, shell, Perl and OCaml bindings for all of these calls, and you don’t need to be root to do any of it.

libguestfs perl bindings

You can now use Perl to examine and modify virtual machine disk images through libguestfs, as in this example:

#!/usr/bin/perl -w
use strict;
use Sys::Guestfs;

# Look for LVM LVs, VGs and PVs in a guest image.

die "Usage: lvs.pl guest.img\n" if @ARGV != 1 || ! -f $ARGV[0];

my $h = Sys::Guestfs->new ();
$h->add_drive ($ARGV[0]);

print "Launching, this can take a few seconds\n";
$h->launch ();
$h->wait_ready ();

print "Looking for PVs on the disk image\n";
my @pvs = $h->pvs ();
print "PVs found: (", join (", ", @pvs), ")\n";

print "Looking for VGs on the disk image\n";
my @vgs = $h->vgs ();
print "VGs found: (", join (", ", @vgs), ")\n";

print "Looking for LVs on the disk image\n";
my @lvs = $h->lvs ();
print "LVs found: (", join (", ", @lvs), ")\n";

Which produces this output:

$ ./lvs.pl RHEL52PV32.img
Creating the libguestfs handle
Launching, this can take a few seconds
Looking for PVs on the disk image
PVs found: (/dev/sda2)
Looking for VGs on the disk image
VGs found: (VolGroup00)
Looking for LVs on the disk image
LVs found: (/dev/VolGroup00/LogVol00, /dev/VolGroup00/LogVol01)

libguestfs: LVM support

libguestfs is the library I’m writing that lets you examine and modify any virtual machine disk image, offline or online.

I spent about 18 hours finding a bug in the LVM parsing code, but that bug has now been squashed and as of version 0.4 on the website you can examine PVs, VGs and LVs, as well as listing devices and partitions.

Next up: language bindings for Perl, OCaml and Python.
(C and C++ and shell scripting access already exist).

$ guestfish 

Welcome to guestfish, the libguestfs
filesystem interactive shell for
editing virtual machine filesystems.

Type: 'help' for help with commands
      'quit' to quit the shell

><fs> add /mnt/share/tmp/RHEL52PV32.img
><fs> run
><fs> pvs
/dev/sda2
><fs> lvs
/dev/VolGroup00/LogVol00
/dev/VolGroup00/LogVol01
><fs> mount /dev/VolGroup00/LogVol00 /
><fs> ls /
bin
boot
dev
etc
home
[...]