Tag Archives: guestfish

xz plugin for nbdkit

I’ve now written an xz plugin for nbdkit (previous discussion on this blog).

This is useful if you’re building up a library of xz-compressed disk images using virt-sparsify and xz, and you want to access them without having to uncompress them.

I certainly learned a lot about the xz file format and liblzma this weekend …

The xz file format consists of multiple streams (but usually one). Each stream contains zero or more blocks of compressed data, followed by an “index”. Like zip, everything in an xz file happens from the end, so the block index is at the end of the stream (this allows xz files to be streamed when writing without needing any reverse seeks).

Crucially the index contains the offset of each block both in the actual xz file and in the uncompressed data, so once you’ve read the index from a file you can find the position of any uncompressed byte and seek to the beginning of that block and read the data. Random access!

Preparing xz files correctly is important in order to be able to get good random access performance with low memory overhead:

$ xz --list /tmp/winxp.img.xz
Strms  Blocks   Compressed Uncompressed  Ratio  Check   Filename
    1     384  2,120.1 MiB  6,144.0 MiB  0.345  CRC64   /tmp/winxp.img.xz

A file with lots of small blocks like the above (16 MB block size) is relatively easy to seek inside. At most 16 MB of data has to be uncompressed to reach any byte.

Perhaps ironically, if your machine has lots of free memory then xz appears to choose a large block size, resulting in some one-block files. Here’s the same file when I originally compressed it for my guest library:

$ xz --list guest-library/winxp.img.xz
Strms  Blocks   Compressed Uncompressed  Ratio  Check   Filename
    1       1  2,100.0 MiB  6,144.0 MiB  0.342  CRC64   guest-library/winxp.img.xz

So unfortunately you may need to recompress some of your xz files using the new xz --block-size option:

$ xz --best --block-size=$((16*1024*1024)) winxp.img

Here’s how you use the new nbdkit xz plugin:

$ nbdkit plugins/nbdkit-xz-plugin.so file=winxp.img.xz
$ guestfish --ro -a nbd://localhost -i

Welcome to guestfish, the guest filesystem shell for
editing virtual machine filesystems and disk images.

Type: 'help' for help on commands
      'man' to read the manual
      'quit' to quit the shell

Operating system: Microsoft Windows XP
/dev/sda1 mounted on /

><fs> ll /
total 1573209
drwxrwxrwx  1 root root       4096 Apr 16  2012 .
drwxr-xr-x 23 1000 1000       4096 Jun 24 13:57 ..
-rwxrwxrwx  1 root root          0 Oct 11  2011 AUTOEXEC.BAT
-rwxrwxrwx  1 root root          0 Oct 11  2011 CONFIG.SYS
drwxrwxrwx  1 root root       4096 Oct 11  2011 Documents and Settings
-rwxrwxrwx  1 root root          0 Oct 11  2011 IO.SYS
-rwxrwxrwx  1 root root          0 Oct 11  2011 MSDOS.SYS
[...]

4 Comments

Filed under Uncategorized

qemu 1.5.0 released, with ssh block device support

qemu 1.5.0 has been released, featuring ssh support so you can access remote disks over ssh, including from libguestfs.

Here’s how to use this from guestfish:

$ export LIBGUESTFS_BACKEND=direct
$ guestfish --ro -a ssh://onuma/mnt/scratch/winxp.img -i

Welcome to guestfish, the guest filesystem shell for
editing virtual machine filesystems and disk images.

Type: 'help' for help on commands
      'man' to read the manual
      'quit' to quit the shell

Operating system: Microsoft Windows XP
/dev/sda1 mounted on /

><fs> ll /
total 1573209
drwxrwxrwx  1 root root       4096 Apr 16  2012 .
drwxr-xr-x 23 1000 1000       4096 May 20 19:47 ..
-rwxrwxrwx  1 root root          0 Oct 11  2011 AUTOEXEC.BAT
-rwxrwxrwx  1 root root          0 Oct 11  2011 CONFIG.SYS
drwxrwxrwx  1 root root       4096 Oct 11  2011 Documents and Settings
-rwxrwxrwx  1 root root          0 Oct 11  2011 IO.SYS
-rwxrwxrwx  1 root root          0 Oct 11  2011 MSDOS.SYS
-rwxrwxrwx  1 root root      47564 Apr 14  2008 NTDETECT.COM
drwxrwxrwx  1 root root       4096 Oct 11  2011 Program Files
drwxrwxrwx  1 root root       4096 Oct 11  2011 System Volume Information
drwxrwxrwx  1 root root      28672 Oct 11  2011 WINDOWS
-rwxrwxrwx  1 root root        211 Oct 11  2011 boot.ini
-rwxrwxrwx  1 root root     250048 Apr 14  2008 ntldr
-rwxrwxrwx  1 root root 1610612736 Oct 11  2011 pagefile.sys

Leave a comment

Filed under Uncategorized

Testing exabyte-sized filesystems using qcow2 and guestfish

You can use qcow2 backing files as a convenient way to test what happens when you try to create exabyte-sized filesystems. Just to remind you, 1 exabyte is a million terabytes, or a pile of ordinary hard disks stacked 8 miles high.

There is a bug in qemu that prevents you from creating very large disks unless you adjust the cluster_size option (thanks Kevin Wolf):

$ qemu-img create -f qcow2 huge.qcow2 \
      $((1024*1024))T -o cluster_size=2M
Formatting 'huge.qcow2', fmt=qcow2 size=1152921504606846976 encryption=off cluster_size=2097152 lazy_refcounts=off 

After that you can just attach the disk to guestfish and start playing with huge filesystems.

[I should note that virt-rescue is probably a better choice of tool here, especially for people who need to experiment with unusual filesystem or LVM options]

$ guestfish -a huge.qcow2

Welcome to guestfish, the guest filesystem shell for
editing virtual machine filesystems and disk images.

Type: 'help' for help on commands
      'man' to read the manual
      'quit' to quit the shell

><fs> run
><fs> blockdev-getsize64 /dev/sda
1152921504606846976
><fs> part-disk /dev/sda gpt

Ext4 (according to Wikipedia) is supposed to support 1 exabyte disks, but I couldn’t get that to work, possibly because there was not enough RAM:

><fs> mkfs ext4 /dev/sda1
libguestfs: error: mkfs: ext4: /dev/sda1: mke2fs 1.42.5 (29-Jul-2012)
/dev/sda1: Not enough space to build proposed filesystem while setting up superblock

XFS could create a filesystem, but I didn’t let it run to completion because it would need about 5 petabytes to store the filesystem metadata:

><fs> mkfs xfs /dev/sda1
[ disks churn for many minutes while qcow2 file grows
and grows and grows ... ]

LVM2 PVs are possible, but creating a VG requires us to adjust the extent size:

><fs> pvcreate /dev/sda1
><fs> vgcreate VG /dev/sda1
libguestfs: error: vgcreate:   PV /dev/sda1 too large for extent size 4.00 MiB.
  Format-specific setup of physical volume '/dev/sda1' failed.
  Unable to add physical volume '/dev/sda1' to volume group 'VG'.
><fs> debug sh "vgcreate -s 1G VG /dev/sda1"
  Volume group "VG" successfully created
><fs> lvcreate LV VG 1000000000
><fs> lvs-full
[0] = {
  lv_name: LV
[...]
  lv_size: 1048576536870912
}

Previously …

Leave a comment

Filed under Uncategorized

Using libguestfs to access iSCSI disks

In libguestfs ≥ 1.21.38 you can access at least some iSCSI disks.

On my server (RHEL 6 in this case) I create an iSCSI target backed by a Windows XP disk image:

# service tgtd start
Starting SCSI target daemon:                       [  OK  ]
# tgtadm --lld iscsi --op new --mode target --tid 1 \
      -T iqn.1994-05.com.redhat
# chcon system_u:object_r:tgtd_var_lib_t:s0 /tmp/winxp.img
# tgtadm --lld iscsi --op new --mode logicalunit --tid 1 \
      --lun 1 -b /tmp/winxp.img
# tgt-admin -s
...

Previously I opened port 3250 on the server. Because libguestfs doesn’t yet support authentication against the iSCSI server, I had to bypass that:

# tgtadm --lld iscsi --mode target --op bind --tid 1 -I ALL

Now on the client, I can connect to the iSCSI target using libguestfs like this:

$ export LIBGUESTFS_BACKEND=direct
$ guestfish --format=raw -a iscsi://x.x.x.x/iqn.1994-05.com.redhat/1 -i

Welcome to guestfish, the guest filesystem shell for
editing virtual machine filesystems and disk images.

Type: 'help' for help on commands
      'man' to read the manual
      'quit' to quit the shell

Operating system: Microsoft Windows XP
/dev/sda1 mounted on /

><fs> ll /
total 1573209
drwxrwxrwx  1 root root       4096 Apr 16  2012 .
drwxr-xr-x 23 1000 1000       4096 May 11 17:16 ..
-rwxrwxrwx  1 root root          0 Oct 11  2011 AUTOEXEC.BAT
-rwxrwxrwx  1 root root          0 Oct 11  2011 CONFIG.SYS
drwxrwxrwx  1 root root       4096 Oct 11  2011 Documents and Settings
-rwxrwxrwx  1 root root          0 Oct 11  2011 IO.SYS
-rwxrwxrwx  1 root root          0 Oct 11  2011 MSDOS.SYS
-rwxrwxrwx  1 root root      47564 Apr 14  2008 NTDETECT.COM
drwxrwxrwx  1 root root       4096 Oct 11  2011 Program Files
drwxrwxrwx  1 root root       4096 Oct 11  2011 System Volume Information
drwxrwxrwx  1 root root      28672 Oct 11  2011 WINDOWS
-rwxrwxrwx  1 root root        211 Oct 11  2011 boot.ini
-rwxrwxrwx  1 root root     250048 Apr 14  2008 ntldr
-rwxrwxrwx  1 root root 1610612736 Oct 11  2011 pagefile.sys

Leave a comment

Filed under Uncategorized

Using libguestfs to read Hyper-V VHDX disk images

(Thanks to Jeff Cody for both writing the code and helping me to get it to work).

New in qemu 1.5 is read-only support for Hyper-V‘s native disk format, vhdx. You can now open vhdx files in libguestfs just like any other:

$ guestfish --ro --format=vhdx -a /tmp/f18x64.vhdx -i

Welcome to guestfish, the guest filesystem shell for
editing virtual machine filesystems and disk images.

Type: 'help' for help on commands
      'man' to read the manual
      'quit' to quit the shell

Operating system: Fedora release 18 (Spherical Cow)
/dev/fedora/root mounted on /
/dev/sda1 mounted on /boot

Notes:

  1. As stated above, you will need the very latest qemu for this to work.
  2. You must specify --format=vhdx at the moment because of a missing feature in libvirt.
  3. You must specify --ro because writing is not supported (by the qemu driver).

Leave a comment

Filed under Uncategorized

Tip: Convert a Windows DVD ISO to a bootable USB key using guestfish

Situation: You have a Windows DVD (or ISO), but like any sane person in 2013 you don’t have a DVD drive on the computer. You want to convert the Windows DVD into a bootable USB key. There are many recipes for this online, but they all require another Windows machine and of course cannot be automated.

However with guestfish (and the always brilliant SYSLINUX doing most of the heavy lifting), this script will unpack the ISO and turn it into a bootable USB key.

Notes:

  1. I am not going to support this script. You will need to read the script, look up the commands in the guestfish man page, and understand what it does. Any requests for help will be deleted unread.
  2. You need to edit the USB key device before using the script.
  3. You need libguestfs ≥ 1.21 with SYSLINUX support compiled in.
#!/bin/bash -

guestfish <<'EOF'
trace on

add-ro en_microsoft_hyper-v_server_2012_x64_dvd_915600.iso

# NB: The next line MUST be changed to your USB drive.
# ANYTHING ON THIS DRIVE WILL BE OVERWRITTEN WITHOUT WARNING.
add /dev/sdX

run

# Inside the appliance, /dev/sda = DVD, /dev/sdb = USB.
# THESE ARE NOT RELATED TO HOST DISK NAMES.

echo "Partitioning the USB disk ..."
part-init /dev/sdb mbr
part-add /dev/sdb p 63 -1
part-set-mbr-id /dev/sdb 1 0xb
part-set-bootable /dev/sdb 1 true
mkfs vfat /dev/sdb1

echo "Copying the contents of the DVD to the USB key ..."
mkmountpoint /cd
mkmountpoint /usb
mount /dev/sda /cd
mount /dev/sdb1 /usb
# XXX We should add cp-r command XXX
debug sh "cp -rP /sysroot/cd/* /sysroot/usb"
#glob cp-a /cd/* /usb
umount /cd
umount /usb
rmmountpoint /cd
rmmountpoint /usb

echo "Making the USB key bootable using SYSLINUX ..."
syslinux /dev/sdb1
mount /dev/sdb1 /
upload /usr/share/syslinux/chain.c32 /chain.c32
write /syslinux.cfg "DEFAULT windows\n\nLABEL windows\nCOM32 chain.c32\nAPPEND fs ntldr=/bootmgr\n"
umount /dev/sdb1
upload /usr/share/syslinux/mbr.bin /dev/sdb

echo "Finished."

EOF

2 Comments

Filed under Uncategorized

Using rsync with libguestfs

Since libguestfs 1.20 it has been possible to use rsync to upload or download files to a disk image incrementally. This is one way to do backups, but note that it won’t work on live guests unless you take a snapshot.

rsync involves using a network connection into or out of the appliance, and is therefore a lot more involved to set up. The script below shows one way to do this, by running an rsync daemon on the host, and letting the libguestfs appliance connect to it.

The script runs rsync inside the appliance, copying /home from the attached disk image out to /tmp/backup on the host. If the operation is repeated, then only incrementally changed files will be copied out. (To incrementally delete files on the target, add the deletedest:true flag).

Note you will need to open port 2999 on your host’s firewall for this to work.

#!/bin/bash -

set -x

# The target directory.
mkdir -p /tmp/backup

# Create the daemon.
rm -f /tmp/rsyncd.pid

cat <<EOF > /tmp/rsyncd.conf
port = 2999
pid file = /tmp/rsyncd.pid

[backup]
  path = /tmp/backup
  use chroot = false
  read only = false
EOF

rsync --daemon --config=/tmp/rsyncd.conf

# Run guestfish and attach to the guest.
guestfish --ro --network -a /dev/fedora/f19rawhidex32 -i <<EOF
trace on
rsync-out /home rsync://rjones@192.168.122.1:2999/backup archive:true
EOF

# Kill the rsync daemon.
kill `cat /tmp/rsyncd.pid`

2 Comments

Filed under Uncategorized

libguestfs and qemu ssh driver

The qemu ssh block device is now up to version 7 … although sadly not upstream yet.

Nevertheless by applying this patch to libguestfs you can use libguestfs to access remote disks over ssh:

$ export LIBGUESTFS_QEMU=~/d/qemu/qemu.wrapper
$ export LIBGUESTFS_BACKEND=direct
$ ./run ./fish/guestfish

Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.

Type: 'help' for help on commands
      'man' to read the manual
      'quit' to quit the shell

><fs> add /tmp/f17x64.img readonly:true format:raw \
        protocol:ssh server:onuma
><fs> run
 100% ⟦▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒⟧ 00:00
><fs> inspect-os
/dev/vg_f17x64/lv_root
><fs> inspect-get-product-name /dev/vg_f17x64/lv_root
Fedora release 17 (Beefy Miracle)
><fs> list-filesystems
/dev/sda1: ext4
/dev/vg_f17x64/lv_root: ext4
/dev/vg_f17x64/lv_swap: swap
><fs> mount /dev/vg_f17x64/lv_root /
><fs> cat /etc/redhat-release
Fedora release 17 (Beefy Miracle)

Everything just works as if this were a local disk.

There are a couple of minor caveats (the major caveat being none of this is upstream): Firstly you have to have ssh-agent set up. Secondly the remote host must be in your known_hosts file (if not, do ssh remotehost first to add it).

Leave a comment

Filed under Uncategorized

Cool new bash-completions of libguestfs tools

Starting in libguestfs ≥ 1.21.23-2, bash tab completions of guestfish, guestmount and virt-* tools have been rewritten and greatly improved.

Note you will need to install the libguestfs-bash-completion package to enable this feature.

You can now tab complete all long options on most tools:

$ virt-df --[tab]
--add             --domain          --human-readable  --uuid
--connect         --format          --inodes          --verbose
--csv             --help            --one-per-guest   --version
$ virt-resize --[tab]
--align-first          --help                 --no-extra-partition
--alignment            --ignore               --ntfsresize-force
--debug                --lvexpand             --output-format
--debug-gc             --lv-expand            --quiet
--delete               --LVexpand             --resize
--dryrun               --LV-expand            --resize-force
--dry-run              --machine-readable     --shrink
--expand               --no-copy-boot-loader  --version
--format               --no-expand-content    

Where appropriate, the -d option will now expand to the list of libvirt domains:

# virt-df -d [tab]
archlinux20121201x64  f19rawhidex32
f18x64                f19rawhidex64

Finally, guestfish commands are expanded on the command line:

$ guestfish add /tmp/disk : run : list-[tab]
list-9p              list-events          list-md-devices
list-devices         list-filesystems     list-partitions
list-disk-labels     list-ldm-partitions  
list-dm-devices      list-ldm-volumes     

To make this less intrusive, so you can really use it daily, I left the default readline expansions enabled. This means that filenames and so on can continue to be used in every position on the command line, and should mean that bash completions won’t try to be cleverer than the user.

Libguestfs bash completions are also demand-loaded now, so that if you’re not using them, they don’t consume any resources in the shell.

Leave a comment

Filed under Uncategorized

Accessing NBD disks using libguestfs

It’s always been possible, but clumsy, to access Network Block Device (NBD) disks from libguestfs, but starting in libguestfs 1.22 we hope to make this (and Gluster, Ceph and Sheepdog access) much simpler.

The first change is upstream in libguestfs 1.21.21. You can add an NBD disk directly.

To show this using guestfish, I’ll start an NBD server. This could be started on another machine, but to make things simple I’ll start this server on the same machine:

$ qemu-nbd f18x64.img -t

f18x64.img is the disk image that I want to export. The -t option makes the qemu-nbd server persistent (ie. it doesn’t just exit after serving the first client).

Now we can connect to this server using guestfish as follows:

$ guestfish

Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.

Type: 'help' for help on commands
      'man' to read the manual
      'quit' to quit the shell

><fs> add-drive "" format:raw protocol:nbd server:localhost
><fs> run

The empty "" (quotes) are the export name. Since qemu-nbd doesn’t support export names, we can leave this empty. The main change is to specify the protocol (nbd) and the server that libguestfs should connect to (localhost, but a remote host would also work). I haven’t specified a port number here because both the client and server are using the standard NBD port (10809), but you could use server:localhost:NNN to use a different port number if needed.

Ordinary guestfish commands just work:

><fs> list-filesystems
/dev/sda1: ext4
/dev/fedora/root: ext4
/dev/fedora/swap: swap
><fs> inspect-os
/dev/fedora/root
><fs> inspect-get-product-name /dev/fedora/root
Fedora release 18 (Spherical Cow)

The next steps are to:

  1. Add support for more technologies. At least: Gluster, Ceph, Sheepdog and iSCSI, since those are all supported by qemu so we can easily leverage that support in libguestfs.
  2. Change the guestfish -a option to make adding remote drives possible from the command line.

The obvious [but not yet implemented] way to change the -a option is to allow a URI to be specified. For example:

$ guestfish -a nbd://localhost/exportname

where the elements of the URI like protocol, transport, server, port number and export name translate naturally into parameters of the add-drive API.

Leave a comment

Filed under Uncategorized