New (Year’s) libguestfs tools: virt-copy-in, virt-copy-out, virt-tar-in, virt-tar-out

One aim with libguestfs development is to make easy and common file operations easy. Although you can already upload and download files into virtual machines using guestfish commands, is there a way to make this common operation easier to discover?

One way is to add more virt commands, which I’ve found that users have least difficulty discovering because they are on the website, autocompleted when you hit virt-[tab], and listed as separated manual pages.

So today I added four more commands for uploading and downloading: virt-copy-in, virt-copy-out, virt-tar-in, virt-tar-out.

The way you use them is very simple:

$ mkdir homes
$ virt-copy-out -d Fedora14 /home homes/

$ virt-tar-out -d Fedora14 /home - | \
    gzip --best > homes.tar.gz

These commands are just small shell script wrappers around guestfish, but I hope they make common things a little bit easier.

You can get these new commands from Fedora Rawhide, or as binaries for Debian or Ubuntu.

Tip: Uploading and downloading

Talked to someone today who was confused by how to get files in and out of a virtual machine disk image. Libguestfs, guestfish, virt-cat, virt-edit, virt-ls and virt-tar offer several ways to do this, and in this article I’m going to summarize all these different ways.

First of all, answer these questions:

1. Upload (ie. from your host into the disk image) or
   download (disk image down to host)?
2. Single file or lots of files?
3. Have you got / do you want a tarball, or a directory containing the individual files?
4. Do you want to download the files or just get a list of files?

I should note as usual that uploading or modifying a guest is only safe if the guest is shut off. If you try to modify a live guest you’ll get disk corruption, so watch out.

Downloading a single file

Easiest way is with virt-cat:

virt-cat GuestName /path/to/file > file

An alternative is to use guestfish download:

guestfish -i --ro -d GuestName \
    download /path/to/file ./localfile

Uploading a single file

If the file exists and it’s a text file that you want to edit, the easiest way is to use virt-edit:

virt-edit GuestName /path/to/file

To upload a single file in general, use guestfish upload:

guestfish -i -d GuestName \
    upload ./localfile /path/to/file

Downloading multiple files

Easiest way is probably to use the copy-out command:

guestfish -i --ro -d GuestName \
    copy-out /directory ./localdir

Uploading multiple files

Use guestfish copy-in command:

guestfish -i -d GuestName <<EOF
  mkdir /directory
  copy-in ./localdir /directory
EOF

If you want to quickly upload lots of files, the fastest method is to prepare an ISO (mkisofs) or squashfs containing the files and attach it as an extra disk. You can then copy files quickly to where you need them, eg:

guestfish -i -d GuestName -a my.iso <<EOF
  mount-ro /dev/sdb /mnt
  cp-a /mnt/files /someplace
EOF

Downloading or uploading a tarball

There is a high level tool called virt-tar which automates this. You can also do it semi-manually using the guestfish commands tar-in, tar-out, tgz-in, tgz-out (and more, see the guestfish documentation).

Listing files

Finally to list out files, use the high level virt-ls tool, or the low level guestfish commands ls (flat list of a single directory) or find0 (recursive listing of subdirectories).

Tip: Pack files into a new disk image

Note: This requires libguestfs ≥ 1.4

If files/ is a directory containing some files, you can create a new disk image populated with those files using the command below (virt-make-fs can also be used for this):

$ guestfish -N fs:ext2:400M -m /dev/sda1 tar-in <(tar -C files/ -cf - .) /

Explanation:

1. guestfish -N fs:ext2:400M creates a prepared disk image which is 400MB in size with a single partition formatted as ext2. The new disk image is called test1.img.
2. -m /dev/sda1 mounts the new filesystem when guestfish starts up.
3. tar-in [...] / uploads and unpacks the tar file into the root directory of the mounted disk image.
4. <(tar -C files/ -cf - .) is a bash process substitution which runs the tar command and supplies the output (ie. the tar file) as the input on the command line.

This is a quick way to create “appliances” using febootstrap and libguestfs, although you should note that I don’t think these appliances would really work, I just use them for testing our virtualization management tools, like the ability to detect and manage disk images:

$ febootstrap -i fedora-release -i bash -i setup -i coreutils fedora-13 f13
$ echo '/dev/sda1 / ext2 defaults 1 1' > fstab
$ febootstrap-install f13 fstab /etc 0644 root.root
$ febootstrap-minimize f13
$ guestfish -N fs:ext2:40M -m /dev/sda1 tar-in <(tar -C f13 -cf - .) /
$ guestfish --ro -i -a test1.img

Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.

Type: 'help' for a list of commands
      'man' to read the manual
      'quit' to quit the shell

Operating system: Fedora release 13 (Goddard)
/dev/vda1 mounted on /
><fs> cat /etc/fstab
/dev/sda1 / ext2 defaults 1 1

Tip: In guestfish, extract files from disk image to a local directory

You’ve loaded your filesystem in guestfish, how do you extract all the files into the local directory on the host? Easy …

><fs> tgz-out / - | tar zxf -

This is a little bit subtle. Here’s how it works:

On the left hand side of the pipe, we run the guestfish “tgz-out” command which turns / into a tarball and sends it to “-” (stdout).

On the right hand side of the pipe is a host command, tar, which unpacks stdin into the local directory. (This explains how pipes work in guestfish)

N.B.: An alternative, for VM images, is to use the higher-level virt-tar command.

guestfish -i, that’s i for “inspector” not “interactive”

I wrote this FAQ entry today …

---

• What’s the deal with guestfish -i?
• Why does virt-cat only work on a real VM image, but virt-df works on any disk image?
• What does “no root device found in this operating system image” mean?

These questions are all related at a fundamental level which may not be immediately obvious.

At the libguestfs API and guestfish level, a “disk image” is just a pile of partitions and filesystems.

In contrast, when the virtual machine boots, it mounts those filesystems into a consistent hierarchy such as:

 /          (/dev/sda2)
 |
 +-- /boot  (/dev/sda1)
 |
 +-- /home  (/dev/vg_external/Homes)
 |
 +-- /usr   (/dev/vg_os/lv_usr)
 |
 +-- /var   (/dev/vg_os/lv_var)

(or drive letters on Windows).

The libguestfs API however knows nothing about how filesystems are mounted. In general, a disk image that we just made up might not contain an operating system or “mountpoints” at all.

Nevertheless, users expect some tools (like virt-cat) to work with VM paths:

virt-cat fedora.img /var/log/messages

How does virt-cat know that /var is a separate partition?

The trick is a large external program called virt-inspector. Virt-inspector contains all sorts of tests and heuristics, so it can take a disk image about which nothing is known beforehand and work out both what it is, and how filesystems would be mounted.

Some tools, such as virt-cat, virt-edit, virt-ls and virt-tar use the same virt-inspector code to map VM paths. Other tools, such as virt-df and virt-list-filesystems operate entirely at the raw “big pile of filesystems” level of the libguestfs API, and don’t use virt-inspector code.

guestfish is in an interesting middle ground. If you use the -a and -m command line options, then you have to tell guestfish exactly how to add disk images and where to mount partitions. This is the raw API level.

However there is also a guestfish -i (for “inspector”) mode. In this mode, guestfish runs virt-inspector, and then (if successful) virt-inspector runs guestfish a second time with the correct -a and -m options. This is much slower, but usually simpler for the user. They can just do this and it all works:

guestfish -i SomeLibvirtGuest

The error “no root device found in this operating system image” is related to this. It means virt-inspector was unable to locate an operating system within the disk image you gave it. You might see this from programs like virt-cat if you try to run them on something which is just a disk image, not a virtual machine.

New tool: virt-tar

I just posted a new libguestfs tool called virt-tar, a sort of general purpose archiving, backup, upload and snooping tool for virtual machines.

You can use this to extract or archive directories from a guest, like this:

# virt-tar -zx someguest /home home.tar.gz

which takes a copy of /home in the VM called someguest and saves it to a local file home.tar.gz.

Or upload to a guest, like this:

# virt-tar -zu someguest /tmp libguestfs-1.0.73.tar.gz

which would unpack the tarball into /tmp/libguestfs-1.0.73/ inside the guest called someguest. You can’t use the upload feature on live guests.

All of the above functionality is already available through guestfish, but having a separate tool to do it makes it easier to remember for the simpler cases.