Tag Archives: virt-cat

Transactions with guestfish

I was asked a few days ago if libguestfs has a way to apply a group of changes to an image together. The question was really about transaction support — applying a group of changes and then committing them or doing a rollback, with the final image either containing all the changes or none of them.

Although libguestfs doesn’t support this, you can do it using libguestfs and the qemu-img tool together. This post shows you how.

First I use virt-builder to quickly get a test image that I can play with:

$ virt-builder fedora-20

We create an overlay which will store the changes until we decide to commit or rollback:

$ qemu-img create -f qcow2 -b fedora-20.img overlay.img

Now open the overlay and make your changes:

$ guestfish -a overlay.img -i

Welcome to guestfish, the guest filesystem shell for
editing virtual machine filesystems and disk images.

Type: 'help' for help on commands
      'man' to read the manual
      'quit' to quit the shell

Operating system: Fedora release 20 (Heisenbug)
/dev/sda3 mounted on /
/dev/sda1 mounted on /boot

><fs> write-append /etc/issue.net \
    "THIS IS A CHANGE TO ISSUE.NET\n"
><fs> cat /etc/issue.net
Fedora release 20 (Heisenbug)
Kernel \r on an \m (\l)
THIS IS A CHANGE TO ISSUE.NET
><fs> exit

The base image (fedora-20.img) is untouched, and the overlay contains the changes we made:

$ virt-cat -a fedora-20.img /etc/issue.net
Fedora release 20 (Heisenbug)
Kernel \r on an \m (\l)
$ virt-cat -a overlay.img /etc/issue.net
Fedora release 20 (Heisenbug)
Kernel \r on an \m (\l)
THIS IS A CHANGE TO ISSUE.NET

Rollback

Rollback is pretty simple!

$ rm overlay.img

Commit

The more interesting one is how to commit the changes back to the original file. Using qemu-img you just do:

$ qemu-img commit overlay.img
Image committed.
$ rm overlay.img

The changes are now contained in the original image file:

$ virt-cat -a fedora-20.img /etc/issue.net
Fedora release 20 (Heisenbug)
Kernel \r on an \m (\l)
THIS IS A CHANGE TO ISSUE.NET

ACID

Have we discovered the ACID properties of disk images? Not quite.

Although the change is atomic (A)1, the disk image is consistent (C) before and after the change, and the change is durable (D)2, the final property is not satisfied.

There is no isolation (I). Because it is infeasible to resolve conflicts at the block layer where qemu-img operates, it would be guaranteed corruption if you tried this technique in parallel on the same disk image. The only way to make it work reliably is to serialize every operation on the disk image with a mutex.

1 The change is only atomic if you don’t look at the backing file for the short time that qemu-img commit runs.

2 Strictly speaking, you must call sync or fsync after the qemu-img commit in order for the change to be durable.

Advertisements

Leave a comment

Filed under Uncategorized

Using libguestfs to find out why a Windows guest was “hanging”

While diagnosing a bug where a Windows guest hangs at boot, I used libguestfs to find out what files were being updated on the disk. Here is how.

First of all I used virt-ls to get a listing of all the files in the guest and the last time they were updated:

$ virt-ls -lR -a /path/to/winxp.img --time-relative / | \
  grep '^-' > /tmp/files

The colums in the output file look like this:

- 0777          0 12022162 12022162 12022162 /AUTOEXEC.BAT

The three numbers in columns 4, 5 and 6 (“12022162”) are the ones we are interested in. These are the time of last access, time of last modification and time of last status change, in seconds before now (because of the --time-relative flag).

So now we’re just looking for the files where column 6 is a small number. Everything that’s been touched in the last 2 minutes, for example:

$ awk '$6 < 120' < /tmp/files
- 0777       1024       40       40       40 /Documents and Settings/rjones/NTUSER.DAT.LOG
- 0777       7414       30       30       30 /WINDOWS/Prefetch/LOGON.SCR-151EFAEA.pf
- 0777    9445376       50       50       50 /WINDOWS/SoftwareDistribution/DataStore/DataStore.edb
- 0777       8192       50       50       50 /WINDOWS/SoftwareDistribution/DataStore/Logs/edb.chk
- 0777     131072       50       50       50 /WINDOWS/SoftwareDistribution/DataStore/Logs/edb.log
- 0777     203243       49       49       49 /WINDOWS/WindowsUpdate.log
- 0777       1024       41       40       40 /WINDOWS/system32/config/SAM.LOG
- 0777     262144      645       42       42 /WINDOWS/system32/config/SECURITY
- 0777      20480       42       41       41 /WINDOWS/system32/config/SECURITY.LOG

Ah.

Looks to me like Windows Update is running.

We can confirm this easily:

$ virt-cat -a /path/to/winxp.img /WINDOWS/WindowsUpdate.log|tail
2012-02-27	19:17:57:718	 824	144	DnldMgr	Error 0x80072f78 occurred while downloading update; notifying dependent calls.
2012-02-27	19:18:12:546	 824	144	DnldMgr	Error 0x80072f78 occurred while downloading update; notifying dependent calls.
2012-02-27	19:18:39:015	 824	14c	DnldMgr	Error 0x80072f78 occurred while downloading update; notifying dependent calls.
2012-02-27	19:18:49:031	 824	7b8	DnldMgr	Error 0x80072f78 occurred while downloading update; notifying dependent calls.
2012-02-27	19:18:58:046	 824	14c	DnldMgr	Error 0x80072f78 occurred while downloading update; notifying dependent calls.
2012-02-27	19:18:58:062	 824	748	AU	AU checked download status and it changed: Downloading is paused

Indeed soon afterwards the guest came back to life, after downloading all its Windows Updates.

3 Comments

Filed under Uncategorized

Tip: Another way to get the IP address of a virtual machine

In this earlier post several ways were discussed of getting the IP address of a virtual machine (specifically if the VM is acquiring a different address each time from DHCP).

Another way is to use virt-cat or virt-win-reg to grab the information out of the virtual machine itself.

When a VM acquires an IP address from DHCP then it writes the address to a log file, or in the Windows case updates the Windows Registry. The idea is simply that we’ll read the information from the VM’s log files or the Registry.

The details vary depending on the precise guest type, and probably we should wrap this up in a nice virt tool. But here’s how you would do it for Fedora/RHEL and Windows guests:

# virt-cat RHEL60x64 /var/log/messages | grep 'dhclient.*bound to'
Mar 30 19:56:22 rhel60x64 dhclient: bound to 192.168.122.220 -- renewal in 1527 seconds.
Mar 30 20:21:49 rhel60x64 dhclient: bound to 192.168.122.220 -- renewal in 1375 seconds.
Mar 30 20:44:44 rhel60x64 dhclient: bound to 192.168.122.220 -- renewal in 1287 seconds.
Mar 30 21:06:11 rhel60x64 dhclient: bound to 192.168.122.220 -- renewal in 1461 seconds.
# virt-win-reg Win7x32 \
  'HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\Tcpip\Parameters' | \
  grep DhcpIPAddress
"DhcpIPAddress"=hex(1):31,00,39,00,32,00,2e,00,31,00,36,00,38,00,2e,00,31,00,32,00,32,00,2e,00,31,00,37,00,38,00,00,00

In the Windows case you need to note that ControlSet001 isn’t always the right control set — consult this note in the virt-win-reg man page to find the correct way to do this. Furthermore virt-win-reg prints out the hex-encoded UTF16-LE string, which requires a little bit of conversion to produce a printable string (192.168.122.178 in this instance).

Update: To add to all the other methods from the previous post, and the method described above, Eric Blake also points out that nwfilter can sniff IP addresses used by VMs.

2 Comments

Filed under Uncategorized

New (Year’s) libguestfs tools: virt-copy-in, virt-copy-out, virt-tar-in, virt-tar-out

One aim with libguestfs development is to make easy and common file operations easy. Although you can already upload and download files into virtual machines using guestfish commands, is there a way to make this common operation easier to discover?

One way is to add more virt commands, which I’ve found that users have least difficulty discovering because they are on the website, autocompleted when you hit virt-[tab], and listed as separated manual pages.

So today I added four more commands for uploading and downloading: virt-copy-in, virt-copy-out, virt-tar-in, virt-tar-out.

The way you use them is very simple:

$ mkdir homes
$ virt-copy-out -d Fedora14 /home homes/
$ virt-tar-out -d Fedora14 /home - | \
    gzip --best > homes.tar.gz

These commands are just small shell script wrappers around guestfish, but I hope they make common things a little bit easier.

You can get these new commands from Fedora Rawhide, or as binaries for Debian or Ubuntu.

Leave a comment

Filed under Uncategorized

Tip: Uploading and downloading

Talked to someone today who was confused by how to get files in and out of a virtual machine disk image. Libguestfs, guestfish, virt-cat, virt-edit, virt-ls and virt-tar offer several ways to do this, and in this article I’m going to summarize all these different ways.

First of all, answer these questions:

  1. Upload (ie. from your host into the disk image) or
    download (disk image down to host)?
  2. Single file or lots of files?
  3. Have you got / do you want a tarball, or a directory containing the individual files?
  4. Do you want to download the files or just get a list of files?

I should note as usual that uploading or modifying a guest is only safe if the guest is shut off. If you try to modify a live guest you’ll get disk corruption, so watch out.

Downloading a single file

Easiest way is with virt-cat:

virt-cat GuestName /path/to/file > file

An alternative is to use guestfish download:

guestfish -i --ro -d GuestName \
    download /path/to/file ./localfile

Uploading a single file

If the file exists and it’s a text file that you want to edit, the easiest way is to use virt-edit:

virt-edit GuestName /path/to/file

To upload a single file in general, use guestfish upload:

guestfish -i -d GuestName \
    upload ./localfile /path/to/file

Downloading multiple files

Easiest way is probably to use the copy-out command:

guestfish -i --ro -d GuestName \
    copy-out /directory ./localdir

Uploading multiple files

Use guestfish copy-in command:

guestfish -i -d GuestName <<EOF
  mkdir /directory
  copy-in ./localdir /directory
EOF

If you want to quickly upload lots of files, the fastest method is to prepare an ISO (mkisofs) or squashfs containing the files and attach it as an extra disk. You can then copy files quickly to where you need them, eg:

guestfish -i -d GuestName -a my.iso <<EOF
  mount-ro /dev/sdb /mnt
  cp-a /mnt/files /someplace
EOF

Downloading or uploading a tarball

There is a high level tool called virt-tar which automates this. You can also do it semi-manually using the guestfish commands tar-in, tar-out, tgz-in, tgz-out (and more, see the guestfish documentation).

Listing files

Finally to list out files, use the high level virt-ls tool, or the low level guestfish commands ls (flat list of a single directory) or find0 (recursive listing of subdirectories).

Leave a comment

Filed under Uncategorized

virt-cat now works on encrypted guests

In libguestfs 1.7.4, virt-cat can be used on encrypted guests:

# virt-cat -d F13x64Encrypted /etc/issue.net
Enter key or passphrase ("/dev/vda2"): ***
Fedora release 13 (Goddard)
Kernel \r on an \m (\l)

We’ve made a change to the way the virt-cat command line works (I’m going to make similar changes to the other libguestfs virt tools), but don’t worry because the old style is still supported and will remain supported indefinitely. The new style is less ambiguous and more flexible though.

Leave a comment

Filed under Uncategorized

Tip: Get the hostname of a guest

Because different operating systems store the hostname in different places, you have to know in advance what sort of OS your guest is (perhaps using virt-inspector). Perhaps we should add the hostname to virt-inspector.

This works for Fedora guests:

# virt-cat F13x64 /etc/sysconfig/network | \
  grep HOSTNAME= | \
  awk -F= '{print $2}'
f13x64.home.annexia.org

This is for Debian/Ubuntu guests:

# virt-cat Debian5x64 /etc/hostname
debian5x64.home.annexia.org

For Windows guests:

# virt-win-reg Win7x32 \
  'HKEY_LOCAL_MACHINE\System\ControlSet001\Services\Tcpip\Parameters' \
  Hostname
win7x32

I’m not completely clear how to get the DNS domain name from Windows. According to this article you should just replace “Hostname” with “Domain” in the above command, but for me that yields just an empty string.

4 Comments

Filed under Uncategorized