Tag Archives: virt-v2v

KVM Forum 2015

Assuming HMG can get my passport back to me in time, I am speaking at the KVM Forum 2015 in Seattle USA (full schedule of talks here).

I’m going to be talking about virt-v2v and new features of qemu/KVM that made it possible for virt-v2v to be faster and more reliable than ever.

Leave a comment

Filed under Uncategorized

New in virt-v2v

Leave a comment

Filed under Uncategorized

Video: virt-v2v integration with RHEV-M

This video shows using the GUI to import a virtual machine from VMware to RHEV-M. It performs the conversion using virt-v2v, which is responsible for installing virtio drivers, fixing the bootloader, and so forth.

Thanks Arik Hadas. Now I just have to fix the epic RHEL 7.2 bug list — 57 bugs at last count :-(

4 Comments

Filed under Uncategorized

Tip: Read guest disks from VMware vCenter using libguestfs

virt-v2v can import guests directly from vCenter. It uses all sorts of tricks to make this fast and efficient, but the basic technique uses plain https range requests.

Making it all work was not so easy and involved a lot of experimentation and bug fixing, and I don’t think it has been documented up to now. So this post describes how we do it. As usual the code is the ultimate repository of our knowledge so you may want to consult that after reading this introduction.

Note this is read-only access. Write access is possible, but you’ll have to use ssh instead.

VMware ESXi hypervisor has a web server but doesn’t support range requests, so although you can download an entire disk image in one go from the ESXi hypervisor, to random-access the image using libguestfs you will need VMware vCenter. You should check that virsh dumpxml works against your vCenter instance by following these instructions. If that doesn’t work, it’s unlikely the rest of the instructions will work.

You will need to know:

  1. The hostname or IP address of your vCenter server,
  2. the username and password for vCenter,
  3. the name of your datacenter (probably Datacenter),
  4. the name of the datastore containing your guest (could be datastore1),
  5. .. and of course the name of your guest.

Tricky step 1 is to construct the vCenter https URL of your guest.

This looks like:

https://root:password@vcenter/folder/guest/guest-flat.vmdk?dcPath=Datacenter&dsName=datastore1

where:

root:password
username and password
vcenter
vCenter hostname or IP address
guest
guest name (repeated twice)
Datacenter
datacenter name
datastore1
datastore

Once you’ve got a URL that looks right, try to fetch the headers using curl. This step is important! not just because it checks the URL is good, but because it allows us to get a cookie which is required else vCenter will break under the load when we start to access it for real.

$ curl --insecure -I https://....
HTTP/1.1 200 OK
Date: Wed, 5 Nov 2014 19:38:32 GMT
Set-Cookie: vmware_soap_session="52a3a513-7fba-ef0e-5b36-c18d88d71b14"; Path=/; HttpOnly; Secure; 
Accept-Ranges: bytes
Connection: Keep-Alive
Content-Type: application/octet-stream
Content-Length: 8589934592

The cookie is the vmware_soap_session=... part including the quotes.

Now let’s make a qcow2 overlay which encodes our https URL and the cookie as the backing file. This requires a reasonably recent qemu, probably 2.1 or above.

$ qemu-img create -f qcow2 /tmp/overlay.qcow2 \
    -b 'json: { "file.driver":"https",
                "file.url":"https://..",
                "file.cookie":"vmware_soap_session=\"...\"",
                "file.sslverify":"off",
                "file.timeout":1000 }'

You don’t need to include the password in the URL here, since the cookie acts as your authentication. You might also want to play with the "file.readahead" parameter. We found it makes a big difference to throughput.

Now you can open the overlay file in guestfish as usual:

$ export LIBGUESTFS_BACKEND=direct
$ guestfish
><fs> add /tmp/overlay.qcow2 copyonread:true
><fs> run
><fs> list-filesystems
/dev/sda1: ext4
><fs> mount /dev/sda1 /

and so on.

4 Comments

Filed under Uncategorized

libguestfs 1.28 released

The new stable version of libguestfs — a C library and tools for accessing and modifying virtual machine disk images — has been released.

There is one brand new tool, virt-log. And I rewrote the virt-v2v and virt-p2v tools. These tools convert VMware and Xen guests and physical machines, to run on KVM. They are now much faster and better than before.

As well as that there are hundreds of other improvements and bug fixes. For a full list, see the release notes.

Libguestfs 1.28 will be available shortly in Fedora 21, Debian/experimental, RHEL and CentOS 7, and elsewhere.

3 Comments

Filed under Uncategorized

Odd/scary RHEL 5 bug

Yesterday my colleague gave me a RHEL 5 VM disk image which failed to boot after converting it using the latest virt-v2v.  Because it booted before conversion but not afterwards, the fingers naturally pointed at something that we were doing during the conversion process. Which is not unusual as v2v conversion is highly complex.

Screenshot_xen-pv-rhel5.8-x86_64
The “GRUB _” prompt after conversion

The thing is that we don’t reinstall grub during conversion, but we do edit a few grub configuration files. Could editing grub configuration cause this error?

I wanted to understand what the grub-legacy “GRUB _” prompt means. There are lots and lots and lots of people reporting this bug (eg), but as is often the case I could find no coherent explanation anywhere of what grub-legacy means when it gets into this state. Lots of the blind leading the blind, and random suggestions about how people had rescued such machines (probably coincidentally), but no hard data anywhere. So I had to go back to first principles and debug qemu to find out what’s happening just before the message is printed.

Tip: To breakpoint qemu when the Master Boot Record (first sector) is loaded, do:

target remote tcp::1234
set architecture i8086
b *0x7c00
cont

After an evening of debugging, I found that it’s the first sector (known in grub-legacy as “stage 1”) which prints the GRUB<space> message. (The same happens to be true of grub2). The stage 1 boot sector has, written into it at a fixed offset, the location of the /boot/grub/stage2 file, ie. the literal disk start sector and length of this file. It sends BIOS int $0x13 commands to load those sectors into memory at address 0x8000, and jumps there to start the stage 2 of grub. The boot sector is 512 bytes, so there’s no luxury to do anything except print 5 characters. It’s after the stage2 file has been loaded when all the nice graphical stuff happens.

Unfortunately in the image after conversion, the stage2 data loaded into memory was all zeroes, and that’s why the boot fails and you see GRUB<space><cursor> and then the VM crashes.

The mystery was how conversion could be changing the location of the /boot/grub/stage2 file so that it could no longer be loaded at the fixed offset encoded in the boot sector.

This morning it dawned on me what was really happening …

The new virt-v2v tries very hard to avoid copying any unused data from the guest, just to save time. No point wasting time copying deleted files and empty space. This makes virt-v2v very fast, but it has an unusual side-effect: If a file is deleted on the source, the contents of the file are not copied over to the target, and turn into zeroes.

It turns out if you take the source disk image and simply zero all of the empty space in /boot, then the source doesn’t boot either, even though virt-v2v is not involved. Yikes … this could be a bug in RHEL 5. Grub is generating a bootloader that references a deleted file.

This is where we are right now with this bug. It appears that a valid sequence of steps can make a RHEL 5 bootloader that references a deleted file, but still works as long as you never overwrite the sectors used by that file.

I have written a simple test script that you can download to find out if your RHEL ≤ 6 virtual machines could be affected by this problem. I’m interested if anyone else sees this. I ran the test over a selection of RHEL 3 – 5 guests, and could not find any which had the problem, but my collection is not very extensive, and there are likely to be common modes in how they were created.

The next steps will likely be to test a lot more RHEL 5 installs to see if this bug is really common or a strange one-off. I will also probably add a workaround to virt-v2v so it doesn’t trim the boot partition — the reason is that we cannot go back and fix old RHEL 5 installs, we have to work with them if they are broken. If it turns out to be a real bug in RHEL 5 then we will need to issue a fix for that.

3 Comments

Filed under Uncategorized

virt-v2v preview packages for RHEL and CentOS 7.1 are available

virt-v2v is a small program for converting guests from VMware or Xen, to run on KVM, RHEV-M or OpenStack. For RHEL 7.1, I am rewriting and enhancing virt-v2v, so it’s much faster and easier to use.

To install, follow the instructions here for setting up the yum repository, and then you can do:

yum install virt-v2v

To use it, start with the manual here that has lots of examples and the full reference documentation.

8 Comments

Filed under Uncategorized