Read and writing VMware .vmdk disks

(This is in answer to an IRC question, but the answer is a bit longer than I can cover in IRC)

Can you read and write at the block level in a .vmdk file? I think the questioner was asking about writing a backup/restore type tool. Using only free software, qemu can do reads. You can attach qemu-nbd to a vmdk file and that will expose the logical blocks as NBD, and you can then read at the block level using libnbd:

#!/usr/bin/python3
import nbd
h = nbd.NBD()
h.connect_systemd_socket_activation(
    ["qemu-nbd", "-t", "/var/tmp/disk.vmdk"])
print("size = %d" % h.get_size())
buf = h.pread(512, 0)

$ ./qemu-test.py 
size = 1073741824

The example is in Python, but libnbd would let you do this from C or other languages just as easily.

While this works fine for reading, I wouldn’t necessarily be sure that writing is safe. The vmdk format is complex, baroque and only lightly documented, and the only implementation I’d trust is the one from VMware.

So as long as you’re prepared to use a bit of closed source software and agree with the (nasty) license, VDDK is the safer choice. You can isolate your own software from VDDK using our nbdkit plugin.

#!/usr/bin/python3
import nbd
h = nbd.NBD()
h.connect_command(
    ["nbdkit", "-s", "--exit-with-parent",
     "vddk", "libdir=/var/tmp/vmware-vix-disklib-distrib",
     "file=/var/tmp/disk.vmdk"])
print("size = %d" % h.get_size())
buf = h.pread(512, 0)
h.pwrite(buf, 512)

I quite like how we’re using small tools and assembling them together into a pipeline in just a few lines of code:

┌─────────┬────────┐          ┌─────────┬────────┐
│ your    │ libnbd │   NBD    │ nbdkit  │ VDDK   │
│ program │     ●──────────────➤        │        │
└─────────┴────────┘          └─────────┴────────┘
                                          disk.vmdk

One advantage of this approach is that it exposes the extents in the disk which you can iterate over using libnbd APIs. For a backup tool this would let you save the disk efficiently, or do change-block tracking.

nbdkit for loopback pt 3: loopback mounting VMware disks

nbdkit is a pluggable NBD server and it comes with a very wide range of plugins (of course you can also write your own). One of them is the VMware VDDK plugin, an interface between nbdkit and the very proprietary VMware VDDK library. The library allows you to read local VMware disks or access remote VMware servers. In this example I’m going to use it to loopback mount a VMDK file:

$ export LD_LIBRARY_PATH=~/tmp/vddk/vmware-vix-disklib-distrib/lib64
$ nbdkit -fv vddk \
    libdir=~/tmp/vddk/vmware-vix-disklib-distrib \
    file=/var/tmp/fedora.17.x86-64.20120529.vmdk

When loopback-mounting you must use a 512 byte sector size (see the mailing list for discussion):

# nbd-client -b 512 localhost /dev/nbd0
Warning: the oldstyle protocol is no longer supported.
This method now uses the newstyle protocol with a default export
Negotiation: ..size = 10240MB
Connected /dev/nbd0

Standard health warning: Loopback mounting any unknown disk is dangerous! You should use libguestfs instead as it protects you from harmful disks and also doesn’t require root.

It turns out this VMDK file contains a partitioned disk with one partition:

# fdisk -l /dev/nbd0
Disk /dev/nbd0: 10 GiB, 10737418240 bytes, 20971520 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 1024 bytes / 1024 bytes
Disklabel type: dos
Disk identifier: 0x000127ae

Device      Boot Start      End  Sectors Size Id Type
/dev/nbd0p1       2048 20971519 20969472  10G 83 Linux

and it can be mounted directly and it’s fully writable:

# mount /dev/nbd0p1 /mnt
# ls -l /mnt
total 76
lrwxrwxrwx.  1 root root     7 May 29  2012 bin -> usr/bin
dr-xr-xr-x.  4 root root  4096 May 29  2012 boot
drwxr-xr-x.  2 root root  4096 Feb  3  2012 dev
drwxr-xr-x. 59 root root  4096 May 29  2012 etc
drwxr-xr-x.  2 root root  4096 Feb  3  2012 home
lrwxrwxrwx.  1 root root     7 May 29  2012 lib -> usr/lib
lrwxrwxrwx.  1 root root     9 May 29  2012 lib64 -> usr/lib64
[etc]
# touch /mnt/hello

VMware VDDK plugin for nbdkit

VDDK is a horribly proprietary library that VMware has released to let you open VMDK files and access the disks of ESX servers. It has some NBD capability already, but that’s not stopped me from creating a VDDK plugin for nbdkit so you can access VMware resources over NBD.

Make sure you read the README.VDDK file first …

New project: nbdkit, liberally licensed NBD server with a plugin API

Last week I started a new project: nbdkit. This is a toolkit for creating NBD servers. The key features are:

1. Multithreaded NBD server written in C with good performance.
2. Well-documented, simple plugin API with a stable ABI guarantee. Let’s you export “unconventional” block devices easily.
3. Liberal license (BSD) allows nbdkit to be linked to proprietary libraries or included in proprietary code.

There are of course many NBD servers already, such as the original nbd project, qemu-nbd and jnbds.

There are also a handful of servers specialized for particular disk sources. A good example of that is this OpenStack Swift server. But you shouldn’t have to write a whole new server just to export a new disk type.

nbdkit hopefully offers a unique contribution to this field because it’s a general server with a plugin architecture, offering a stable ABI and a liberal license so you can link it to proprietary code (say hello, VDDK).

The motivation for this is to make many more data sources available to libguestfs. Especially I want to write plugins for libvirt, VDDK and some OpenStack sources.