Tag Archives: libnbd

libnbd + FUSE = nbdfuse

I’ve talked before about libnbd, our NBD client library. New in libnbd 1.2 is a tool called nbdfuse which lets you turn NBD servers into virtual files.

A couple of weeks ago I mentioned you can use libnbd as a C library to edit qcow2 files. Now you can turn qcow2 files into virtual raw files:

$ mkdir dir
$ nbdfuse dir/file.raw \
      --socket-activation qemu-nbd -f qcow2 file.qcow2
$ ls -l dir/
total 0
-rw-rw-rw-. 1 nbd nbd 1073741824 Jan  1 10:10 file.raw

Reads and writes to file.raw are backed by the original qcow2 file which is updated in real time.

Another fun thing to do is to use nbdkit, xz filter and curl to turn xz-compressed remote disk images into uncompressed local files:

$ mkdir dir
$ nbdfuse dir/disk.img \
      --command nbdkit -s curl --filter=xz \
$ ls -l dir/
total 0
-rw-rw-rw-. 1 nbd nbd 6442450944 Jan  1 10:10 disk.img
$ file dir/disk.img
dir/disk.img: DOS/MBR boot sector
$ qemu-system-x86_64 -m 4G \
      -drive file=dir/disk.img,format=raw,if=virtio,snapshot=on

Leave a comment

Filed under Uncategorized

Using American Fuzzy Lop on network clients

Previously I’ve fuzzed hivex and nbdkit using my favourite fuzzing tool, Michał Zalewski’s American Fuzzy Lop (AFL).

AFL works by creating test cases which are files on disk, and then feeding those to programs which have been specially compiled so that AFL can trace into them and find out which parts of the code are run by the test case. It then adjusts the test cases and repeats, aiming to run more parts of the code and find ways to crash the program.

This works well for programs that parse files (like hivex, but also binary parsers of all sorts and XML parsers and similar). It can also be used to fuzz some servers where you can feed a file to the server and discard anything the server sends back. In nbdkit we can use the nbdkit -s option to do exactly this, making it easy to fuzz.

However it’s not obvious how you could use this to fuzz network clients. As readers will know we’ve been writing a new NBD client library called libnbd. But can we fuzz this? And find bugs? As it happens yes, and ooops — yes — AFL found a remote code execution bug allowing complete takeover of the client by a malicious server.

The trick to fuzzing a network client is to do the server thing in reverse. We set up a phony server which feeds the test case back to the client socket, while discarding anything that the client writes:


This is wrapped up into a single wrapper program which takes the test case on the command line and forks itself to make the client and server sides connected by a socket. This allows easy integration into an AFL workflow.

We found our Very Serious Bug within 3 days of fuzzing.

Leave a comment

Filed under Uncategorized

How to edit a qcow2 file from C

Suppose you want to edit or read or write the data inside a qcow2 file? One way is to use libguestfs, and that’s the recommended way if you need to mount a filesystem inside the file.

But for accessing the data blocks alone, you can now use the libnbd API and qemu-nbd together and this has a couple of advantages: It’s faster and you can open snapshots (which libguestfs cannot do).

We start by creating a libnbd handle and connecting it to a qemu-nbd instance. The qemu-nbd instance is linked with qemu’s internal drivers that know how to read and write qcow2.

  const char *filename;
  struct nbd_handle *nbd;

  nbd = nbd_create ();
  if (nbd == NULL) {
    fprintf (stderr, "%s\n", nbd_get_error ());
    exit (EXIT_FAILURE);

  char *args[] = {
    "qemu-nbd", "-f", "qcow2",
    // "-s", snapshot,
    (char *) filename,
  if (nbd_connect_systemd_socket_activation (nbd, args) == -1) {
    fprintf (stderr, "%s\n", nbd_get_error ());
    exit (EXIT_FAILURE);

Now you can get the virtual size:

  int64_t size = nbd_get_size (nbd);
  printf ("virtual size = %" PRIi64 "\n", size);

Or read and write sectors from the file:

  if (nbd_pread (nbd, buf, sizeof buf, 0, 0) == -1) {
    fprintf (stderr, "%s\n", nbd_get_error ());
    exit (EXIT_FAILURE);

Once you’re done with the file, call nbd_close on the handle which will also shut down the qemu-nbd process.

A complete example can be found here.

1 Comment

Filed under Uncategorized

nbdkit supports exportnames

(You’ll need the very latest version of libnbd and nbdkit from git for this to work.)

The NBD protocol lets the client send an export name string to the server. The idea is a single server can serve different content to clients based on a requested export. nbdkit has largely ignored export names, but we recently added basic support upstream.

One consequence of this is you can now write a shell plugin which reflects the export name back to the client:

$ cat export.sh
#!/bin/bash -
case "$1" in
    open) echo "$3" ;;
    get_size) LC_ALL=C echo ${#2} ;;
    pread) echo "$2" | dd skip=$4 count=$3 iflag=skip_bytes,count_bytes ;;
    *) exit 2 ;;
$ chmod +x export.sh
$ nbdkit -f sh export.sh

The size of the disk is the same as the export name:

$ nbdsh -u 'nbd://localhost/fooooo' -c 'print(h.get_size())'

The content and size of the disk is the exportname:

│ f o o o o o │

Not very interesting in itself. But we can now pass the content of small disks entirely in the export name. Using a slightly more advanced plugin which supports base64-encoded export names (so we can pass in NUL bytes):

$ cat export-b64.sh
#!/bin/bash -
case "$1" in
    open) echo "$3" ;;
    get_size) echo "$2" | base64 -d | wc -c ;;
    pread) echo "$2" | base64 -d |
           dd skip=$4 count=$3 iflag=skip_bytes,count_bytes ;;
    can_write) exit 0 ;;
    pwrite) exit 1 ;;
    *) exit 2 ;;
$ chmod +x export-b64.sh
$ nbdkit -f sh export-b64.sh

We can pass in an entire program to qemu:

qemu-system-x86_64 -fda 'nbd:localhost:10809:exportname=uBMAzRD8uACgjtiOwLQEo5D8McC5SH4x//OriwVAQKuIxJK4AByruJjmq7goFLsQJbELq4PAFpOr/sST4vUFjhWA/1x167+g1LEFuAQL6IUBg8c84vW+lvyAfAIgci3+xYD9N3SsrZetPCh0CzwgdQTGRP4o6F4Bgf5y/XXbiPAsAnLSNAGIwojG68qAdAIIRYPlB1JWVXUOtADNGjsWjPx09okWjPy+gPy5BACtPUABl3JD6BcBge9CAYoFLCByKVZXtAT25AHGrZfGBCC4CA7oAgFfXusfrQnAdC09APCXcxTo6ACBxz4BuAwMiXz+gL1AAQt1BTHAiUT+gD0cdQbHBpL8OAroxgDizL6S/K0IwHQMBAh1CLQc/g6R/HhKiUT+izzorgB1LrQCzRaoBHQCT0+oCHQCR0eoA3QNgz6A/AB1Bo1FCKOA/Jc9/uV0Bz0y53QCiQRdXlqLBID6AXYKBYACPYDUchvNIEhIcgODwARQ0eixoPbx/syA/JRYcgOAzhaJBAUGD5O5AwDkQDz8cg2/gvyDPQB0A6/i+Ikd6cL+GBg8JDx+/yQAgEIYEEiCAQC9234kPGbDADxa/6U8ZmYAAAAAAAAAAHICMcCJhUABq8NRV5xQu6R9LteTuQoA+Ij4iPzo4f/Q4+L1gcdsAlhAqAd14J1fWcNPVao='


1 Comment

Filed under Uncategorized

libnbd and nbdkit man pages online

Let’s seed those search engines …



And yes I know there are a few broken links.

Leave a comment

Filed under Uncategorized

fio has support for testing NBD servers directly

fio (“flexible I/O tester”) is a key tool for testing performance of filesystems and block devices. A couple of weeks ago I added support for testing NBD servers directly.

You will need libnbd 0.9.8 and to compile fio from git with:

./configure --enable-libnbd

Testing either nbdkit or qemu-nbd is trivial. Just follow the instructions here.


Filed under Uncategorized

libnbd 0.9.8 and stable APIs

I announced libnbd yesterday. The libnbd 0.9.8 is a pre-release for the upcoming 1.0 where we will finalize the API and offer API and ABI stability.

Stable APIs aren’t in fashion these days, but they’re important because people who choose to use your platform for their software shouldn’t be screwed over and have to change their software every time you change your mind. In C it’s reasonably easy to offer a stable API while allowing long term evolution and even incompatible changes. This is what we do for nbdkit and will be doing for libnbd.

The first concept to get to know is ELF symbol versioning. Chapter 3 of Uli’s paper on the subject covers this in great detail. In libnbd all our initial symbols will be labelled with LIBNBD_1.0.

The second concept is not as well known, but is used in glibc, nbdkit and elsewhere: Allowing callers to opt in to newer versions of the API. Let’s say we want to add a new parameter to nbd_connect_uri. The current definition is:

int nbd_connect_uri (struct nbd_handle *h, const char *uri);

but we decide to add flags:

int nbd_connect_uri (struct nbd_handle *h, const char *uri,
                     uint32_t flags);

You can cover up the ABI change with ELF symbol versions. However the API has still changed, and you’re forcing your callers to change their source code eventually. To prevent breaking the API, you can allow callers to opt in to the change by defining a new symbol:

#include <libnbd.h>

The definition is saying “I want to upgrade to the new version and I’ve made the required changes”, but crucially if the definition is missing or has an earlier version number then you continue to offer the earlier version, perhaps with a simple wrapper that calls the new version with a default flags value.

That’s what we intend to do with libnbd to offer a stable API and ABI (once 1.0 is released) without breaking callers either at the source or binary level.

1 Comment

Filed under Uncategorized