With this patch series, nbdkit, the pluggable Network Block Device server, supports FreeBSD ≥ 11.2.
Tag Archives: nbd
The Network Block Device (NBD) protocol is really useful to us when we deal with virtual machines and disk images. It lets us share disk images between machines and is also the universal protocol we use for communicating disk images between different bits of software. I wrote a pluggable NBD server called nbdkit to make this even easier.
However there was a problem: The protocol has no concept of logins. If you have an open NBD port, then anyone can connect and read or write your disk image. This is not quite as terrible as it sounds since when two processes are talking NBD to each other, we use a Unix domain socket and we hide the socket in a directory with restrictive permissions. But there are still cases — such as communicating between separate servers — where authentication would be useful.
NBD does let you upgrade the protocol to use TLS, and all the important NBD servers support that. You can use TLS to do client authentication but it’s seriously clunky and difficult to set up because you have to use X.509 certificates, and if we’ve learned anything from the web we know that X.509 is a plot by the NSA to stop us using encryption (only joking, spooks!)
It turns out there’s a more sensible corner of the TLS specification called TLS-PSK. This uses usernames and randomly generated Pre-Shared Keys (PSK). As long as you can ensure that both the client and server can read a simple
username:key file of keys, and the keys are kept secret, you can both authenticate and communicate securely.
Unfortunately just implementing TLS doesn’t get you PSK as well, and no existing NBD server supports TLS-PSK.
Amazingly it all works, and qemu and nbdkit interoperate too. Here’s how you could use it:
$ mkdir -m 0700 /tmp/keys $ psktool -u rich -p /tmp/keys/keys.psk
$ nbdkit -n \ --tls=require --tls-psk=/tmp/keys/keys.psk \ file file=disk.img
$ qemu-img info \ --object "tls-creds-psk,id=tls0,endpoint=client,username=rich,dir=/tmp/keys" \ --image-opts "file.driver=nbd,file.host=localhost,file.port=10809,file.tls-creds=tls0"
The qemu command line is a bit clunky, but it’s overall much simpler than setting up certificates, although not as scalable for large installations.
nbdkit is our toolkit for creating Network Block Device (NBD) servers from “unusual” data sources. nbdkit was already configurable by writing simple plugins in several programming languages. Last week Eric Blake and I added a nice new feature: You can now modify existing plugins by placing “filters” in front of them.
- Serving only an offset + range within the underlying disk.
- Serving a single partition if the disk is a partitioned disk image.
- Injecting delays for simulating slow servers and testing for race conditions.
- Adding a copy-on-write overlay if the plugin is read-only.
(You can also layer filters to arbitrary depth)
nbdkit 1.1.27 has three simple filters, and 1.1.28 will include two more, and you can write your own although (unlike plugins) filters do not yet have a stable ABI and we haven’t decided if we will offer a stable ABI in future.
Eric Blake has been doing some great stuff for nbdkit, the flexible plugin-based NBD server.
- Full parallel request handling.
You’ve always been able to tell nbdkit that your plugin can handle multiple requests in parallel from a single client, but until now that didn’t actually do anything (only parallel requests from multiple clients worked).
- An NBD forwarding plugin, so if you have another NBD server which doesn’t support a feature like encryption or new-style protocol, then you can front that server with nbdkit which does.
As well as that he’s fixed lots of small bugs with NBD compliance so hopefully we’re now much closer to the protocol spec (we always check that we interoperate with qemu’s nbd client, but it’s nice to know that we’re also complying with the spec). He also fixed a potential DoS where nbdkit would try to handle very large writes which would delay a thread in the server indefinitely.
Finally I got around to adding TLS (encryption and authentication) support. The support is complete and appears to interoperate with QEMU. It also supports a certificate authority, client certificate verification, certificate revocation, server verification (by the client), and configurable algorithms.
Actually using TLS with NBD is no easy matter. It takes a few pages of instructions just to explain how to set up the public-key infrastructure. On the client (QEMU) side, the command line parameter for connecting to a TLS-enabled NBD server is lengthy.
Then there’s the question of how you ensure TLS is being used. In nbdkit as in other NBD servers you can either turn on TLS in which case it’s used when the client requests it, or you can require TLS. In the latter case nbdkit will reject non-TLS connections (thus ensuring TLS is really being used), but most clients won’t be able to connect to such a server.
As usual, where SSH got it right, SSL/TLS/HTTPS got it all horribly wrong.