nbdkit is a liberally licensed NBD server which a stable plugin API for serving disks from unconventional sources.
Finally I got around to adding TLS (encryption and authentication) support. The support is complete and appears to interoperate with QEMU. It also supports a certificate authority, client certificate verification, certificate revocation, server verification (by the client), and configurable algorithms.
Actually using TLS with NBD is no easy matter. It takes a few pages of instructions just to explain how to set up the public-key infrastructure. On the client (QEMU) side, the command line parameter for connecting to a TLS-enabled NBD server is lengthy.
Then there’s the question of how you ensure TLS is being used. In nbdkit as in other NBD servers you can either turn on TLS in which case it’s used when the client requests it, or you can require TLS. In the latter case nbdkit will reject non-TLS connections (thus ensuring TLS is really being used), but most clients won’t be able to connect to such a server.
As usual, where SSH got it right, SSL/TLS/HTTPS got it all horribly wrong.
Richard WM Jones 
“As usual, where SSH got it right, SSL/TLS/HTTPS got it all horribly wrong” — can you elaborate on this? I vaguely remember reading (several) articles about overly complex configuration and bad defaults, but a good summary would be appreciated. Thanks!
The problem is we shouldn’t be afraid of self-signed certificates and pinning which is what ssh does by default. The alternative is setting up a full public key infrastructure, which is both stupidly complex but requires that the CA bothers to verify your identity correctly, which for a £3.99/year certificate basically never happens.
Ah, by “self-signed certificates and pinning” you mean “~/.ssh/known_hosts”, right? (Possibly tweaked with the StrictHostKeyChecking config option.) Makes sense. Thanks!
That’s cool. I’ve started to explore the nbd world just today.
I have a kind of holy dream to store my data on a remote server through a LUKS container.
So the answer to my first question “does nbd support authentication and encryption or am I required to use SSH” is “yes”.
The second question: does nbdkit support trim/discard? How could I shrink the size of image if I delete some files in it?
It supports trim if the plugin supports it. The file plugin actually doesn’t support it (through laziness), but it would be trivial to add.
Most people use nbdkit because they’re writing their own plugin for some unusual use case. There are (arguably) better general purpose nbd servers out there.
Hi,
I once started to write an nbd server based on netty. It should be easy to add TLS there too:
https://github.com/thomasmey/nnbd
Pingback: Great new changes coming to nbdkit | Richard WM Jones
Hi Rich,
I am trying to use TLS in combination of nbdkit & nbd-client but it is failing during negotiation.
I followed https://libguestfs.org/nbdkit-tls.1.html to generate certificates & keys at /home/kapilwsl/keys.
Then I am starting nbd service as –
$ nbdkit –unix /tmp/nbd.sock -f -v –tls=require –tls-certificates /home/kapilwsl/keys file disk.img
and nbd-client as –
$ nbd-client -u /tmp/nbd.sock -x -cacertfile /home/kapilwsl/keys/ca-cert.pem -certfile /home/kapilwsl/keys/client-cert.pem -n /dev/nbd0
Warning: the oldstyle protocol is no longer supported.
This method now uses the newstyle protocol with a default export
Negotiation: ..E: received invalid negotiation magic 6984491201319600128 (expected 1100100111001001)
Added “keyfile” parameter to nbd-client and it is still failing –
$ nbd-client -u /tmp/nbd2.sock -x -cacertfile /home/kapilwsl/keys/ca-cert.pem -certfile /home/kapilwsl/keys/client-cert.pem -n -keyfile /home/ka
pilwsl/keys/client-key.pem /dev/nbd0
Warning: the oldstyle protocol is no longer supported.
This method now uses the newstyle protocol with a default export
Negotiation: ..E: received invalid negotiation magic 19429238037741568 (expected 1100100111001001)
Yes I think there’s a genuine interop problem here (of course this is exactly the case that we don’t test …). I will discuss this with Eric.
Filed an upstream bug: https://gitlab.com/nbdkit/nbdkit/-/issues/1
The problem is actually with using TLS over a Unix domain socket. This works OK with nbdkit, but nbd-client cannot connect for some reason. If you use a port number then it works fine. Or use an alternate client like nbdfuse which supports TLS over Unix domain sockets.
When using a Unix socket with nbd-client, you MUST also use the ‘-H $hostname’ option. Otherwise, nbd-client tries to use your socket’s name as the host name, and fails during the TLS handshake when that does not match the expected hostname of the certificate.
Hi Rich,
Thanks a lot for testing it so quickly. Appreciate that. I will explore nbdfuse.