nbdkit new eval plugin and ip filter

nbdkit is our flexible toolkit for building block devices. I just added a couple of new features which will appear in the next stable release, nbdkit 1.18.

Previously I’ve talked on this blog and gave a talk at FOSDEM about how you can write block devices in shell script using nbdkit-sh-plugin. But that requires you to use an extra file for the script. What if opening an extra file is too much work? Well now you can specify the script directly on the nbdkit command line using the new eval plugin.

You can write code like:

nbdkit eval \
       config='ln -sf "$(realpath "$3")" $tmpdir/file' \
       get_size='stat -Lc %s $tmpdir/file' \
       pread='dd if=$tmpdir/file skip=$4 count=$3 iflag=count_bytes,skip_bytes' \
       pwrite='dd of=$tmpdir/file seek=$4 conv=notrunc oflag=seek_bytes' \
       file=disk.img

which is a complete NBD server / block device backed by a local file. Of course it’s probably easier to use nbdkit-file-plugin for this, but the shell script gives you more control like letting you simulate failures or delays.

The other new feature is connected to a CVE we had earlier this year. CVE-2019-14850 happened because nbdkit used to open the plugin as soon as any client established a TCP connection. For some plugins opening them is quite a heavyweight action (eg. it might mean that the plugin has to establish a connection to a second server). This is before NBD negotiation or TLS had started, and it allowed clients potentially to overwhelm the server with requests even if those clients would not be authorized to connect.

To fix this we delay opening plugins until after the NBD handshake (and thus TLS authentication) has completed. But this in turn meant there was no way for plugins to reject connections early, for example based on IP address. So now I have added a preconnect method which gets runs on first TCP connection and can be used to do lightweight early filtering. There is a new nbdkit-ip-filter which implements simple TCP-wrappers-style allow/deny lists.

nbdkit inline scripts

I have proposed a patch for nbdkit, our flexible, pluggable Network Block Device server, to make writing Linux block devices into a (long) single command.

Here’s a simple block device with virtual size 1M that reads as zeroes:

nbdkit sh - <<'EOF'
    case "$1" in
        get_size) echo 1M ;;
        pread) dd if=/dev/zero count=$3 iflag=count_bytes ;;
        *) exit 2 ;;
    esac
EOF

Tip: List all files in a virtual machine

Note: This requires libguestfs 1.5.x or the libguestfs 1.4.3 + backports that we are shipping in Fedora 13 updates testing.

Quick tip: list all the files in a VM:

$ guestfish --ro -a disk.img -i find0 / - | tr '\000' '\n' | sort | less

Explanation:

1. guestfish --ro -a disk.img -i is the new style way to inspect a disk image using guestfish. For a libvirt domain, use guestfish --ro -d GuestName -i instead.
2. find0 / - lists all the files to stdout.
3. The filenames are separated by ASCII NUL character, tr '\000' '\n' translates that to newlines.
4. sort | less sorts and pages the output.

Quick tip: Timing things in a shell script

We all know about the ‘time’ command. That’s great if you have one command that you want to time.

But how about if you want to time several actions together in a shell script, eg:

#!/bin/sh -
start_the_clock
A
B
C
how_long_so_far
D
E
stop_the_clock

The ‘time’ command isn’t so useful for this, unless you want to stick ‘time’ in front of each action A, B, C, D & E and add them up in your head.

But there’s a nice little date/awk trick you can use for this:

#!/bin/sh -
date +%s.%N > /tmp/times
A
B
C
date +%s.%N >> /tmp/times
D
E
date +%s.%N >> /tmp/times
awk '{ if (!start) { start = $1 } else { print $1-start } }' < /tmp/times

which would print something out like this:

25.2957   # the time that actions A-C took
128.529   # the time that actions A-E took

(Thanks Jim Meyering for suggesting date +%s.%N)