Write nbdkit plugins in shell script(!)

nbdkit is a pluggable NBD server and you can write plugins in C or several other scripting languages. But not shell script – until now. Shell script turns out to be a reasonably nice language for this:

case "$1" in
  get_size)
    stat -L -c '%s' $f ||
      exit 1 ;;
  pread)
    dd iflag=skip_bytes,count_bytes skip=$4 count=$3 if=$f ||
      exit 1 ;;
  pwrite)
    dd oflag=seek_bytes conv=notrunc seek=$4 of=$f ||
      exit 1 ;;

Now that coreutils util-linux provides the fallocate program we can even implement efficient trim and zeroing operations.

Quick tip: Timing things in a shell script

We all know about the ‘time’ command. That’s great if you have one command that you want to time.

But how about if you want to time several actions together in a shell script, eg:

#!/bin/sh -
start_the_clock
A
B
C
how_long_so_far
D
E
stop_the_clock

The ‘time’ command isn’t so useful for this, unless you want to stick ‘time’ in front of each action A, B, C, D & E and add them up in your head.

But there’s a nice little date/awk trick you can use for this:

#!/bin/sh -
date +%s.%N > /tmp/times
A
B
C
date +%s.%N >> /tmp/times
D
E
date +%s.%N >> /tmp/times
awk '{ if (!start) { start = $1 } else { print $1-start } }' < /tmp/times

which would print something out like this:

25.2957   # the time that actions A-C took
128.529   # the time that actions A-E took

(Thanks Jim Meyering for suggesting date +%s.%N)

Tip: Shell redirection

Did you know you can open file descriptors in the shell? My colleague didn’t, so here’s my 30 second summary on file descriptor redirection. It’s a POSIX standard too!

Shell	Equivalent in C
# Open a log file on fd 5 for the duration # of this shell script. exec 5>/tmp/log	dup2 (open ("/tmp/log", O_WRONLY), 5);
# Send a log message to fd 5. echo hello >&5	write (5, "hello\n", ..);
# Open fd 6 for input for the duration # of this shell script. exec 6</tmp/input	dup2 (open ("/tmp/input", O_RDONLY), 6);
# Read a line from fd 6. read line <&6	Approximately like doing read on fd 6, but it’s more like fgets in that it reads one line, and it can optionally split the line into whitespace-separated fields.

Redirection is actually far more powerful than this. You can close file descriptors, duplicate them and append to them. But the four uses above are the most common. For the rest, plough through the bash manual page section on redirection here.

Tip: ^ and ! in the shell

I’m astounded. So don’t people know about using ^ and ! in the shell?!? Last week I watched an experienced Linux user carefully hit the ↑ cursor key to get a previous line of history, then ←&→ just to make a simple edit!

Here’s my 30 second guide:

!!	Repeat the previous command. Example: $ ls bin d Desktop rpmbuild tmp $ !! ls bin d Desktop rpmbuild tmp
!-2 (etc)	Repeat the command 2 previously (so !! = !-1), or for any number previously. This is the most useful I think.
^foo^bar	Replace foo with bar in the previous command. eg: $ ls -l /etx/httpd/conf.d/local.conf ls: cannot access /etx/httpd/conf.d/local.conf: No such file or directory $ ^etx^etc ls -l /etc/httpd/conf.d/local.conf -rw-r--r-- 1 root root 76 2009-07-16 14:59 /etc/httpd/conf.d/local.conf
!foo	Run the most recent command that started foo, eg: $ !ls ls -l /etc/httpd/conf.d/local.conf -rw-r--r-- 1 root root 76 2009-07-16 14:59 /etc/httpd/conf.d/local.conf

Today I discovered that these are called event designators. When I first saw someone using them it was on an Apollo workstation circa 1991, and I pretty quickly picked these up myself.