New nbdkit “remote tmpfs” (tmpdisk plugin)

I was making some thin clients for the Fedora RISC-V project a few weeks ago. These are based on the HiFive Unleashed U540 board and so they have no local SATA, only slow, unreliable SD cards. Any filesystems that might be heavily used must be network filesystems.

As these are Linux clients we essentially have three possible choices for network filesystems: NFS, NBD or a cluster FS. As I didn’t want to set up multiple server nodes or need the redundancy, cluster filesystems are immediately discounted. NFS is — “fine”. And indeed I selected that for /home where performance is not a problem. But for the clients that are build servers I selected NBD as a high-performance block-based storage. I’m using nbdkit, the high performance flexible NBD server.

However nbdkit didn’t quite do what I wanted, so I had to write a new plugin. I needed a “remote tmpfs“.

To get a new “remote tmpfs”, a fresh filesystem each time, the client does:

modprobe nbd
nbd-client server /dev/nbd0
mount /dev/nbd0 /var/scratch

Backing this is a flexible new plugin called tmpdisk. By default it creates a new disk for each connection, formats it with mkfs and serves it to the client. But it’s also scriptable, so you can substitute any command you like instead of mkfs to create your own custom disks (I recommend looking at mke2fs -d in case you need to have pre-populated scratch disks).

It’s important to note that these are scratch disks so when the client unmounts the filesystem it is completely deleted from the server. But that’s fine for temporary directories and (for my purposes) package builds.

To find out more about nbdkit, watch my video from FOSDEM 2019.

Leave a comment

Filed under Uncategorized

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.