I was making some thin clients for the Fedora RISC-V project a few weeks ago. These are based on the HiFive Unleashed U540 board and so they have no local SATA, only slow, unreliable SD cards. Any filesystems that might be heavily used must be network filesystems.
As these are Linux clients we essentially have three possible choices for network filesystems: NFS, NBD or a cluster FS. As I didn’t want to set up multiple server nodes or need the redundancy, cluster filesystems are immediately discounted. NFS is — “fine”. And indeed I selected that for
/home where performance is not a problem. But for the clients that are build servers I selected NBD as a high-performance block-based storage. I’m using nbdkit, the high performance flexible NBD server.
However nbdkit didn’t quite do what I wanted, so I had to write a new plugin. I needed a “remote tmpfs“.
To get a new “remote tmpfs”, a fresh filesystem each time, the client does:
modprobe nbd nbd-client server /dev/nbd0 mount /dev/nbd0 /var/scratch
Backing this is a flexible new plugin called tmpdisk. By default it creates a new disk for each connection, formats it with mkfs and serves it to the client. But it’s also scriptable, so you can substitute any command you like instead of mkfs to create your own custom disks (I recommend looking at
mke2fs -d in case you need to have pre-populated scratch disks).
It’s important to note that these are scratch disks so when the client unmounts the filesystem it is completely deleted from the server. But that’s fine for temporary directories and (for my purposes) package builds.
To find out more about nbdkit, watch my video from FOSDEM 2019.
2 responses to “New nbdkit “remote tmpfs” (tmpdisk plugin)”
Previous examples of yours used nbd-client -b 512 localhost/dev/nbd0. But you dropped the – b 512, presumably due to the change in nbd-client’s default. Do you forsee any problems with using -b 4096 on a regular basis?
Thought I’d replied to this, but I found out just now not only didn’t I reply but the comment was stuck in moderation for a few weeks. Sorry!
So I believe you may use
-b 4096to emulate an Advanced Format drive. However I doubt anyone has ever tested it, so unfortunately you may end up keeping all the pieces if it breaks.