I’m writing a new nbdkit plugin called linuxdisk. nbdkit is our flexible, plugin-based NBD server, and this new plugin lets you create a complete Linux-compatible virtual disk from a host directory on the fly.
One of the many uses for this is booting minimal VMs very quickly. Here’s an example you can set up in a few seconds. It boots to an interactive busybox shell:
$ mkdir /tmp/root /tmp/root/sbin /tmp/root/bin /tmp/root/dev
$ sudo mknod /tmp/root/dev/console c 5 1
$ cp /sbin/busybox /tmp/root/sbin/
$ ln /tmp/root/sbin/busybox /tmp/root/bin/sh
$ ln /tmp/root/sbin/busybox /tmp/root/bin/ls
$ ln /tmp/root/sbin/busybox /tmp/root/sbin/init
$ nbdkit -U - linuxdisk /tmp/root \
--run 'qemu-kvm -display none -kernel /boot/vmlinuz-4.20.8-200.fc29.x86_64 -drive file=nbd:unix:$unixsocket,snapshot=on -append "console=ttyS0 root=/dev/sda1 rw" -serial stdio'
If you need any extra files in the VM just drop them straight into
/tmp/root before booting it.
Edit: How the heck does
/dev get populated in this VM?
nbdkit is our flexible, pluggable NBD server. See my FOSDEM video to find out more about some of the marvellous things you can do with it.
The news is you can now write nbdkit plugins in Rust. As with the OCaml bindings, Rust plugins compile to native
*.so plugin files which are loaded and called directly from nbdkit. However it really needs a Rust expert to implement a more natural and idiomatic way to use the API than what we have now.
Loop mounting is popular, but very limited in what it can do on Linux. I gave a talk at FOSDEM on Saturday entitled Better loop mounts with NBD: Take your loop mounts to the next level with nbdkit, and it’s online already!
Download the WebM format or MP4 format files directly.
Also I did subtitles! Download the subtitles directly here. (The subs only cover the first 30 minutes of the talk, not the Summary and Q&A.)
(Thanks to Thomas Huth for the photo)
There are a few small problems and corrections:
- There’s a part of the talk where I refer to the light blue trimmed blocks. Unfortunately the video feed didn’t capture that, so the light blue looks like white. If you really want to see that then go look at the video in my earlier post.
- During the Q&A I mentioned that we could support writing to xz files. This is true, sort of, but I forgot that there’s a problem: nbdkit doesn’t support file resizing (and I believe that’s even experimental in the NBD protocol), so someone would have to add that first. There are other serious down-sides to implementing writable XZ, I doubt it could ever be fast.
For subtitles I used gaupol which is actually quite nice, although subtitling is inherently slow and tedious. It took me a good 4 hours to subtitle 30 minutes of video.
My talk was accepted: https://fosdem.org/2019/schedule/event/nbdkit/
If you’re coming to FOSDEM, please come and say hello. In the meantime if you want to watch a rough early run-through of the talk, see: https://rwmj.wordpress.com/2018/11/26/nbdkit-fosdem-test-presentation/
I have proposed a patch for nbdkit, our flexible, pluggable Network Block Device server, to make writing Linux block devices into a (long) single command.
Here’s a simple block device with virtual size 1M that reads as zeroes:
nbdkit sh - <<'EOF'
case "$1" in
get_size) echo 1M ;;
pread) dd if=/dev/zero count=$3 iflag=count_bytes ;;
*) exit 2 ;;
I’ve submitted a talk about nbdkit, our flexible pluggable NBD server, to FOSDEM next February. This is going to be about using NBD as a better way to do loop mounts in Linux.
In preparation I gave a very early version of the talk to a small Red Hat audience.
Video link: http://oirase.annexia.org/rwmj.wp.com/rjones-nbdkit-tech-talk-2018-11-19.mp4
Sorry about the slow start. You may want to skip to 2 mins to get past the intro.
Summary of what’s in the talk:
- Demo of regular, plain loop mounting.
- Demo of loop mounting an XZ-compressed disk image using NBD + nbdkit.
- Slides about how loop device compares to NBD.
- Slides about nbdkit plugins and filters.
- Using VMware VDDK to access a VMDK file.
- Creating a giant disk costing EUR 300 million(!)
- Visualizing a single filesystem.
- Visualizing RAID 5.
- Writing a plugin in shell script (live demo).