$ touch /var/tmp/disk.img
$ nbdfuse /var/tmp/disk.img --command nbdkit -s memory 7E &
$ ll /var/tmp/disk.img
-rw-rw-rw-. 1 rjones rjones 8070450532247928832 Nov 4 13:37 /var/tmp/disk.img
$ ls -lh /var/tmp/disk.img
-rw-rw-rw-. 1 rjones rjones 7.0E Nov 4 13:37 /var/tmp/disk.img
What can you actually do with this file, and more importantly does anything break? As in the talk, creating a Btrfs filesystem boringly just works. mkfs.ext4 spins using 100% of CPU. I let it go for 15 minutes but it seemed no closer to either succeeding or crashing. Emacs said:
File disk.img is large (7 EiB), really open? (y)es or (n)o or (l)iterally
and I was too chicken to find out what it would do if I really opened it.
I do wonder if there’s a DoS attack here if I leave this seemingly massive regular file lying around in a public directory.
Our tool nbdfuse lets you mount an NBD block device as a file, using Linux FUSE. For example you could create a directory with a single file in it (called nbd) which contains the contents of the NBD export:
Frama-C is a giant modular system for writing formal proofs of C code. For months I’ve been on-and-off trying to see if we could use it to do useful proofs for any parts of the projects we write, like qemu, libvirt, libguestfs, nbdkit etc. I got side-tracked at first with this frama-c tutorial which is fine, but I got stuck trying to make the GUI work.
You have to give all the include directories and define out some qemu-isms.
The first time you run it, this won’t work for “reasons”. You have to initialize the why3 verifier using:
why3 config --full-config
Really frama-c should just do this for you, or at least tell you what you need to do in the obscure error message it prints.
This still won’t work because util/range.c includes glib headers which use GCC attributes and builtins and frama-c simply cannot parse any of that. So I ended up hacking on the source to replace the headers with standard C headers and remove the one glib-based function in the file.
At this point it does compile and the frama-C WP plugin runs. Of course without having added any annotations it simply produces a long list of problems. Also it takes a fair bit of time to run, which is interesting. I wonder if it will get faster with annotations?
That’s as far as I’ve got for the moment. I’ll come back later and try to add annotations.
I got Fedora 32 installed on an RPi 4 8GB, booting off USB, with UEFI and ACPI. I followed Robert Grimm’s instructions here, and had an additional set of complications summarised here. There’s not much to say except that it was fiendishly complicated. But it works beautifully now, and is reasonably quick too especially when you consider how little it cost.
So let’s talk about costs (all include tax and delivery):
Raspberry Pi 4 8GB
SanDisk 500GB SSD x 2
small SD card needed for booting
Only one of the SSDs is actually used, but if you follow Robert’s instructions you will need two. I didn’t have any external USB SSDs that were both USB 3 and not spinning hard disks, so I had to buy these, but I’ll be able to reuse one in a future project. The SD card is required to work around a bug in the UEFI firmware, but I happened to have one lying around.
We ported nbdkit to Windows. That port is now upstream and should appear in the next stable release (1.24). There is also a new native file plugin for Windows which supports Windows files and volumes, hole punching for sparse files, querying file sparseness, and efficient zeroing.
This week I ported nbdkit, our high performance plugin-based Network Block Device server, to Windows. Currently it’s not upstream but you can download the Windows branch from here.
There were several possible ways we could have done this including Cygwin which might have been easier, but in the end I did a port to the raw Win32/Winsock APIs. You can compile it on Fedora using the mingw-w64-based Fedora Windows Cross Compiler and run it using Wine. Familiar commands like this work:
I got many existing plugins and filters compiled. Not all of those listed will be working, but the main features are fine. You can also write your own plugins to the same API as Linux ones. I’m hoping that someone can write a Windows block device plugin (especially one which integrates with features like VSS).
AMD Zen 2 laptops are a thing, and they’re blazingly fast.
I just bought the HP Envy x360 which has a 6 core AMD Ryzen 5 4500U. Measuring some real world compiles it’s comfortably two and half times faster than my year old Intel-based Thinkpad T480s (which has 4 cores but 8 threads, and cost at least twice as much).
$ tar tf rhel.ova
Since tar files usually store their content unmangled, this opens an interesting possibility for reading (or even writing) the embedded disk image without needing to unpack the tar. You just have to work out the offset of the disk image within the tar file. virt-v2v has used this trick to save a copy when importing OVAs for years.
nbdkit has also included a tar plugin which can access a file inside a local tar file, but the problem is what if the tar file doesn’t happen to be a local file? (eg. It’s on a webserver). Or what if it’s compressed?
To fix this I’ve turned the plugin into a filter. Using nbdkit-tar-filter you can unpack even non-local compressed tar files:
I am Richard W.M. Jones, a computer programmer. I have strong opinions on how we write software, about Reason and the scientific method. Consequently I am an atheist [To nutcases: Please stop emailing me about this, I'm not interested in your views on it] By day I work for Red Hat on all things to do with virtualization. I am a "citizen of the world".
My motto is "often wrong". I don't mind being wrong (I'm often wrong), and I don't mind changing my mind.
This blog is not affiliated or endorsed by Red Hat and all views are entirely my own.