Tag Archives: gzip

nbdkit for loopback pt 4: loopback-mounting compressed images

nbdkit is a pluggable NBD server with lots of plugins and filters. Two of the plugins[1] handle compressed files (for gzip and xz respectively). We can uncompress and serve a file on the fly. For gzip it’s kind of inefficient. For xz it’s very efficient provided you prepared your xz files ahead of time with a smaller than default block size.

Let’s use nbdkit to loopback mount an xz file:

$ nbdkit -fv xz file=/var/tmp/fedora-28.img.xz
# nbd-client -b 512 localhost /dev/nbd0
Warning: the oldstyle protocol is no longer supported.
This method now uses the newstyle protocol with a default export
Negotiation: ..size = 6144MB
Connected /dev/nbd0
# ls /dev/nbd0p*
/dev/nbd0p1  /dev/nbd0p2  /dev/nbd0p3  /dev/nbd0p4
# fdisk -l /dev/nbd0
Device        Start      End Sectors  Size Type
/dev/nbd0p1    2048     4095    2048    1M BIOS boot
/dev/nbd0p2    4096  2101247 2097152    1G Linux filesystem
/dev/nbd0p3 2101248  3360767 1259520  615M Linux swap
/dev/nbd0p4 3360768 12580863 9220096  4.4G Linux filesystem
# mount -o ro /dev/nbd0p4 /mnt

Of course it’s read-only. To write to a compressed file would involve changing the size of inner parts of the file. Use qcow2 compression if you want a writable compressed file (although writes to that format are not compressed).

Also loopback mounting in general is unsafe. Use libguestfs to safely mount untrusted disk images.

[1] These should really be filters, not plugins, so that you can chain an uncompression filter into an existing plugin, and one day I’ll get around to writing that.


Leave a comment

Filed under Uncategorized

Three plugins for nbdkit

So far I’ve written six plugins for nbdkit. However three of those are examples which don’t really count.

The first (more) interesting plugin is called file and it just turns nbdkit into a regular NBD server, serving files or devices. It’s almost complete, the only significant missing features being access logging and hole punching.

The second interesting plugin is called libvirt-plugin. It serves disks from libvirt guests. An example of using it can be found here.

The final interesting plugin is called gzip. It uses the zlib API to open a .gz file, exposing it uncompressed. Because zlib is a stream-oriented API it’s not very usable at the moment (especially for large images) because it has to uncompress the data stream as it’s seeking. However it may be possible to improve that by caching positions in the stream. What would be more interesting for me would be a lzma-based plugin to support xz files.


Filed under Uncategorized