Half-baked ideas: Kernel modules-in-a-file

For more half-baked ideas, see the ideas tag.

The Linux kernel:

/boot/vmlinuz-3.11.9-200.fc19.x86_64

5 MB of highly optimized code that runs the world’s computers. You can copy it around and boot it on different hardware.

No wait, that’s all wrong. I’m forgetting about the modules. There are another 2832 files in /lib/modules/3.11.9-200.fc19.x86_64 and without them this kernel will hardly function.

It’s nice to be able to copy the kernel around — to compile it on fast x86-64 and run it on slow ARM, or to run the host kernel in a VM, but all those module files make everyone’s lives a lot more complex.

So my half-baked idea is this:

Combine all the modules into a single file, let’s call it /boot/modules-3.11.9-200.fc19.x86_64. This file might start off as an ext4 filesystem which can be loopback-mounted on /lib/modules. Eventually I’d hope that the module tools could load modules from the file directly.

Now you can happily cross-compile and copy just two kernel files from x86 to ARM or into your VMs, and that’s a lot less hassle.

How does mount load the right kernel module?

On any recent Linux distro, you can mount any filesystem type directly. For example:

# dd if=/dev/zero of=/tmp/test.img bs=4k count=4096
# mkfs.xfs /tmp/test.img
# mount -v -o loop /tmp/test.img /mnt/tmp

The mount command works even though I didn’t have the xfs.ko kernel module loaded, and I didn’t tell mount that it’s xfs.

How does it do that? I asked around several people at work and no one could give me the correct answer. So in this article I’ll describe exactly how it works.

First of all, I’ll mention two wrong answers to this: (a) the kernel doesn’t have a magic “mount any filesystem” syscall, and (b) it’s nothing to do with either /proc/filesystems or /etc/filesystems.

For (a), the mount(2) syscall clearly takes a filesystem type (string). As for (b), /proc/filesystems only lists filesystems which are known to the kernel already, ie. ones for which we’ve already loaded the right module. Since I didn’t have the xfs module loaded, xfs wasn’t listed in /proc/filesystems before I ran the mount command.

This should be enough of a clue that there must be some utility in userspace which knows how to probe the type by just looking at the header of any arbitrary filesystem. This utility is blkid, which used to be part of e2fsprogs but has now been combined with util-linux-ng.

blkid can probe a filesystem that it has not seen before and tell what type it is:

# blkid /tmp/test.img 
/tmp/test.img: UUID="c80ebc11-3b26-4b93-acbb-f52bdfaa9ac5" TYPE="xfs" 

Looking at the source for blkid confirms there is a directory full of probe tools for every conceivable filesystem.

The mount utility calls out to blkid — actually to the libblkid library, not to the command line tool, but it comes to the same thing.

So /bin/mount knows what it’s mounting, and requests the “xfs” filesystem type when it issues the system call into the kernel.

That still leaves the question of how the xfs module gets loaded. The answer is that the mount syscall eventually calls the kernel function __request_module. This strange function actually calls out to the userspace /sbin/modprobe binary, causing the module to get loaded. Meanwhile the mount syscall itself is paused. And yes, it even deals with the recursive situation where modprobe might need to mount filesystems or load other modules in order to succeed.

So there you have it, mounting a filesystem can magically load the right kernel module for that filesystem. All done using some userspace probing and some kernel trickery.