48 core ARM 64 bit server

# cat /proc/cpuinfo
processor	: 0
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer	: 0x43
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0x0a1
CPU revision	: 1

processor	: 1
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer	: 0x43
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0x0a1
CPU revision	: 1

[... very long output truncated here ...]

processor	: 44
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer	: 0x43
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0x0a1
CPU revision	: 1

processor	: 45
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer	: 0x43
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0x0a1
CPU revision	: 1

processor	: 46
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer	: 0x43
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0x0a1
CPU revision	: 1

processor	: 47
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer	: 0x43
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0x0a1
CPU revision	: 1

It’s a Cavium ThunderX. Thanks Cavium and Jon.

Leave a comment

Filed under Uncategorized

Booting Fedora on the Dragonboard

TL;DR: All the crazy bootloader shit that affected 32 bit ARM infects 64 bit ARM too.

I pretty much followed Robert Gadsdon’s instructions here, except I had to compile mkbootimg by hand from the CyanogenMod sources as it doesn’t seem to be available in Fedora.

Luckily I have all the bits you need to attach a USB serial port:


U-boot was installed as the bootloader as above.

For the disk image, I started with the virt-builder Fedora 23 aarch64 image. I copied this image to the SD card by doing:

virt-builder --arch aarch64 fedora-23 --output /dev/mmcblkX

I didn’t expect this would boot, because the kernel is wrong, and the u-boot I flashed to the Dragonboard is in any case looking for a different file. So I cross-compiled a kernel by following the Linaro instructions here (only up to the make Image step).

I didn’t manage to get the kernel to boot, because it just doesn’t load. Or maybe it does load but doesn’t find the serial port. Whatever.


Filed under Uncategorized



Leave a comment

November 12, 2015 · 3:13 pm

Linux Kernel Library backend for libguestfs

For a very long time I’ve been keeping an eye on rump kernels. These are a fancy way of saying ordinary kernel code which has been ported or altered in some way so you can link it to userspace programs, usually as a library. There have been claims that rump kernels make libguestfs obsolete, which is a load of nonsense for reasons I’ll talk about in a minute. But rump kernels are interesting and finally I’ve been able to add a rump kernel library to libguestfs.

The first time I encountered rump kernels was in 2010 with Prof. Renzo Davoli’s virtualsquare View-OS project. The more recent rump kernel project was based on NetBSD. Since NetBSD doesn’t use Linux drivers, but reimplementations, it was never very interesting. We couldn’t have included it in RHEL, and it would be only a matter of time before someone found an important feature or data-corrupting difference between the real ext2/3 driver from Linux and the reimplementation in BSD. Last I looked, NetBSD didn’t even support ext4. The choice of NetBSD made it a non-starter for us.

libos arrived earlier this year and, being based on Linux, was a lot more interesting, but their aim was only the Linux network stack so it wasn’t directly relevant to libguestfs, which is only concerned with disk and filesystem access.

However last week Richard Weinberger (Linux UML maintainer) CC’d me on a much more interesting project. LKL (Linux Kernel Library) is a relatively small patchset on top of Linux that turns Linux into a library. Quite literally you just link your program with -llkl and you can start a Linux kernel and make Linux system calls. A few example programs can be found here.

Today I wrote a work-in-progress LKL backend for libguestfs. [Note for those keen to jump in and use it, it doesn’t do very much at the moment. There is a small to-do list. But it’s only a few hours more work to have it as a useful backend for libguestfs.]

What could an LKL-enabled libguestfs do? Well it could open a raw disk image that contains only a Linux filesystem, or possibly a partitioned disk image with a Linux filesystem. And it can do that very quickly – in about 0.03 seconds in my tests (compared to 2.5 seconds for the qemu-based libguestfs).

There are a long list of things that LKL cannot do however. It cannot open anything containing LVM (because LVM requires userspace tools, that would all have to be modified to use liblkl, and there would be all kinds of synchronization problems between the LVM-linked liblkl kernel and the libguestfs-linked kernel). It cannot run anything that requires a userspace tool. That includes btrfs, ntfs-3g (FUSE based), some MD devices, encrypted disks, dm-thinp and Windows Dynamic Disks.

It cannot create new LVs or filesystems or any of the other disk structures that libguestfs can create.

It cannot open disk formats like qcow2 and vmdk. The qcow2 code in particular is very specific to qemu, and cannot possibly be ported to the kernel.

It cannot open remote devices like Ceph, Gluster, https, ssh or ftp (although there is an nbd client in the kernel, so one day that may be possible).

It cannot run commands in the context of the guest.

It’s also less secure than libguestfs with the qemu backend, because it doesn’t isolate your host from exploits in the guest or filesystem using virtualization and sVirt.

All of those things can be done by libguestfs today, but may never be possible (or only in limited cases) with LKL.

But within the limits of what LKL can do, it should become an excellent choice (assuming it gets upstream). Adding an LKL backend to libguestfs brings to the table the large, but clean and well-tested libguestfs API and the language bindings and virt tools, so libguestfs gives you the best of both worlds: the performance of LKL by using that backend, or the massive feature set of libguestfs by switching over to the qemu or libvirt backend, which for most programs is a single line code change.


Filed under Uncategorized

Tip: guestmount (FUSE mount) every filesystem in a disk image

Maxim asks an interesting question which is if you’ve got a disk image, how do you mount every filesystem onto your host. Like this:

$ ./fs-mount.pl rhel-5.11.img /tmp/fs &
$ cd /tmp/fs
/tmp/fs$ ls
/tmp/fs$ cd dev
/tmp/fs/dev$ ls
sda1  sda2  sda3
/tmp/fs/dev$ cd sda2
/tmp/fs/dev/sda2$ ls
bin   dev  home  lib64       media  mnt  proc  sbin     srv  tmp  var
boot  etc  lib   lost+found  misc   opt  root  selinux  sys  usr
$ cd /tmp
$ guestunmount /tmp/fs

The answer is this surprisingly short Perl script.


use warnings;
use strict;

use Sys::Guestfs;

die "usage: $0 disk1 [disk2 ...] mountpoint\n" if @ARGV <= 1;

my $mp = pop;

my $g = Sys::Guestfs->new ();
foreach (@ARGV) {
    $g->add_drive ($_);
$g->launch ();

# Examine the filesystems.
my %fses = $g->list_filesystems ();

# Create the mountpoint directories (in the libguestfs namespace)
# and mount the filesystems on them.
foreach my $fs (sort keys %fses) {
    # mkmountpoint is really the same as mkdir.  Unfortunately there
    # is no 'mkdir -p' equivalent, so we have to do this instead:
    my @components = split ("/", $fs);
    for (my $i = 1; $i < @components; ++$i) {
        my $dir = "/" . join ("/", @components[1 .. $i]);
        eval { $g->mkmountpoint ($dir) }

    # Don't fail if the filesystem can't be mounted, eg. it's swap.
    eval { $g->mount ($fs, $fs) }

# Export the filesystem on the host.
$g->mount_local ($mp);
$g->mount_local_run ();

# Close nicely since we mounted everything writable.
$g->shutdown ();
$g->close ();

Leave a comment

Filed under Uncategorized

Virt-builder Fedora 23 image

Fedora 23 was released today. Get it through virt-builder in just two simple commands:

$ virt-builder fedora-23 \
    --root-password password:123456 \
    --size 20G
$ qemu-system-x86_64 -drive file=fedora-23,if=virtio \
    -m 2048


Leave a comment

Filed under Uncategorized

USB Per Port Power Switching

Increasing numbers of single board computers are powered by USB ports using a cable like this:


Wouldn’t it be cool if you could plug these into a USB hub and have the hub individually power up and down the computers? Like a cheap APC power management board.

It turns out you can — with difficulty. The USB standard defines Per-Port Power Switching (PPPS) but unfortunately almost no hub in existence actually supports it. To save a few cents on the BOM, the manufacturers generally leave out the extra power transistors needed to make it work. One guy has modified his USB hub to add the extra part and full marks to him but that’s a lot of effort.

There is one hub which actually supports this: The D-Link DUB-H7, but only the first version (the silver/grey case). In the second version (black case), D-Link too realized their “mistake” and saved on the extra bits.

Armed with this knowledge, I bought one of these from the US (shipping cost 3x the cost of the hub itself).

And it works! Well, for a while.


I used the C hub-ctrl program from this page, and the instructions from here.

# lsusb
Bus 003 Device 005: ID 2001:f103 D-Link Corp. DUB-H7 7-port USB 2.0 hub

Switch off port 7 (right-most port):

# ./hub-ctrl -b 3 -d 5 -P 4 -p 0

Switch on port 7:

# ./hub-ctrl -b 3 -d 5 -P 4 -p 1

The ports are not numbered in the same sequence as the ports on the hub itself. I found by experimentation that the ports correspond as follows. I don’t know if it’ll be the same on every model:

Port hub-ctrl option
1 (left) -P 5
2 -P 6
3 -P 1
4 -P 2
5 -P 7
6 -P 3
7 (right) -P 4

The bigger problem is the hub is fairly unreliable. Switching ports on or off too frequently seems to result in the hub crashing, which appears to only be recoverable by powering the hub off for several seconds.

So I’m not quite there.


Filed under Uncategorized