Note: This is the Fedora version of this Debian document so full credit must go to Debian and SuSE for assembling the bits.
Qemu (not quite upstream) now has ARM 64 bit emulation. It only does the userspace emulation at the moment, although system emulation is being worked on. This is much faster than the ARM “Foundation Model” (basically ARM’s proprietary emulator).
You will need to clone SuSE’s ARM64 qemu git repository and compile it statically linked (configure it with
--target-list=arm64-linux-user --static). Static glib2 for Fedora is here.
You will also need to download this Fedora 19 arm64 disk image and convert it into a root filesystem (unfortunately this process requires, just temporarily, 14 GB of free disk space — should have used virt-sparsify!!):
mkdir /tmp/arm64 cd /tmp/arm64 tar zxf F19-aarch64-efi.tar.xz virt-tar-out -a ./F19-aarch64-efi/aarch64-efi.img / - | sudo tar xf - rm -rf F19-aarch64-efi
So now what you should have is a statically linked
qemu-arm64 binary (that’s the userspace arm64 emulator), and a root filesystem containing lots of arm64 binaries. To run them we’ll need to set up a systemd binfmt handler, which will notice when we’re about to try to run an arm64 binary and inject our emulator so it’s possible to run it on our host (which is obviously not arm64 since that hardware is unobtainium for most mortals).
Copy the statically linked
qemu-arm64 binary into the chroot, ie:
cp ./arm64-linux-user/qemu-arm64 /tmp/arm64/
Create a binfmt file
/etc/binfmt.d/qemu-arm64.conf with the content below. Note that it refers to the absolute path
/qemu-arm64 which (will be) correct once we chroot into the arm64 filesystem.
# AArch64 binaries. :qemu-arm64:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xb7:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff:/qemu-arm64:OC
$ sudo service systemd-binfmt restart
/proc/sys/fs/binfmt_misc/qemu-arm64 has been created with the expected contents.
Now you’re ready to go!
$ cd /tmp/arm64 $ sudo mount -o bind /dev ./dev $ sudo mount -o bind /dev/pts ./dev/pts $ sudo mount -o bind /proc ./proc $ sudo mount -o bind /sys ./sys $ sudo chroot . # file bin/ls bin/ls: ELF 64-bit LSB executable, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 3.7.0, BuildID[sha1]=0x31bcaf7de1bb3c6a4400983ac31f0dff1bbfc394, stripped # /bin/ls bin dev home lib64 media opt qemu-arm64 run srv tmp var boot etc lib lost+found mnt proc root sbin sys usr # uname -a Linux choo.home.annexia.org 3.8.0 #1 SMP Thu Oct 10 14:11:18 UTC 2013 aarch64 aarch64 aarch64 GNU/Linux
And somehow convince yourself you really are running arm64 binaries!
One problem I had is the supplied chroot is quite minimal and I’m not exactly sure how to install more. But it’s fun to play and a lot faster than the Foundation Model.
# arch aarch64
/etc/yum.repos.d/stage4.repo file in the disk image above is broken. Peter Robinson notes that you can use http://arm.koji.fedoraproject.org/aarch64/stage4/ as a yum repo for installing more packages.
Outside the chroot — when you’re running commands in the chroot — you’ll see qemu interpreting the arm64 binaries in ps output:
$ ps ax | grep qemu 26051 pts/14 S 0:03 /qemu-arm64 /bin/bash -i 28170 pts/14 S 0:00 /qemu-arm64 /bin/su rjones 28171 pts/14 S 0:01 /qemu-arm64 /bin/bash 28226 pts/14 Sl+ 0:03 /qemu-arm64 /bin/git clone git://libvirt.org/libvirt.git 28228 pts/14 S+ 0:12 /qemu-arm64 /usr/libexec/git-core/git index-pack --stdin -v --fix-thin --keep=fetch-pack 28226 on choo.home.annexia.org
7 responses to “How to run aarch64 binaries on an x86-64 host using qemu userspace emulation”
You can skip all the mounting silliness by:
systemd-nspawn -D /tmp/arm64
I agree that way would be better. However some commands fail inside the container:
I’ll see if I can track this one down …
Pingback: Fedora 21 has a working OCaml ARM64 | Richard WM Jones
I’m using this on an F19 host and can’t see the network from within the chroot, which prevents package installation from the repos. Attempting to install packages from outside the chroot with ‘yum –installroot=…’ fails because of db version mismatches (I suspect but haven’t checked that the stage4 ‘rpm’ is pre-F19 or uses a different db version than the F19 x86_64 rpm). Is there a way to gain network access from inside the chroot?
Network works fine for me on F20. Note that qemu emulates the network, I think by thunking system calls and using SLIRP, so various things (like ping) definitely won’t work. Try testing “ssh”, “telnet” etc. For example from the chroot:
You’re right … the inability to do basic network things like ‘ip addr’ was leading me in the wrong direction — the real problem was the resolver configuration. Happily yum installing now!
Pingback: Run aarch64 binary on x86_64 machines | Robin On Linux