New home gateway router (part 2) — the Gigabyte Brix

As explained in the previous post I’m building a new home gateway router, and for this I chose the Gigabyte Brix GB-BXBT-2807, the lowest end product in Gigabyte’s Brix range.

It was a bit of a saga to get it working. The main problem was the RAM that I bought was faulty, but I didn’t know that at the time since all you get is a blank display. The Brix uses strange low voltage SO-DIMMs and it’s unlikely you’d have any lying around — I certainly did not. Guessing that it could be the RAM, I ordered more which took another two days, and that worked.

Suggestion for Gigabyte: solder a minimal amount of RAM on the motherboard (eg. 256 MB) so that the Brix at least boots to the EFI shell with no SO-DIMM inserted.

Disassembling and reassembling the Brix is fiddly, but I got there in the end.

Without an operating system installed, it boots into the EFI shell. Booting with a RHEL USB key boots straight into the RHEL installer.

The dual core 64 bit Celeron is pretty nice given that the whole system cost me £137.

lstopo looks like this:

and /proc/cpuinfo like this:

processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 55
model name	: Intel(R) Celeron(R) CPU  N2807  @ 1.58GHz
stepping	: 8
microcode	: 0x811
cpu MHz		: 981.246
cache size	: 1024 KB
physical id	: 0
siblings	: 2
core id		: 0
cpu cores	: 2
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 movbe popcnt tsc_deadline_timer rdrand lahf_lm 3dnowprefetch ida arat epb dtherm tpr_shadow vnmi flexpriority ept vpid tsc_adjust smep erms
bogomips	: 3166.40
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 55
model name	: Intel(R) Celeron(R) CPU  N2807  @ 1.58GHz
stepping	: 8
microcode	: 0x811
cpu MHz		: 1831.644
cache size	: 1024 KB
physical id	: 0
siblings	: 2
core id		: 1
cpu cores	: 2
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 movbe popcnt tsc_deadline_timer rdrand lahf_lm 3dnowprefetch ida arat epb dtherm tpr_shadow vnmi flexpriority ept vpid tsc_adjust smep erms
bogomips	: 3166.40
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:

Edit UEFI varstores

See end of post for an important update

UEFI firmware has a concept of persistent variables. They are used to control the boot order amongst other things. They are stored in non-volatile RAM on the system board, or for virtual machines in a host file.

When a UEFI machine is running you can edit these variables using various tools, such as Peter Jones’s efivar library, or the efibootmgr program.

These programs don’t actually edit the varstore directly. They access the kernel /sys/firmware/efi interface, but even the kernel doesn’t edit the varstore. It just redirects to the UEFI runtime “Variable Services”, so what is really running is UEFI code (possibly proprietary, but more usually from the open source TianoCore project).

So how can you edit varstores offline? The NVRAM file format is peculiar to say the least, and the only real specification is the code that writes it from Tianocore. So somehow you must reuse that code. To make it more complicated, the varstore NVRAM format is tied to the specific firmware that uses it, so varstores used on aarch64 aren’t compatible with those on x86-64, nor are SecureBoot varstores compatible with normal ones.

virt-efivars is an attempt to do that. It’s rather “meta”. You write a small editor program (an example is included), and virt-efivars compiles it into a tiny appliance. You then boot the appliance using qemu + UEFI firmware + varstore combination, the editor program runs and edits the varstore, using the UEFI code.

It works .. at least on aarch64 which is the only convenient machine I have that has virtualized UEFI.

Git repo: http://git.annexia.org/?p=virt-efivars.git;a=summary

Update:

After studying this problem some more, Laszlo Ersek came up with a different and better plan:

1. Boot qemu with only the OVMF code & varstore attached. No OS or appliance.
2. This should drop you into a UEFI shell which is accessible over qemu’s serial port.
3. Send appropriate setvar commands to update the variables. Using expect this should be automatable.

Bring back the BIOS

… all is forgiven.

ARM booting sucks in several very concrete ways:

• As there is no standard display hardware nor standard serial port, it’s essentially impossible for a random firmware or kernel to output debug messages. (Like text mode + 16550 on a PC). The most likely outcome for the poor developer is a black screen of nothingness. This could be easily fixed if ARM allocated a standard serial port address and put it into every shipping ARM core.
• ARM instruction sets aren’t standardized. There’s no core of instructions that will work on any ARM (compare to 8086 on a PC). So there’s no way to write a bootloader that works everywhere. [For privileged instructions — unprivileged instructions are usually backwards compatible, except when they’re not (Thumb), but that doesn’t help when writing a bootloader.]
• There is a standard bootloader of sorts, or at least one which is very frequently used, called U-Boot. That’s good, but everyone and their uncle modifies it so it’s forked all over the place. And make sure you’ve got the exact fork for your hardware, otherwise you’ll brick it.
• U-Boot is generally bloody confusing, uses a binary blob + special tools for boot configuration, is hard to compile on x86, etc.
• U-Boot may not be replaceable on your hardware, or there may be some other, closed firmware underneath it (hello, R-Pi).
• The hardware isn’t discoverable and self-describing (unlike, say, PCI, USB and every recent PC peripheral). Thus you have Device Trees, big files that describe every piece of hardware, must be written specially for each piece of hardware, and must be correct else you’ll be looking at a black screen of death.
• Kernel signing. No standard toolset for signing kernels. Unfortunately this particular strain of madness is now infecting PCs too.

Anyhow, this explains how I’ve gone from 3 working Fedora ARM systems to 0 in the space of two days.

This is also the reason I think EFI is a terrible mistake. Having now used UEFI a lot more, I think it’s a vast improvement on the ARM bootloader zoo.