… all is forgiven.
ARM booting sucks in several very concrete ways:
- As there is no standard display hardware nor standard serial port, it’s essentially impossible for a random firmware or kernel to output debug messages. (Like text mode + 16550 on a PC). The most likely outcome for the poor developer is a black screen of nothingness. This could be easily fixed if ARM allocated a standard serial port address and put it into every shipping ARM core.
- ARM instruction sets aren’t standardized. There’s no core of instructions that will work on any ARM (compare to 8086 on a PC). So there’s no way to write a bootloader that works everywhere. [For privileged instructions — unprivileged instructions are usually backwards compatible, except when they’re not (Thumb), but that doesn’t help when writing a bootloader.]
- There is a standard bootloader of sorts, or at least one which is very frequently used, called U-Boot. That’s good, but everyone and their uncle modifies it so it’s forked all over the place. And make sure you’ve got the exact fork for your hardware, otherwise you’ll brick it.
- U-Boot is generally bloody confusing, uses a binary blob + special tools for boot configuration, is hard to compile on x86, etc.
- U-Boot may not be replaceable on your hardware, or there may be some other, closed firmware underneath it (hello, R-Pi).
- The hardware isn’t discoverable and self-describing (unlike, say, PCI, USB and every recent PC peripheral). Thus you have Device Trees, big files that describe every piece of hardware, must be written specially for each piece of hardware, and must be correct else you’ll be looking at a black screen of death.
- Kernel signing. No standard toolset for signing kernels. Unfortunately this particular strain of madness is now infecting PCs too.
Anyhow, this explains how I’ve gone from 3 working Fedora ARM systems to 0 in the space of two days.
This is also the reason I think EFI is a terrible mistake. Having now used UEFI a lot more, I think it’s a vast improvement on the ARM bootloader zoo.
Richard WM Jones 
Didn’t the RISC folks invent (and standardise) OpenFirmware to solve these problems? And yet somehow, SPARC and POWER are almost dead, and the new hotness ARM has gone *backwards* from PC-BIOS by the sounds of things…
Well obviously ARM is massively successful unlike SPARC. Nevertheless there are some fundamental issues which make it hard to debug. I think my first point (lack of standard serial port) is the most serious one.
Didn’t apple file a claim against anything with letters “ios”?
> This could be easily fixed if ARM allocated a standard serial port address and put it into every shipping ARM core.
Using time machine, or what? How that would help billions existing ARM cores?
That said, you can pray and hope. Look at Cortex-M for example – they broke compatibility with ARM7 and how clean and pure it went out. Did you know that you can bootstrap Coretx-M in pure C, without resorting to Asm? It also has standard – no, net serial port – timer! Can write fully portable RTOS – isn’t it cool?
So, you can pray: “Dear ARM, please break backwards compatibility of Cortex-A cores”. But then you’ll chant:
> ARM instruction sets aren’t standardized. There’s no core of instructions that will work on any ARM
Umm, just adding it to new devices would be sufficient. Why would adding a new port break backwards compatibility?
Just adding a serial port wouldn’t, but who’d stop at that? Example of Cortex-M was given.
Actually, on a second thought, ARM did what you asked for – and more. It’s called CoreSight debugger http://www.arm.com/products/system-ip/coresight/index.php , speaking to the world using very serial JTAG/SWD port. That’s standardized component available from Crotex-M0 and up. For example, randomly checking, Chinese guys didn’t forget to put it into RK3066 SoC. So where’s that JTAG/SWD connectors on your boards? If nowhere, then we know whom to blame – it’s ODM/OEMs who don’t do their jobs right. Plus, community inertia and veil of mystery around JTAG (like, what would be first project to do with Arduino – do JTAG and plug into something? nope, people keep blinking LEDs). (Disclaimer: I never did JTAG in my life, it’s 6 months in my priority queue, and I still didn’t get to it. I feel ashamed.).
Ok, for someone who will google this up: Cortex-Ms have ITM (Instrumentation Trace Macrocell). Roughly, it’s FIFO with serializer, which kinda even allows real-time output of small messages. Output however requires 3rd pin in addition to 2 of SWD. But Cortex-A doesn’t have that. Why? Because you don’t need funky hardware to do that, you can FIFO in software. And rough idea is that when you want to print something, you write that to buffer, when it’s full or flushed, you breakpoint, which causes debug halt recognized by debugger which extracts buffer, voila. It’s called semihosting and could be called well-kept ARM industry secret, except that it’s not secret at all and “has been available for many years”: http://blogs.arm.com/software-enablement/418-semihosting-a-life-saver-during-soc-and-board-bring-up/ .
Great. So why is it the kernel is unable to output anything when it fails to boot? But the same thing all just works on x86?
> Great. So why is it the kernel is unable to output anything when it fails to boot?
Well, I shared my humble hypothesis why – see phrase “community inertia” above. Getting to technical specifics, do you have all relevant kernel options enabled? arch/arm/Kconfig.debug is 25K, has something to offer. DEBUG_ICEDCC and DEBUG_SEMIHOSTING are particular options which would work for ~all ARM CPUs.
> But the same thing all just works on x86?
Does it? To start with, “big” x86 CPUs doesn’t have any builtin UART either, so if you’d put CPU and RAM on a ~2x3cm board (just like good deal of ARM boards are), nothing would work at all. So, guys who put that stuff on boards, make boards really big, to put more stuff there, then charge hundred(s) of dollars for that. Then it works. Correction: it did 20 years ago. Now it doesn’t, because they don’t put serial ports any longer. So, you need to get another board, put into motherboard, for the whole construction to take quite sizable volume, and then it works. But definitely not “just” ;-).
Anyway, that’s not my point. My point is that folks interested in low-level debug on ARM should raise own awareness on how that’s done, and share with other folks interested. Hope I’m adding my 2 cents on both accounts. 😉
Pingback: (Not) getting Fedora 19 on the ODROID XU | Richard WM Jones
Richard now you have the Kernel booting you definitely need one of these in your life:
http://com.odroid.com/sigong/blog/blog_list.php?bid=128
Failing that a few transistors to do the Level shifting to use a “standard” Serial to USB adaptor.
I need one too so I’ll be looking for an EBAY device that does 1.8 v logic
Pingback: Booting Fedora on the Dragonboard | Richard WM Jones
Pingback: Linaro Connect: Jon Masters talking about the importance of standards | Richard WM Jones