Gigabyte just announced a bunch of full ARM servers, with between 32 and 96 cores. They are based around the Cavium ThunderX processors that we’ve had at Red Hat for a while so they should run RHEL either out of the box or very soon after release.
Tag Archives: gigabyte
I finally got UEFI flashed onto the Gigabyte board so now it is SBSA/SBBR compliant [edit: see note at end] and will just work with RHEL. Instructions here: https://lists.centos.org/pipermail/arm-dev/2016-March/001743.html
Here are the boot messages from TianoCore:
TianoCore 1.20.03 UEFI 2.4.0 Jan 26 2016 18:09:04 CPU: APM ARM 64-bit Potenza Rev B0 2400MHz PCP 2400MHz 32 KB ICACHE, 32 KB DCACHE SOC 2000MHz IOBAXI 400MHz AXI 250MHz AHB 200MHz GFC 125MHz Board: X-Gene Mp30ar0 Board Slimpro FW: Ver: 2.4 (build 01.15.01.00 2015/05/22) TPC: disable AVS: support PMD: 970 mV SOC: 950 mV The default boot selection will start in 1 second
Note: A few people have pointed out that the Gigabyte isn’t SBSA compliant because it lacks the right serial port, RTC and WDT. However it still boots RHELSA fine if you add
IPMI works out of the box. I’m using FreeIPMI to test this (not ipmitool) since FreeIPMI is a lot easier to use.
You need to know that:
- The default user name is
adminand the default password is
- You have to use the dedicated management interface, marked “f” in the software reference guide (the ethernet port above the two USB sockets).
Here are the sensors:
$ ipmi-sensors -h 192.168.0.104 -u admin -p password Caching SDR repository information: /home/rjones/.freeipmi/sdr-cache/sdr-cache-moo.192.168.0.104 Caching SDR record 34 of 34 (current record ID 205) ID | Name | Type | Reading | Units | Event 4 | CPU0_TEMP | Temperature | 49.00 | C | 'OK' 9 | DIMM_P0_A0 | Temperature | N/A | C | N/A 10 | DIMM_P0_A1 | Temperature | N/A | C | N/A 12 | DIMM_P0_B0 | Temperature | N/A | C | N/A 13 | DIMM_P0_B1 | Temperature | N/A | C | N/A 15 | DIMM_P0_C0 | Temperature | N/A | C | N/A 16 | DIMM_P0_C1 | Temperature | N/A | C | N/A 18 | DIMM_P0_D0 | Temperature | N/A | C | N/A 19 | DIMM_P0_D1 | Temperature | N/A | C | N/A 59 | P12V | Voltage | 11.83 | V | 'OK' 60 | P5V | Voltage | 5.11 | V | 'OK' 61 | P3V3 | Voltage | 3.33 | V | 'OK' 62 | P5V_STBY | Voltage | 5.13 | V | 'OK' 64 | P_VBAT | Voltage | 3.07 | V | 'OK' 65 | P_VCCP | Voltage | 0.97 | V | 'OK' 66 | P_1V2_HUB | Voltage | 1.20 | V | 'OK' 67 | P_VDDQ_AB | Voltage | 1.50 | V | 'OK' 68 | P_VDDQ_CD | Voltage | 1.50 | V | 'OK' 71 | P_0V9_VDD | Voltage | 0.96 | V | 'OK' 72 | P_1V5_VDD | Voltage | 1.52 | V | 'OK' 73 | P_2V5_VDD | Voltage | 2.50 | V | 'OK' 74 | P_1V8_VDD | Voltage | 1.82 | V | 'OK' 136 | CPU0_FAN | Fan | 4000.00 | RPM | 'OK' 138 | SYS_FAN1 | Fan | N/A | RPM | N/A 139 | SYS_FAN2 | Fan | N/A | RPM | N/A 140 | SYS_FAN3 | Fan | N/A | RPM | N/A 141 | SYS_FAN4 | Fan | N/A | RPM | N/A 190 | CPU0 | Processor | N/A | N/A | 'Processor Presence detected' 202 | MB_TEMP1 | Temperature | 37.00 | C | 'OK' 203 | MB_TEMP2 | Temperature | 31.00 | C | 'OK' 204 | MB_TEMP3 | Temperature | 28.00 | C | 'OK' 205 | SEL | Event Logging Disabled | N/A | N/A | 'OK'
Turns out that
acpi=off is only needed by the RHEL 7.2 installer kernel. After installation, ACPI works fine. That might be because the installer kernel is older than the current RHEL 7 kernel. dmesg output after the break.
eth0 is the top left connector (the upper connector marked “c” in the motherboard manual).
I burned RHELSA
RHEL-7.2-20151030.0-Server-aarch64-boot.iso to a USB key and inserted it into the machine.
I had to chain-load Tianocore (UEFI firmware) as described here. This requires that you run a TFTP server. You should also set the u-boot
serverip environment variables so that u-boot can contact your TFTP server.
Once in UEFI, do these commands at the UEFI shell:
set MAC0 xx:xx:xx:xx:xx:xx
Then reboot again, go into the UEFI shell again, and:
fs0: cd \EFI\BOOT BOOTAA64
I also had to stop the grub configuration and edit the linux kernel command line, adding
console=ttyS0,115200 acpi=off vnc at the end.
acpi=off is needed because the SBBR requires ACPI support, which the version of TianoCore which APM/Gigabyte have provided does not supply (or it’s broken in some way).
acpi=off is required to boot the installer, I found that RHELSA itself will actually work with ACPI. So you have to remove
acpi=off from the kernel command line after installation.
After that you should see lots of kernel messages, and with luck you’ll arrive in Anaconda and it will print details of the VNC server. Finally, I have a RHELSA installer …
The Gigabyte uses the APM X-gene1, which is an 8 core ARM processor. This is rather old now — I’ve had a Mustang at home with the same processor for a couple of years now and the chip design itself is 3+ years old. It uses Cortex A53 cores. Even APM have a newer X-gene 2. Nevertheless it has two very big advantages over the alternatives:
- You can actually buy it. I had this one shipped overnight from UK supplier Xcase.
- It’s (sort of — see below) server class hardware, unlike the very cheap but ultimately crap development boards based on phone chips.
A particularly annoying thing at the moment is that Gigabyte are claiming this board is SBSA compliant, but then they ship it with u-boot as firmware. However UEFI firmware can be downloaded, see this thread, so it should run RHEL.
Here are the boot messages to the u-boot prompt:
U-Boot 2013.04 (Jun 02 2015 - 10:54:10) REV: 1.15.01-F05 ( uart0 ) CPU0: APM ARM 64-bit Potenza Rev B0 2400MHz PCP 2400MHz 32 KB ICACHE, 32 KB DCACHE SOC 2000MHz IOBAXI 400MHz AXI 250MHz AHB 200MHz GFC 125MHz Boot from SPI-NOR Slimpro FW: Ver: 2.4 (build 01.15.01.00 2015/05/22) TPC: disabled AVS: supported (margin: -0mV) RST: supported PWROFF: supported PMD: 970 mV SOC: 950 mV Board: GIGABYTE MP30AR0 - AppliedMicro APM883408-xNA24SPT Customer Board I2C: ready DRAM: 32 GiB @ 1600MHz... SF: Detected MX25L25635F with page size 64 KiB, total 32 MiB MMC: X-Gene SD/SDIO/eMMC: 0 PCIE0: (RC) link down PCIE2: (RC) X1 GEN-1 link up PCIE3: (RC) link down 00:00.0 - 10e8:e004 - Bridge device 01:00.0 - 1a03:1150 - Bridge device 02:00.0 - 1a03:2000 - Display controller Video: ASPEED VGA Card (1a03, 2000) found @(2:0:0) Mode: 1024x768x32 48kHz 60Hz In: serial Out: vga Err: serial CPUs: 11111111 Net: eth0 USB0: scanning bus 0 for devices... XHCI: WARN: Didn't find a matching TT 3 USB Device(s) found USB1: scanning bus 1 for devices... 2 USB Device(s) found scanning usb for storage devices... 0 Storage Device(s) found XHCI: ep 0x1 - rounding interval to 128 microframes XHCI-ERR: xhci_submit_async_int ! Register 1 keyboards Hit any key to stop autoboot: 0