http://b2b.gigabyte.com/products/list.aspx?s=92&p=190&v=1029&ck=102
Gigabyte just announced a bunch of full ARM servers, with between 32 and 96 cores. They are based around the Cavium ThunderX processors that we’ve had at Red Hat for a while so they should run RHEL either out of the box or very soon after release.
Filed under Uncategorized
I finally got UEFI flashed onto the Gigabyte board so now it is SBSA/SBBR compliant [edit: see note at end] and will just work with RHEL. Instructions here: https://lists.centos.org/pipermail/arm-dev/2016-March/001743.html
Here are the boot messages from TianoCore:
TianoCore 1.20.03 UEFI 2.4.0 Jan 26 2016 18:09:04
CPU: APM ARM 64-bit Potenza Rev B0 2400MHz PCP 2400MHz
32 KB ICACHE, 32 KB DCACHE
SOC 2000MHz IOBAXI 400MHz AXI 250MHz AHB 200MHz GFC 125MHz
Board: X-Gene Mp30ar0 Board
Slimpro FW:
Ver: 2.4 (build 01.15.01.00 2015/05/22)
TPC: disable
AVS: support
PMD: 970 mV
SOC: 950 mV
The default boot selection will start in 1 second
Note: A few people have pointed out that the Gigabyte isn’t SBSA compliant because it lacks the right serial port, RTC and WDT. However it still boots RHELSA fine if you add console=ttyS0,115200.
Filed under Uncategorized
IPMI works out of the box. I’m using FreeIPMI to test this (not ipmitool) since FreeIPMI is a lot easier to use.
You need to know that:
admin and the default password is password.Here are the sensors:
$ ipmi-sensors -h 192.168.0.104 -u admin -p password Caching SDR repository information: /home/rjones/.freeipmi/sdr-cache/sdr-cache-moo.192.168.0.104 Caching SDR record 34 of 34 (current record ID 205) ID | Name | Type | Reading | Units | Event 4 | CPU0_TEMP | Temperature | 49.00 | C | 'OK' 9 | DIMM_P0_A0 | Temperature | N/A | C | N/A 10 | DIMM_P0_A1 | Temperature | N/A | C | N/A 12 | DIMM_P0_B0 | Temperature | N/A | C | N/A 13 | DIMM_P0_B1 | Temperature | N/A | C | N/A 15 | DIMM_P0_C0 | Temperature | N/A | C | N/A 16 | DIMM_P0_C1 | Temperature | N/A | C | N/A 18 | DIMM_P0_D0 | Temperature | N/A | C | N/A 19 | DIMM_P0_D1 | Temperature | N/A | C | N/A 59 | P12V | Voltage | 11.83 | V | 'OK' 60 | P5V | Voltage | 5.11 | V | 'OK' 61 | P3V3 | Voltage | 3.33 | V | 'OK' 62 | P5V_STBY | Voltage | 5.13 | V | 'OK' 64 | P_VBAT | Voltage | 3.07 | V | 'OK' 65 | P_VCCP | Voltage | 0.97 | V | 'OK' 66 | P_1V2_HUB | Voltage | 1.20 | V | 'OK' 67 | P_VDDQ_AB | Voltage | 1.50 | V | 'OK' 68 | P_VDDQ_CD | Voltage | 1.50 | V | 'OK' 71 | P_0V9_VDD | Voltage | 0.96 | V | 'OK' 72 | P_1V5_VDD | Voltage | 1.52 | V | 'OK' 73 | P_2V5_VDD | Voltage | 2.50 | V | 'OK' 74 | P_1V8_VDD | Voltage | 1.82 | V | 'OK' 136 | CPU0_FAN | Fan | 4000.00 | RPM | 'OK' 138 | SYS_FAN1 | Fan | N/A | RPM | N/A 139 | SYS_FAN2 | Fan | N/A | RPM | N/A 140 | SYS_FAN3 | Fan | N/A | RPM | N/A 141 | SYS_FAN4 | Fan | N/A | RPM | N/A 190 | CPU0 | Processor | N/A | N/A | 'Processor Presence detected' 202 | MB_TEMP1 | Temperature | 37.00 | C | 'OK' 203 | MB_TEMP2 | Temperature | 31.00 | C | 'OK' 204 | MB_TEMP3 | Temperature | 28.00 | C | 'OK' 205 | SEL | Event Logging Disabled | N/A | N/A | 'OK'
Filed under Uncategorized
Turns out that acpi=off is only needed by the RHEL 7.2 installer kernel. After installation, ACPI works fine. That might be because the installer kernel is older than the current RHEL 7 kernel. dmesg output after the break.
Filed under Uncategorized
eth0 is the top left connector (the upper connector marked “c” in the motherboard manual).
I burned RHELSA RHEL-7.2-20151030.0-Server-aarch64-boot.iso to a USB key and inserted it into the machine.
I had to chain-load Tianocore (UEFI firmware) as described here. This requires that you run a TFTP server. You should also set the u-boot ipaddr and serverip environment variables so that u-boot can contact your TFTP server.
Once in UEFI, do these commands at the UEFI shell:
set MAC0 xx:xx:xx:xx:xx:xx
Then reboot again, go into the UEFI shell again, and:
fs0: cd \EFI\BOOT BOOTAA64
I also had to stop the grub configuration and edit the linux kernel command line, adding console=ttyS0,115200 acpi=off vnc at the end.
acpi=off is needed because the SBBR requires ACPI support, which the version of TianoCore which APM/Gigabyte have provided does not supply (or it’s broken in some way).acpi=off is required to boot the installer, I found that RHELSA itself will actually work with ACPI. So you have to remove acpi=off from the kernel command line after installation.
After that you should see lots of kernel messages, and with luck you’ll arrive in Anaconda and it will print details of the VNC server. Finally, I have a RHELSA installer …
Filed under Uncategorized
This is the Gigabyte MP30-AR0 which now forms the top “layer” of my cluster (I had to retire one of the AMD boards in order to scavenge the RAM for this).
The Gigabyte uses the APM X-gene1, which is an 8 core ARM processor. This is rather old now — I’ve had a Mustang at home with the same processor for a couple of years now and the chip design itself is 3+ years old. It uses Cortex A53 cores. Even APM have a newer X-gene 2. Nevertheless it has two very big advantages over the alternatives:
A particularly annoying thing at the moment is that Gigabyte are claiming this board is SBSA compliant, but then they ship it with u-boot as firmware. However UEFI firmware can be downloaded, see this thread, so it should run RHEL.
Here are the boot messages to the u-boot prompt:
U-Boot 2013.04 (Jun 02 2015 - 10:54:10) REV: 1.15.01-F05 ( uart0 )
CPU0: APM ARM 64-bit Potenza Rev B0 2400MHz PCP 2400MHz
32 KB ICACHE, 32 KB DCACHE
SOC 2000MHz IOBAXI 400MHz AXI 250MHz AHB 200MHz GFC 125MHz
Boot from SPI-NOR
Slimpro FW:
Ver: 2.4 (build 01.15.01.00 2015/05/22)
TPC: disabled
AVS: supported (margin: -0mV)
RST: supported
PWROFF: supported
PMD: 970 mV
SOC: 950 mV
Board: GIGABYTE MP30AR0 - AppliedMicro APM883408-xNA24SPT Customer Board
I2C: ready
DRAM: 32 GiB @ 1600MHz...
SF: Detected MX25L25635F with page size 64 KiB, total 32 MiB
MMC: X-Gene SD/SDIO/eMMC: 0
PCIE0: (RC) link down
PCIE2: (RC) X1 GEN-1 link up
PCIE3: (RC) link down
00:00.0 - 10e8:e004 - Bridge device
01:00.0 - 1a03:1150 - Bridge device
02:00.0 - 1a03:2000 - Display controller
Video: ASPEED VGA Card (1a03, 2000) found @(2:0:0)
Mode: 1024x768x32 48kHz 60Hz
In: serial
Out: vga
Err: serial
CPUs: 11111111
Net: eth0
USB0: scanning bus 0 for devices... XHCI: WARN: Didn't find a matching TT
3 USB Device(s) found
USB1: scanning bus 1 for devices... 2 USB Device(s) found
scanning usb for storage devices... 0 Storage Device(s) found
XHCI: ep 0x1 - rounding interval to 128 microframes
XHCI-ERR: xhci_submit_async_int !
Register 1 keyboards
Hit any key to stop autoboot: 0
Filed under Uncategorized
As explained in the previous post I’m building a new home gateway router, and for this I chose the Gigabyte Brix GB-BXBT-2807, the lowest end product in Gigabyte’s Brix range.
It was a bit of a saga to get it working. The main problem was the RAM that I bought was faulty, but I didn’t know that at the time since all you get is a blank display. The Brix uses strange low voltage SO-DIMMs and it’s unlikely you’d have any lying around — I certainly did not. Guessing that it could be the RAM, I ordered more which took another two days, and that worked.
Suggestion for Gigabyte: solder a minimal amount of RAM on the motherboard (eg. 256 MB) so that the Brix at least boots to the EFI shell with no SO-DIMM inserted.
Disassembling and reassembling the Brix is fiddly, but I got there in the end.
Without an operating system installed, it boots into the EFI shell. Booting with a RHEL USB key boots straight into the RHEL installer.
The dual core 64 bit Celeron is pretty nice given that the whole system cost me £137.
lstopo looks like this:
and /proc/cpuinfo like this:
processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 55 model name : Intel(R) Celeron(R) CPU N2807 @ 1.58GHz stepping : 8 microcode : 0x811 cpu MHz : 981.246 cache size : 1024 KB physical id : 0 siblings : 2 core id : 0 cpu cores : 2 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 movbe popcnt tsc_deadline_timer rdrand lahf_lm 3dnowprefetch ida arat epb dtherm tpr_shadow vnmi flexpriority ept vpid tsc_adjust smep erms bogomips : 3166.40 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 55 model name : Intel(R) Celeron(R) CPU N2807 @ 1.58GHz stepping : 8 microcode : 0x811 cpu MHz : 1831.644 cache size : 1024 KB physical id : 0 siblings : 2 core id : 1 cpu cores : 2 apicid : 2 initial apicid : 2 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 movbe popcnt tsc_deadline_timer rdrand lahf_lm 3dnowprefetch ida arat epb dtherm tpr_shadow vnmi flexpriority ept vpid tsc_adjust smep erms bogomips : 3166.40 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management:
Filed under Uncategorized
For about 12 months I’ve been using the ASUS RT-N16 + OpenWRT as my gateway. But you know what? It sucks — I hate that it’s not a real Linux distro, that it’s difficult to upgrade, that it randomly reboots itself, that it only kinda manages IPv6, that it uses a half-assed packaging system, that I can’t run regular tools, and that it’s insecure.
And since I switched over to VDSL @ 80 Mbps which uses PPPoE (ie. no need for DSL terminated at the gateway), it’s time to switch to running a real distro on a real computer.
The first step is to choose a small form-factor PC, crucially with two ethernet ports. Unfortunately Intel still don’t sell NUCs with two ethernet ports. Also Intel NUCs are really expensive. So I’ve settled for a much cheaper alternative:
Total cost (including tax and delivery): £137.87
Note that I already have a spare 2.5″ SSD, but you would need to add the cost of a SATA HDD/SSD if you don’t have one already.
It’ll be interesting to see how fast the USB 3.0 ethernet adapter is in real life, because it’s the obvious weak spot. However I only need the router to be able to forward at 80 Mbps, and even a Celeron and a weak ethernet adapter surely should be able to handle that.
Filed under Uncategorized
Richard WM Jones 