libguestfs 1.12.6 for Debian

Thanks to the tireless work of Hilko Bengen, libguestfs 1.12.6 is now available as an official Debian package.

Also, you can compile hivex on Mac OS X and Windows, thanks to Alex Nelson and Gillen Daniel respectively.

New tech toys

A SheevaPlug from New IT (UK), and an old MacBook Pro 17″. I’m going to try to get Fedora for ARM working on the SheevaPlug. The MacBook Pro is 4 years old and was struggling and almost unusably slow under Mac OS X. I put a new hard drive in it and upgraded to Fedora 14, and it’s amazing how fast it has become.

Notes

1. 8GB card from New IT was a dud. Random errors and timeouts when I tried to overwrite it. I’m using the nfsroot configuration until I can get another one.
2. Do not use the kernel suggested in the wiki (uImage-2.6.30-sheevaplug). See this thread. Use the kernels and modules from this site instead.
3. Use usbserial as described here.
4. Don’t use minicom — it always has sucked and always will. Try gtkterm instead.

U-boot (bootloader) voodoo used to get the initial nfsroot Fedora ARM working:

setenv mainlineLinux yes
setenv arcNumber 2097
setenv ipaddr [my static IP]
setenv serverip [my TFTP server IP]
tftpboot 0x6400000 [name of kernel image]
setenv bootargs_root 'root=/dev/nfs rw nfsroot=[NFS server IP]:/mnt/rootfs ip=[static IP]'
setenv bootargs $(console) $(bootargs_root)
bootm 0x6400000

libguestfs now works on Mac OS X

$ LIBGUESTFS_PATH=appliance ./fish/guestfish
Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.

Type: 'help' for help with commands
      'quit' to quit the shell

><fs> add /tmp/test.img
><fs> run
><fs> list-devices
/dev/vda
><fs> list-partitions
/dev/vda1
><fs> vfs-type /dev/vda1
ext3
><fs> mount /dev/vda1 /
><fs> ll /
total 13
drwxr-xr-x  3 root root  1024 Mar 21 21:45 .
dr-xr-xr-x 19 root root     0 Mar 21 17:20 ..
drwx------  2 root root 12288 Mar 21 21:45 lost+found
><fs> touch /hello
><fs> ll /
total 13
drwxr-xr-x  3 root root  1024 Mar 21 22:02 .
dr-xr-xr-x 19 root root     0 Mar 21 17:20 ..
-rw-r--r--  1 root root     0 Mar 21 22:02 hello
drwx------  2 root root 12288 Mar 21 21:45 lost+found
><fs> !uname -a
Darwin koneko.home.annexia.org 9.8.0 Darwin Kernel Version 9.8.0: Wed Jul 15 16:55:01 PDT 2009; root:xnu-1228.15.4~1/RELEASE_I386 i386

We used porting plan 3 from here, in this case running a Fedora appliance. A curious side-effect of this is that libguestfs on Mac OS X can read ext3 partitions and LVM, something which OS X itself wouldn’t normally be able to do. On the downside, it’s kind of slow, because qemu is not accelerated on OS X.

Interested people should look at the lengthy series of patches posted on the mailing list.

Unpack a Mac .dmg installer using guestfish

If you’re not familiar with Mac OS X, it uses a packaging format derived from NextStep where applications are directories and to install an application, you just drag and drop the package directory into the “Applications” folder (or wherever you choose because apps are self-contained and can run from anywhere).

Mac OS X installers use a file format called an “Apple disk image” or *.dmg file, which is a sort of bzip2-compressed filesystem. In the example below, I downloaded the binary installer for the brilliant open source video player VLC, and double-clicked on it, whereupon Mac OS X mounts the packaged filesystem. In the image below, the VLC application looks like a file but in reality is a directory full of application files.

Can we decode and unpack these .dmg files in Linux using libguestfs? Well, not always, but sometimes (because of some limitations in the Linux HFS+ driver described below).

First I should note that this probably doesn’t work with password-protected / encrypted files, but they’re not common for software distribution. DMGExtractor claims it can handle those.

Secondly the image is usually bzip2-compressed, and you have to run bunzip2 on it by hand first:

$ mv Firefox\ 3.5.3.dmg firefox-3.5.3.dmg.bz2
$ bunzip2 firefox-3.5.3.dmg.bz2
bunzip2: firefox-3.5.3.dmg.bz2: trailing garbage after EOF ignored

(Yeah, I don’t know how important that error was either … But it doesn’t seem to affect things.)

Thirdly the file system image is not at the start of the file. You have to find it and strip off some sort of header from the file. Look for the HFS+ superblock in the file:

$ hexdump -C firefox-3.5.3.dmg | grep '  48 2b 00 04'
00008400  48 2b 00 04 00 00 01 00  63 65 72 64 00 00 00 00  |H+......cerd....|
^C

and strip it, less 0x400 bytes:

$ dd if=firefox-3.5.3.dmg of=firefox.img bs=$((0x8400 - 0x400)) skip=1

This should produce an HFS+ filesystem image. Check that:

$ file firefox.img
firefox.img: Macintosh HFS Extended version 4 data last mounted by: 'cerd',
created: Mon Aug 24 10:37:21 2009, last modified: Mon Aug 24 18:37:21 2009,
last backup: Mon Aug 24 18:37:21 2009, last checked: Mon Aug 24 18:37:21 2009,
block size: 2048, number of blocks: 25773, free blocks: 0

Now this can be loaded directly into guestfish:

$ guestfish --ro -a firefox.img -m /dev/sda

Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.

Type: 'help' for help with commands
      'quit' to quit the shell

><fs> ll /
total 180
lrwxrwxr-x  1  502   80     13 Aug 24 13:37   -> /Applications
drwxr-xr-x  1  502   80      7 Aug 24 13:37 .
dr-xr-xr-x 29 root root      0 Oct 25 17:41 ..
-rwxr-xr-x  1  502   80  12292 Aug 24 12:41 .DS_Store
-rw-r--r--  1  502   80 165905 Aug 24 12:41 .VolumeIcon.icns
drwxr-xr-x  1  502   80      3 Aug 24 13:37 .background
drwxr-xr-x  1  502   80      3 Aug 24 13:33 Firefox.app

and in this case, I just wanted to extract the contents to a tarball, so:

><fs> tgz-out / firefox.tgz

$ zcat firefox.tgz | tar tf -
./
./ 
./.background/
./.background/background.png
./.DS_Store
./.VolumeIcon.icns
./Firefox.app/
./Firefox.app/Contents/
./Firefox.app/Contents/Info.plist
./Firefox.app/Contents/MacOS/
./Firefox.app/Contents/MacOS/.autoreg
./Firefox.app/Contents/MacOS/application.ini
./Firefox.app/Contents/MacOS/blocklist.xml
./Firefox.app/Contents/MacOS/browserconfig.properties
./Firefox.app/Contents/MacOS/chrome/
[etc]

“hfs: failed to load catalog file”

It worked with the Firefox installer, but two other installers I tried (for VLC and Disk Inventory X) didn’t go so well. The HFS+ driver produced this error for those:

hfs: failed to load catalog file

which as far as I can see is either a problem that these .dmg files don’t follow the HFS+ [proprietary] standard by missing out the catalog B-tree, or else the HFS+ driver can’t find that B-tree for unknown reasons.