Is there a music player which doesn’t suck?

$ find /mnt/media -name '*.mp3' -o -name '*.m4a' | wc -l
17763

(and that doesn’t count things like radio programs encoded in mp4 or flv)

I still haven’t found a music player that can deal with this. In particular:

1. ID3 tags are often wrong — don’t use them. Use a database like Shazam to work out what’s really in each file.
2. There are duplicates. I’m never going to be able to fix that. Transparently pick the highest quality file when playing, and don’t show me duplicates ever.
3. Give me an intelligent way to navigate this. I’d like to have the player automatically group them or suggest related music, perhaps by looking for similarities in the audio (and definitely not by using the ID3 tag).
4. Related to the previous point, search should be super simple.
5. When choosing a party playlist, don’t mix in radio programs or podcasts.
6. Some of the titles are not US ASCII. It still seems like this isn’t a completely solved problem (sigh).

It’d be nice if it could stream the music to my squeezebox players (of which I now have two, yay!), but I’m not expecting miracles …

“User Stories”

I’ve noticed recently people writing “User stories”. Examples: GNOME, Xen, Fedora doing something similar.

These are IMHO lazy, unscientific nonsense. For a couple of reasons:

1. Unless you have done usability studies and surveyed real users, you have no idea what users want from your software.
2. There’s no closed loop: You have to evaluate at the end of development how successful you were at meeting the needs of each user in each story. But no one is doing that, so they have no idea if they were successful or not.

I’d say this is yet another example of how software development really needs to grow up and become an actual engineering discipline.

Why is virt-builder written in OCaml?

Docker is written in Go. virt-builder is written in OCaml. Why? (Or as someone at work asked me — apparently seriously — why did you write it in a language which only you can use?)

Virt-builder is a fairly thin wrapper around libguestfs and libguestfs has bindings for a dozen languages, and I’m pretty handy in most programming languages, so it could have been done in Python or C or even Go. In this case there are reasons why OCaml is a much better choice:

• It’s a language I’m familiar with and happy programming in. Never underestimate how much that matters.
• OCaml is strongly typed, helping to eliminate many errors. If it had been written in Python we’d be running into bugs at customer sites that could have been eliminated by the compiler before anything shipped. That doesn’t mean virt-builder is bug free, but if the compiler can help to remove a bug, why not have the compiler do that?
• Virt-builder has to be fast, and OCaml is fast. Python is fucking slow.
• I had some C code for doing parallel xzcat and with OCaml I can just link the C code and the OCaml code together directly into a single native binary. Literally you just mix C object files and OCaml object files together on the linker command line. Doing this in, say, Perl/Python/Ruby would be far more hassle. We would have ended up with either a slow interpreted implementation, or having to ship a separate .so file and have the virt-builder program find it and dynamically load it. Ugh.
• There was a little bit of common code used by another utility called virt-sysprep which started out as a shell script but is now also written in OCaml. Virt-sysprep regularly gets outside contributions, despite being written in OCaml. I could have written the small amount of common code in C to get around this, but every little helps.

Is OCaml a language that only I can understand? Judge for yourself by looking at the source code. I think if you cannot understand that enough to at least make small changes, you should hand in your programmer’s card at the door.

Edit: Hacker News discussion of this article.

Things I learned

It’s impossible to post a bottle of vodka (legally) from the UK to the Czech Republic.

Two of the many courier companies I contacted insisted that spirits are illegal in CZ. There’s conflicting information on whether this is true or not. Or, more usually, just won’t ship alcohol as a matter of policy.

In any case, no one at all will ship anything by air which has an alcohol content over 25%, even though this doesn’t seem to be a problem on passenger jets.

Bring back the BIOS

… all is forgiven.

ARM booting sucks in several very concrete ways:

• As there is no standard display hardware nor standard serial port, it’s essentially impossible for a random firmware or kernel to output debug messages. (Like text mode + 16550 on a PC). The most likely outcome for the poor developer is a black screen of nothingness. This could be easily fixed if ARM allocated a standard serial port address and put it into every shipping ARM core.
• ARM instruction sets aren’t standardized. There’s no core of instructions that will work on any ARM (compare to 8086 on a PC). So there’s no way to write a bootloader that works everywhere. [For privileged instructions — unprivileged instructions are usually backwards compatible, except when they’re not (Thumb), but that doesn’t help when writing a bootloader.]
• There is a standard bootloader of sorts, or at least one which is very frequently used, called U-Boot. That’s good, but everyone and their uncle modifies it so it’s forked all over the place. And make sure you’ve got the exact fork for your hardware, otherwise you’ll brick it.
• U-Boot is generally bloody confusing, uses a binary blob + special tools for boot configuration, is hard to compile on x86, etc.
• U-Boot may not be replaceable on your hardware, or there may be some other, closed firmware underneath it (hello, R-Pi).
• The hardware isn’t discoverable and self-describing (unlike, say, PCI, USB and every recent PC peripheral). Thus you have Device Trees, big files that describe every piece of hardware, must be written specially for each piece of hardware, and must be correct else you’ll be looking at a black screen of death.
• Kernel signing. No standard toolset for signing kernels. Unfortunately this particular strain of madness is now infecting PCs too.

Anyhow, this explains how I’ve gone from 3 working Fedora ARM systems to 0 in the space of two days.

This is also the reason I think EFI is a terrible mistake. Having now used UEFI a lot more, I think it’s a vast improvement on the ARM bootloader zoo.

Opted out of the mobile phone filter

I opted out (actually, a while back) from the intrusive mobile phone filter that affects all mobile phones in the UK.

You didn’t ask, but here are some thoughts on the “porn filter” for landline connections:

It will require smaller ISPs to make infrastructure investments, inevitably making them less price-competitive and some will go out of business. This is the exact opposite of what the UK needs right now, which is more competition and faster broadband speeds. There’s not much more I could add to what the Open Rights Group have already written on this topic.

It’s designed to distract attention from the economy and appease the Daily Mail. It’s certainly working to distract attention from the dismal state of the economy — look at all the ink that’s been spilled on this issue — so well done DC!

Appeasement, on the other hand, never works. The Daily Mail exists to sell newspapers, and so “campaigning” (making up absurd stories to sell newspapers) is what they do. They won’t stop here. It would be better for the Tories to sideline and belittle the Daily Mail, except that they won’t do that because Daily Mail readers are their core voters. For people who don’t vote Tory, it just looks spineless, so I can’t see how this can be part of a long-term strategy to win people over.

If they wanted to do something useful for the internet, how about hardening critical systems and cutting fraud? Instead, they’re cutting funding for this.

Golang bindings for libguestfs

libguestfs ≥ 1.23.7 now has almost completely functional golang bindings.

So what (you are probably not asking) did I like and dislike about Go? There are some good points:

• Very fast compilation
• Interoperability with C
• Compiles to native code
• Appears to have proper garbage collection

Unfortunately I think that, like Vala, it’s a bit of a missed opportunity to fill the C replacement niche. It looks as if the language designers have never used a functional language, or if they did, they didn’t “get” it. Some of the real downsides include:

• No type inference. Because obviously it’s 2013 and I want to write types everywhere.
• Bondage-and-discipline, but in odd ways and to no apparent benefit. Like what’s the point of all the odd rules around := vs =, and what types you can and can’t assign and pass to functions? And why do you have to declare imports, when the compiler could work them out for you (it’ll even moan if you import something which is not used!)
• Hello, world is about 8 lines of code. Camel case! Java rang, wants its boilerplate back.
• No breakthrough on error handling.
• The whole design of GOROOT/GOPATH is completely broken. It’s actually worse than Java’s broken CLASSPATH crap which is some kind of achievement, I guess.

They refuse to add exceptions (probably a good point), but the alternative is C-style error checking on every other line. It’s not even enforced error checking, so bad programmers will still be able to write bad code. And there are good ways to handle errors, eg. process-based transactions, Erlang-style, etc. But that seems to have passed them by.

Oh well, there’s still room for the breakthrough C replacement language.

And so begins Summertime absurdity

I have no objection if you want to get up an hour earlier. I don’t even object if you think it makes the daylight longer / gives you more sunlight / or whatever silly reasoning you have. Just don’t make me do it too, ‘m kay?

My rant about Haskell

I spent the last couple of days trying to get the libguestfs Haskell bindings in a better shape, and by and large they are now mostly working for simple cases.

But … how I’ve come to dislike Haskell in the process.

I should say this is not just because I’m a big fan of OCaml and other ML derivative languages, ie. fast and useful functional programming that’s very practical. There are some real problems with Haskell which make it less than useful as a real programming language.

Significant whitespace. It’s not just very difficult to understand how the whitespace works (far more so than Python, where it’s merely annoying), but it also makes it almost near impossible to automatically generate Haskell code, which is what we do in libguestfs.

The IO monad. Most Haskell examples use the IO monad, which serializes everything, making the code the same as more ordinary languages. The disadvantage is that monads are obscure and hard to understand. The advantage is .. unclear: your code is all still serialized, mostly, as well as being slow because of the overhead, so it’s not clear what the point is.

Unexpressive FFI. After dealing with a lot of FFIs I think I’m qualified to talk about this one. Haskell’s is terrible: The documentation is obscure verging on bad. The examples are rare (for anything that’s more complex than calling “sin”). There’s a great deal of brokenness in major features, eg. passing or returning structures. A lot of stuff is simply not possible without delving into the depths of compilers. It would have been much better to define a C API and write FFIs in C.

Laziness .. should not be the default. It’s not how any real computers work, or have ever worked, or are likely to work in the future.

Lack of optional/labelled args. Everyone else has them. Haskell has a huge hack. (If you try to implement this huge hack in reality you’ll see it’s not practical if you have a large number of functions that want optional args).

Also I get the impression from reading online that Haskell is widely studied and often pimped, but not used very much in reality.

Why is gettext not an ordinary library?

Not being a regular library causes no end of constant build problems.

Like this crap because we did “gettextize” without doing “autoconf” (or vice versa):

make[2]: Entering directory `/builddir/build/BUILD/libguestfs-1.14.7/po'
*** error: gettext infrastructure mismatch: using a Makefile.in.in from gettext version 0.17 but the autoconf macros are from gettext version 0.18

I just don’t get why gettext can’t be a regular, ordinary, plain library so we don’t have to constantly suffer from this sort of thing. There is surely no other library that needs to rewrite your entire build system.