On the awesomeness of ocaml-bitstring

I used bitstring to reverse engineer the Windows registry “hive” format. I know that bitstring is my own program, but coming back to it two years after I wrote it and using it again for this, I really think this is a brilliant tool. (Bitstring wasn’t my idea — it was inspired by the bitstring manipulation feature in Erlang).

C is supposed to be a good natural programming language for dealing with bits and bytes, right? The ocaml-bitstring program, which analyzes hive files in far more detail than the C program, is half the size and just as fast.

As an example, here’s how we load the hive file and analyze the first part of the header:

let bits = bitstring_of_file filename

(* Split into header + data at the 4KB boundary. *)
let header, data =
  takebits (4096 * 8 ) bits, dropbits (4096 * 8 ) bits

let () =
  bitmatch header with
  { "regf" : 4*8 : string;
    seq1 : 4*8 : littleendian;
    seq2 : 4*8 : littleendian, check (seq1 = seq2);
    last_modified : 64
      : littleendian, bind (nt_to_time_t last_modified);
    1_l (* major *) : 4*8 : littleendian;
    minor : 4*8 : littleendian } ->
      (* ... *)

The bitmatch statement elegantly matches the file. It rejects the file if the first four bytes aren’t “regf” (the file magic number) or if the major version number is not 1. It then unpacks the following fields, converting from the file’s littleendian ordering to host ordering, converting the NT timestamp into a time_t and so on.

Although not shown there, bitstring will also work just fine on arbitrary bit boundaries, albeit more slowly because the generated code is able to make fewer optimizations.

Even though the Windows hive file format is moronic, I successfully used bitstring to reverse engineer it in about 3 days, with some help from the contradictory and often incorrect public documentation out there.


Filed under Uncategorized

5 responses to “On the awesomeness of ocaml-bitstring

  1. bluestorm

    Agreed. I was at JFLA [1] a few days ago, and heard a lot of praise about bitstring there. Your tool is appreciated.

    [1] http://jfla.inria.fr/2010/

  2. lbook

    Thanks. I would be very interested in your code, both as an example of bitstring use and as a tool for dealing with Windows. And I’m sure I’m not the only one. Since you’ve already done the work, would you consider sharing it?

  3. Sigh, if only OCaml’s metaprogramming facilities weren’t so obtuse — https://github.com/deepfire/cl-io-mod/blob/master/mod.lisp.
    Disclaimer: I’m the author.

  4. Eric Blake

    Erlang paper link has gone stale, looks like this one works now: http://www.erlang.se/euc/07/papers/1700Gustafsson.pdf

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.