nbdkit ruby plugin

NBD is a protocol for accessing Block Devices (actual hard disks, and things that look like hard disks). nbdkit is a toolkit for creating NBD servers.

You can now write nbdkit plugins in Ruby.

(So in all that makes: C/C++, Perl, Python, OCaml or Ruby as your choices for nbdkit plugins)

Which foreign function interface is the best?

I’ve written libguestfs language bindings for Perl, Python, Ruby, Java, OCaml, PHP, Haskell, Erlang and C#. But which of these is the best? Which is the easiest? What makes this hard? Grubbing around in the internals of a language reveals mistakes made by the language designers, but what are the worst mistakes?

Note: There is source that goes with this. Download libguestfs-1.13.13.tar.gz and look in the respective directories.

The best

It’s going to be a controversial choice, but in my opinion: C#. You just add some simple annotations to your functions and structs, and you can call into shared libraries (or “DllImport”s as Microsoft insisted on calling them) directly. It’s just about as easy as directly calling C and that is no simple achievement considering how the underlying runtime of C# is very different from C.

Example: a C struct:

[StructLayout (LayoutKind.Sequential)]
public class _int_bool {
  int i;
  int b;
}

The worst

There are two languages in the doghouse: Haskell and PHP. PHP first because their method of binding is just very broken. For example, 64 bit types aren’t possible on a 32 bit platform. It requires a very complex autoconf setup. And the quality of their implementation is very poor verging on broken — it makes me wonder if the rest of PHP can be this bad.

Haskell: even though I’m an experienced functional programmer and have done a fair bit of Haskell programming in the past, the FFI is deeply strange and very poorly documented. I simply could not work out how to return anything other than integers from my functions. You end up with bindings that look like this:

write_file h path content size = do
  r <- withCString path $ \path -> withCString content $ \content -> withForeignPtr h (\p -> c_write_file p path content (fromIntegral size))
  if (r == -1)
    then do
      err <- last_error h
      fail err
    else return ()

The middle tier

There’s not a lot to choose between OCaml, Ruby, Java and Erlang. For all of them: you write bindings in C, there’s good documentation, it’s a bit tedious but basically mechanical, and in 3 out of 4 you’re dealing with a reasonable garbage collector so you have to be aware of GC issues.

Erlang is slightly peculiar because the method I chose (out of many possible) is to write an external process that talks to the Erlang over stdin/stdout. But I can’t fault their documentation, and the rest of it is sensible.

Example: Here is a function binding in OCaml, but with mechanical changes this could be Ruby, Java or Erlang too:

CAMLprim value
ocaml_guestfs_add_drive_ro (value gv, value filenamev)
{
  CAMLparam2 (gv, filenamev);
  CAMLlocal1 (rv);

  guestfs_h *g = Guestfs_val (gv);
  if (g == NULL)
    ocaml_guestfs_raise_closed ("add_drive_ro");

  char *filename = guestfs_safe_strdup (g, String_val (filenamev));
  int r;

  caml_enter_blocking_section ();
  r = guestfs_add_drive_ro (g, filename);
  caml_leave_blocking_section ();
  free (filename);
  if (r == -1)
    ocaml_guestfs_raise_error (g, "add_drive_ro");

  rv = Val_unit;
  CAMLreturn (rv);
}

The ugly

Perl: Get reading. You’d better start with perlxs because Perl uses its own language — C with bizarre macros on top so your code looks like this:

SV *
is_config (g)
      guestfs_h *g;
PREINIT:
      int r;
   CODE:
      r = guestfs_is_config (g);
      if (r == -1)
        croak ("%s", guestfs_last_error (g));
      RETVAL = newSViv (r);
 OUTPUT:
      RETVAL

After that, get familiar with perlguts. Perl has only 3 structures and you’ll be using them a lot. There are some brilliant things about Perl which shouldn’t be overlooked, including POD which libguestfs uses to make effortless manual pages.

Python: Best described as half arsed. Rather like the language itself.

Python, Ruby, Erlang: If your language depends on “int”, “long”, “long long” without defining what those mean, and differing based on your C compiler and platform, then you’ve made a big mistake that will unfortunately dog you throughout the runtime, FFIs and the language itself. It’s better either to define them precisely (like Java) or to just use int32 and int64 (like OCaml).

And finally, reference counting (Perl, Python). It’s tremendously easy to make mistakes that are fiendishly difficult to track down. It’s a poor way to do GC and it indicates to me that the language designer didn’t know any better.

Ruby bindings for Hivex

Hivex is a library for reading and writing Windows Registry “hive” files. New in version 1.3.0 and Fedora 16 is the ability to access the library from Ruby.

As an example, first grab some hive files from a Windows virtual machine. The simplest way is using virt-copy-out:

# virt-copy-out -a win.img \
    'win:c:\windows\system32\config' .
# ls config/
...
SOFTWARE
SYSTEM
...

Using the following Ruby script you can extract and display registry keys from the hive files:

#!/usr/bin/ruby

require 'hivex'

h = Hivex::open("config/SOFTWARE", {})

# Use this instead if you want to make changes:
# h = Hivex::open("config/SOFTWARE", { :write => 1 })

root = h.root()
node = h.node_get_child(root, "Microsoft")
if node.nil? then
  puts "no HKLM\\SOFTWARE\\Microsoft node: Probably not the correct hive"
end

node = h.node_get_child(node, "Windows NT")
node = h.node_get_child(node, "CurrentVersion")
val = h.node_get_value(node, "ProductName")

hash = h.value_value(val)
puts "Windows product name:", hash[:value]

Tip: Call libguestfs from JRuby

You can call libguestfs from JRuby.

JRuby only recently got support for Ruby C extensions, and support is currently quite wobbly. This excellent blog post summarizes the situation.

There are two ways to get access to libguestfs from JRuby. One way would be via the usual Ruby C extension, but as I said above, that’s wobbly and in fact does not work. The second way, which is what we’ll use, is to use the native Java module for libguestfs.

One thing to note is that the Ruby extension and the Java-via-Ruby extension are similar but not quite the same. In fact I found you can copy and paste most Ruby libguestfs code directly, and have it run, but some things need alteration.

Here is the example JRuby libguestfs program:

include Java

Dir["/usr/share/java/libguestfs-*.jar"].each { |jar| require jar }

guestfs = com.redhat.et.libguestfs

g = guestfs.GuestFS().new
version = g.version

printf("libguestfs %d.%d.%d%s\n",
       version.major, version.minor, version.release, version.extra)

# Create an empty, raw format 500 MB disk.
File.open("test.img", "w") {
  |f| f.seek(500*1024*1024); f.write("")
}

# Note that the key must be a string ("format") and
# not an atom (:format).
optargs = java.util.HashMap.new( {"format" => "raw"} )
g.add_drive_opts("test.img", optargs)
g.launch()

g.pvcreate("/dev/sda")
g.vgcreate("VG", ["/dev/sda"]);
g.lvcreate("LV1", "VG", 200);
g.lvcreate("LV2", "VG", 200);

puts "Created test.img"
puts "Now try running: virt-filesystems -a test.img --all --long -h"

You can run it by saving the above to test.rb, and then doing (on Fedora Rawhide or upcoming Fedora 16):

# yum install jruby libguestfs-java-devel
$ jruby test.rb

Tip: Using libguestfs from Perl

I translated the standard libguestfs examples (already available in C/C++, OCaml, Python, Ruby) into Perl.

If you want to call libguestfs from Perl, you have to use Sys::Guestfs.

All 300+ libguestfs API calls are available to all language bindings equally because we generate the bindings.

libguestfs examples in C, OCaml, Python and Ruby

I took two example C programs (one appeared on this blog earlier) and translated them precisely into OCaml, Python and Ruby:

1. C/C++ example
2. OCaml example
3. Python example
4. Ruby example

Tip: Checking if a libguestfs method exists (Ruby)

Use the method respond_to? to check if a method exists before calling it. See also the libguestfs availability API for larger features.

$ irb
irb(main):001:0> require 'guestfs'
=> true
irb(main):002:0> g = Guestfs::create()
=> #<Guestfs::Guestfs:0x7f7b0d88c450>
irb(main):003:0> g.respond_to?( 'set_network' )
=> true
irb(main):004:0> g.respond_to?( 'no_such_method' )
=> false

libguestfs 1.0.0 released, and Ruby bindings

libguestfs 1.0.0 is out, sources and binary RPMs here.

I also added Ruby bindings:

$ irb 
irb(main):002:0> require 'guestfs'
=> true
irb(main):003:0> g = Guestfs::create()
=> #<Guestfs::Guestfs:0x7feab981b630>
irb(main):004:0> g.add_drive("RHEL52PV32.img")
=> nil
irb(main):005:0> g.launch()
=> nil
irb(main):006:0> g.wait_ready()
=> nil
irb(main):007:0> g.lvs()
=> ["/dev/VolGroup00/LogVol00", "/dev/VolGroup00/LogVol01"]
irb(main):008:0> g.pvs()
=> ["/dev/sda2"]
irb(main):009:0> g.vgs()
=> ["VolGroup00"]
irb(main):010:0> g.list_partitions()
=> ["/dev/sda1", "/dev/sda2"]