Tag Archives: libvirt

The Facebook Platform

“The rapidness of web distribution has made older development practices seem quaint and antiquated. But something that’s not antiquated, or shouldn’t be, is providing a service that does what claims, that provides more value than it takes back, and that earnestly cares about the way it gets used”

I don’t often link to other blog posts, but this one is excellent. The Facebook Platform sounds a lot like the Google AdWords API that I used to have the misfortune to write software for. A sociopathic API, constantly changing, that didn’t care about the developers that had to use it. How many of those exist also in the open source world.

I think the best thing libvirt did was to offer a stable API and ABI from day 1. Programs written to a published API should rarely (ideally, never) need to be changed.

Leave a comment

Filed under Uncategorized

Reminder: I’m speaking next week

At the CentOS Dojo in Aldershot, on Friday 12th July. Tickets are £15 per person (don’t worry, that’s not just for me, there are lots of other speakers!), and there’s beer and pig roast in the evening included in the price.

Ask me anything you want about virtualization. I might even be able to answer.

Leave a comment

Filed under Uncategorized

New project: nbdkit, liberally licensed NBD server with a plugin API

Last week I started a new project: nbdkit. This is a toolkit for creating NBD servers. The key features are:

  1. Multithreaded NBD server written in C with good performance.
  2. Well-documented, simple plugin API with a stable ABI guarantee. Let’s you export “unconventional” block devices easily.
  3. Liberal license (BSD) allows nbdkit to be linked to proprietary libraries or included in proprietary code.

There are of course many NBD servers already, such as the original nbd project, qemu-nbd and jnbds.

There are also a handful of servers specialized for particular disk sources. A good example of that is this OpenStack Swift server. But you shouldn’t have to write a whole new server just to export a new disk type.

nbdkit hopefully offers a unique contribution to this field because it’s a general server with a plugin architecture, offering a stable ABI and a liberal license so you can link it to proprietary code (say hello, VDDK).

The motivation for this is to make many more data sources available to libguestfs. Especially I want to write plugins for libvirt, VDDK and some OpenStack sources.


Filed under Uncategorized

CentOS Dojo and Barbecue (UK)

It looks like I might be doing a short talk at the CentOS Dojo and Barbecue at Aldershot, UK, Friday 12th July 2013.

It’ll probably be about scripting/programming libvirt and the virt tools, but mainly it’ll be a chance for Q&A about any virtualization topic in RHEL / CentOS.

Also they have a BBQ — with beer! Sadly since I’m driving there I won’t be able to drink any of the beer.

(Thanks Karanbir Singh, Justin Clift)

Leave a comment

Filed under Uncategorized

Fedora 19 virtualization test day 2013-05-28

Put it in your calendars .. May 28th is Fedora 19 virtualization test day.

New features include nested virtualization on Intel, new Boxes, new libosinfo, new qemu, KMS-based spice driver, live storage migration and virtio RNG.

Every day is libguestfs test day. Just follow the instructions here.


Filed under Uncategorized

More static analysis with CIL

Years ago I played around with CIL to analyze libvirt. More recently Dan used CIL to analyze libvirt’s locking code.

We didn’t get so far either time, but I’ve been taking a deeper look at CIL in an attempt to verify error handling in libguestfs.

Here is my partly working code so far.

 * Analyse libguestfs APIs to find error overwriting.
 * Copyright (C) 2008-2013 Red Hat, Inc.
 * This library is free software; you can redistribute it and/or
 * modify it under the terms of the GNU Lesser General Public
 * License as published by the Free Software Foundation; either
 * version 2.1 of the License, or (at your option) any later version.
 * This library is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * Lesser General Public License for more details.
 * You should have received a copy of the GNU Lesser General Public
 * License along with this library.  If not, see
 * <http://www.gnu.org/licenses/>.
 * Author: Daniel P. Berrange <berrange@redhat.com>
 * Author: Richard W.M. Jones <rjones@redhat.com>

open Unix
open Printf

open Cil

let debug = ref false

(* Set of ints. *)
module IntSet = Set.Make (struct type t = int let compare = compare end)

(* A module for storing any set (unordered list) of functions. *)
module FunctionSet = Set.Make (
    type t = varinfo
    let compare v1 v2 = compare v1.vid v2.vid

(* Directed graph of functions.
 * Function = a node in the graph
 * FunctionDigraph = the directed graph
 * FunctionPathChecker = path checker module using Dijkstra's algorithm
module Function =
  type t = varinfo
  let compare f1 f2 = compare f1.vid f2.vid
  let hash f = Hashtbl.hash f.vid
  let equal f1 f2 = f1.vid = f2.vid
module FunctionDigraph = Graph.Imperative.Digraph.Concrete (Function)
module FunctionPathChecker = Graph.Path.Check (FunctionDigraph)

(* Module used to analyze the paths through each function. *)
module ErrorCounter =
  let name = "ErrorCounter"
  let debug = debug

  (* Our current state is very simple, just the number of error
   * function calls did encountered up to this statement.
  type t = int

  let copy errcalls = errcalls

  (* Start data for each statement. *)
  let stmtStartData = Inthash.create 97

  let printable errcalls = sprintf "(errcalls=%d)" errcalls

  let pretty () t = Pretty.text (printable t)

  let computeFirstPredecessor stmt x = x (* XXX??? *)

  let combinePredecessors stmt ~old:old_t new_t =
    if old_t = new_t then None
    else Some new_t

  (* This will be initialized after we have calculated the set of all
   * functions which can call an error function, in main() below.
  let error_functions_set = ref FunctionSet.empty

  (* Handle a Cil.Instr. *)
  let doInstr instr _ =
    match instr with
    (* A call to an error function. *)
    | Call (_, Lval (Var callee, _), _, _)
        when FunctionSet.mem callee !error_functions_set ->
      Dataflow.Post (fun errcalls -> errcalls+1)

    | _ -> Dataflow.Default

  (* Handle a Cil.Stmt. *)
  let doStmt _ _ = Dataflow.SDefault

  (* Handle a Cil.Guard. *)
  let doGuard _ _ = Dataflow.GDefault

  (* Filter statements we've seen already to prevent loops. *)
  let filter_set = ref IntSet.empty
  let filterStmt { sid = sid } =
    if IntSet.mem sid !filter_set then false
    else (
      filter_set := IntSet.add sid !filter_set;

  (* Initialize the module before each function that we examine. *)
  let init stmts =
    filter_set := IntSet.empty;
    Inthash.clear stmtStartData;
    (* Add the initial statement(s) to the hash. *)
    List.iter (fun stmt -> Inthash.add stmtStartData stmt.sid 0) stmts

module ForwardsErrorCounter = Dataflow.ForwardsDataFlow (ErrorCounter)

(* The always useful filter + map function. *)
let rec filter_map f = function
  | [] -> []
  | x :: xs ->
      match f x with
      | Some y -> y :: filter_map f xs
      | None -> filter_map f xs

let rec main () =
  (* Read the list of input C files. *)
  let files =
    let chan = open_process_in "find src -name '*.i' | sort" in
    let files = input_chan chan in
    if close_process_in chan <> WEXITED 0 then
      failwith "failed to read input list of files";
    if files = [] then
      failwith "no input files; is the program running from the top directory? did you compile with make -C src CFLAGS=\"-save-temps\"?";
    files in

  (* Load and parse each input file. *)
  let files =
    List.map (
      fun filename ->
        printf "loading %s\n%!" filename;
        Frontc.parse filename ()
    ) files in

  (* Merge the files. *)
  printf "merging files\n%!";
  let sourcecode = Mergecil.merge files "libguestfs" in

  (* CFG analysis. *)
  printf "computing control flow\n%!";
  Cfg.computeFileCFG sourcecode;

  let functions =
    filter_map (function GFun (f, loc) -> Some (f, loc) | _ -> None)
      sourcecode.globals in

  (* Examine which functions directly call which other functions. *)
  printf "computing call graph\n%!";
  let call_graph = make_call_graph functions in
  FunctionDigraph.iter_edges (
    fun caller callee ->
      printf "%s calls %s\n" caller.vname callee.vname
  ) call_graph;

  (* The libguestfs error functions.  These are global function names,
   * but to be any use to us we have to look these up in the list of
   * all global functions (ie. 'functions') and turn them into the
   * corresponding varinfo structures.
  let error_function_names = [ "guestfs_error_errno";
                               "guestfs_perrorf" ] in

  let find_function name =
    try List.find (fun ({ svar = { vname = n }}, _) -> n = name) functions
    with Not_found -> failwith ("function '" ^ name ^ "' does not exist")
  let error_function_names = List.map (
    fun f -> (fst (find_function f)).svar
  ) error_function_names in

  (* Get a list of functions that might (directly or indirectly) call
   * one of the error functions.
  let error_functions, non_error_functions =
    functions_which_call call_graph error_function_names functions in

  List.iter (
    fun f -> printf "%s can call an error function\n" f.vname
  ) error_functions;

  List.iter (
    fun f -> printf "%s can NOT call an error function\n" f.vname
  ) non_error_functions;

  (* Save the list of error functions in a global set for fast lookups. *)
  let set =
    List.fold_left (
      fun set elt -> FunctionSet.add elt set
    ) FunctionSet.empty error_functions in
  ErrorCounter.error_functions_set := set;

  (* Analyze each top-level function to ensure it calls an error
   * function exactly once on error paths, and never on normal return
   * paths.
  printf "analyzing correctness of error paths\n%!";
  List.iter compute_error_paths functions;


(* Make a directed graph of which functions directly call which other
 * functions.
and make_call_graph functions =
  let graph = FunctionDigraph.create () in

  List.iter (
    fun ({ svar = caller; sallstmts = sallstmts }, _) ->
      (* Evaluate which other functions 'caller' calls.  First pull
       * out every 'Call' instruction anywhere in the function.
      let insns =  List.concat (
        filter_map (
          | { skind = Instr insns } -> Some insns
          | _ -> None
        ) sallstmts
      ) in
      let calls = List.filter (function Call _ -> true | _ -> false) insns in
      (* Then examine what function is being called at each place. *)
      let callees = filter_map (
        | Call (_, Lval (Var callee, _), _, _) -> Some callee
        | _ -> None
      ) calls in

      List.iter (
        fun callee ->
          FunctionDigraph.add_edge graph caller callee
      ) callees
  ) functions;


(* [functions_which_call g endpoints functions] partitions the
 * [functions] list, returning those functions that call directly or
 * indirectly one of the functions in [endpoints], and a separate list
 * of functions which do not.  [g] is the direct call graph.
and functions_which_call g endpoints functions =
  let functions = List.map (fun ({ svar = svar }, _) -> svar) functions in

  let checker = FunctionPathChecker.create g in
  List.partition (
    fun f ->
      (* Does a path exist from f to any of the endpoints? *)
      List.exists (
        fun endpoint ->
          try FunctionPathChecker.check_path checker f endpoint
          (* It appears safe to ignore this exception.  It seems to
           * mean that this function is in a part of the graph which
           * is completely disconnected from the other part of the graph
           * that contains the endpoint.
          | Invalid_argument "[ocamlgraph] iter_succ" -> false
      ) endpoints
  ) functions

and compute_error_paths ({ svar = svar } as f, loc) =
  (*ErrorCounter.debug := true;*)

  (* Find the initial statement in this function (assumes that the
   * function can only be entered in one place, which is normal for C
   * functions).
  let initial_stmts =
    match f.sbody.bstmts with
    | [] -> []
    | first::_ -> [first] in

  (* Initialize ErrorCounter. *)
  ErrorCounter.init initial_stmts;

  (* Compute the error counters along paths through the function. *)
  ForwardsErrorCounter.compute initial_stmts;

  (* Process all Return statements in this function. *)
  List.iter (
    fun stmt ->
        let errcalls = Inthash.find ErrorCounter.stmtStartData stmt.sid in

        match stmt with
        (* return -1; *)
        | { skind = Return (Some i, loc) } when is_literal_minus_one i ->
          if errcalls = 0 then
            printf "%s:%d: %s: may return an error code without calling error/perrorf\n"
              loc.file loc.line svar.vname
          else if errcalls > 1 then
            printf "%s:%d: %s: may call error/perrorf %d times (more than once) along an error path\n"
          loc.file loc.line svar.vname errcalls

        (* return 0; *)
        | { skind = Return (Some i, loc) } when is_literal_zero i ->
          if errcalls >= 1 then
            printf "%s:%d: %s: may call error/perrorf along a non-error return path\n"
              loc.file loc.line svar.vname

        (* return; (void return) *)
        | { skind = Return (None, loc) } ->
          if errcalls >= 1 then
            printf "%s:%d: %s: may call error/perrorf and return void\n"
              loc.file loc.line svar.vname

        | _ -> ()

        Not_found ->
          printf "%s:%d: %s: may contain unreachable code\n"
            loc.file loc.line svar.vname
  ) f.sallstmts

(* Some convenience CIL matching functions. *)
and is_literal_minus_one = function
  | Const (CInt64 (-1L, _, _)) -> true
  | _ -> false

and is_literal_zero = function
  | Const (CInt64 (0L, _, _)) -> true
  | _ -> false

(* Convenient routine to load the contents of a channel into a list of
 * strings.
and input_chan chan =
  let lines = ref [] in
  try while true; do lines := input_line chan :: !lines done; []
  with End_of_file -> List.rev !lines

and input_file filename =
  let chan = open_in filename in
  let r = input_chan chan in
  close_in chan;

let () =
  try main ()
    exn ->
      prerr_endline (Printexc.to_string exn);
      Printexc.print_backtrace Pervasives.stderr;
      exit 1


Filed under Uncategorized

Extracting filesystems from guest images, reconstructing guest images from filesystems, part 3

In part 1 I “exploded” a disk image into its constituent filesystems. In part 2 I “imploded” those filesystems back into a disk image without LVM. Now let’s see if we can get this thing to boot.

Since there is no boot sector nor grub, the disk image produced by virt-implode won’t boot normally. You could boot it using an external kernel and initrd, but let’s see if we can install grub first.

Using virt-rescue we can interactively run programs from the guest:

$ virt-rescue -a output.img
><rescue> mount /dev/sda2 /sysroot
><rescue> mount /dev/sda1 /sysroot/boot
><rescue> mount --bind /dev /sysroot/dev
><rescue> mount --bind /sys /sysroot/sys
><rescue> mount --bind /proc /sysroot/proc
><rescue> chroot /sysroot
sh: no job control in this shell
><rescue> cat /etc/redhat-release
Red Hat Enterprise Linux AS release 4 (Nahant Update 8)
><rescue> vi /etc/fstab

You may need to fix /etc/fstab in the guest so that it points to the new partitions. For guests using LABELs or UUIDs, this won’t be necessary.

At this point, it should be simply a matter of running grub-install. But here’s where I remember how much I hate grub, because it just throws up peculiar, non-actionable error messages for every conceivable variation in the command.

So I give up that, and decide to extract the kernel and initrd and use the external boot method after all:

$ guestfish --ro -a output.img -i

Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.

Type: 'help' for help on commands
      'man' to read the manual
      'quit' to quit the shell

Operating system: Red Hat Enterprise Linux AS release 4 (Nahant Update 8)
/dev/sda2 mounted on /
/dev/sda1 mounted on /boot

><fs> ll /boot
total 4397
drwxr-xr-x.  5 root root    1024 Nov 15 13:49 .
drwxr-xr-x. 24 root root    4096 Nov 15 13:50 ..
-rw-r--r--.  1 root root  909034 Apr 20  2009 System.map-2.6.9-89.EL
drwxr-xr-x   3 root root    1024 Nov 15 13:49 boot
-rw-r--r--.  1 root root   45145 Apr 20  2009 config-2.6.9-89.EL
drwxr-xr-x.  2 root root    1024 Nov 15 13:58 grub
-rw-r--r--.  1 root root 1546843 Oct 27  2010 initrd-2.6.9-89.EL.img
drwx------   2 root root   12288 Oct 27  2010 lost+found
-rw-r--r--.  1 root root   23108 Aug  3  2005 message
-rw-r--r--.  1 root root   21282 Aug  3  2005 message.ja
-rw-r--r--.  1 root root   67352 Apr 20  2009 symvers-2.6.9-89.EL.gz
-rw-r--r--.  1 root root 1829516 Apr 20  2009 vmlinuz-2.6.9-89.EL
><fs> download /boot/vmlinuz-2.6.9-89.EL /tmp/vmlinuz-2.6.9-89.EL
><fs> download /boot/initrd-2.6.9-89.EL.img /tmp/initrd-2.6.9-89.EL.img
$ qemu-kvm -m 512 \
  -kernel vmlinuz-2.6.9-89.EL \
  -initrd initrd-2.6.9-89.EL.img \
  -append 'ro root=/dev/hda2' \
  -hda output.img

(I should really use libvirt, but this quick test proves that the guest works fine)

1 Comment

Filed under Uncategorized