Tag Archives: python

What I learned about AMQP

I’m playing with AMQP at the moment. I thought I’d start off with RabbitMQ first.

The good:

  • It works.
  • OCaml and Python programs can talk to each other.
  • It works across remote hosts. You need to open port 5672/tcp on the firewall.

The bad:

  • RabbitMQ and Apache Qpid use different versions of AMQP and are not interoperable! Good summary of the mess here. This might be resolved when everyone gets around to supporting AMQP 1-0, but even though that standard has been published, no one is expecting interop to happen for at least a year.
  • You can’t cluster different versions of the RabbitMQ broker together.
  • Even if all your hosts are at the same RabbitMQ version, you have to open more firewall ports and make changes to the start-up scripts. (Dynamic ports? Really? Did we learn nothing from NFS?)
  • Long, obscure Erlang error messages which don’t point to the problem. eg. You’ll get a good 25 lines of error message if another process is already bound to a port.
  • Possibly just a Fedora packaging problem: I managed to get my host into some state where it’s impossible to stop the RabbitMQ server except by kill -9, and after that I can’t start or stop it.
Advertisements

2 Comments

Filed under Uncategorized

Tip: Using libguestfs from Perl

I translated the standard libguestfs examples (already available in C/C++, OCaml, Python, Ruby) into Perl.

If you want to call libguestfs from Perl, you have to use Sys::Guestfs.

All 300+ libguestfs API calls are available to all language bindings equally because we generate the bindings.

Leave a comment

Filed under Uncategorized

Use hivex from Python to read and write Windows Registry “hive” files

I added Python bindings to hivex today.

Here is an example using Python, libguestfs and hivex to download the user preferences registry from a Windows virtual machine and print out the Internet Explorer start page for a particular user. When you run it, it should print out something like:

User rjones's IE home page is http://go.microsoft.com/fwlink/?LinkId=69157

This example shows downloading and printing values, but libguestfs and hivex can also be used to make changes (but not to live guests).

#!/usr/bin/python

import guestfs
import hivex

# The name of a Windows virtual machine on this host.  This
# example script makes some assumptions about the registry
# location and contents which only apply on Windows Vista
# and later versions.
windows_domain = "Win7x32"

# Username on the Windows VM.
username = "rjones"

# Use libguestfs to download the HKEY_CURRENT_USER hive.
g = guestfs.GuestFS ()
g.add_domain (windows_domain, readonly=1)
g.launch ()

roots = g.inspect_os ()
root = roots[0]
g.mount_ro (root, "/")

path = "/users/%s/ntuser.dat" % username
path = g.case_sensitive_path (path)
g.download (path, "/tmp/ntuser.dat")

# Use hivex to pull out a registry key.
h = hivex.Hivex ("/tmp/ntuser.dat")

key = h.root ()
key = h.node_get_child (key, "Software")
key = h.node_get_child (key, "Microsoft")
key = h.node_get_child (key, "Internet Explorer")
key = h.node_get_child (key, "Main")

val = h.node_get_value (key, "Start Page")
start_page = h.value_value (val)
#print start_page

# The registry key is encoded as UTF-16LE, so reencode it.
start_page = start_page[1].decode ('utf-16le').encode ('utf-8')

print "User %s's IE home page is %s" % (username, start_page)

9 Comments

Filed under Uncategorized

libguestfs examples in C, OCaml, Python and Ruby

I took two example C programs (one appeared on this blog earlier) and translated them precisely into OCaml, Python and Ruby:

  1. C/C++ example
  2. OCaml example
  3. Python example
  4. Ruby example

Leave a comment

Filed under Uncategorized

Python script to replace templates in configuration files in VMs

The idea behind this script is you create a template VM, and when you need a new VM, you duplicate the disk image and use the script to put final values into configuration files. This script will take a disk image containing a configuration file like:

HOSTNAME=@HOSTNAME@

and edit it to:

HOSTNAME=yourhost.example.com

#!/usr/bin/python

image = "disk.img"
root_filesystem = "/dev/vg_template/lv_root"
filename = "/etc/sysconfig/network"
pattern = "@HOSTNAME@"
replacement = "yourhost.example.com"

import tempfile
import os
import fileinput
import shutil
import guestfs

g = guestfs.GuestFS ()

g.add_drive (image)
g.launch ()
g.mount_options ("", root_filesystem, "/")

tmpdir = tempfile.mkdtemp ()
tmpfile = os.path.join (tmpdir, "file")

g.download (filename, tmpfile)

for line in fileinput.FileInput (tmpfile, inplace=1):
    line = line.replace (pattern, replacement)
    print line,

g.upload (tmpfile, filename)

g.umount_all ()
g.sync ()

shutil.rmtree (tmpdir)

If you have to make many changes to a VM image, don’t run this script repeatedly. Instead, duplicate the download ... upload section of the code as required.

2 Comments

Filed under Uncategorized

Tip: daily Bugzilla reports

This is my inbox, and it sucks:

Bugzilla is like a black hole for bugs. The search tools fail so badly it’s often impossible to find a bug that you were working on the day before. It’s slow, clumsy, and disorganized.

But one bright point is it has a reasonable command line reporting tool also available as a Fedora package. So I decided yesterday to write a Bugzilla report script and have it email me daily from cron.

The starting point is to identify “bugs of interest”. The bugs of interest to me are:

  • bugs I reported
  • bugs I am assigned to fix
  • bugs I’m CC’d on

I put some thought into this set of criteria for bugs:

  1. I should be able to register an interest in any bug: Yes, by adding myself to the CC field.
  2. I should be able to unregister an interest: Yes, by removing myself from the CC, or closing or reassigning those bugs where I’m reporter or assignee.
  3. It shouldn’t tell me about bugs I’m not interested in: Yes, I would be reporter, assignee or in the CC list for any bug related to me or a package I maintain.
  4. It shouldn’t miss out any bugs I might be interested in: Yes, if I’ve ever added a comment, I will be in the CC field at least, and I can always unregister myself from any I no longer care about.

With the command line tool, getting the raw list of bug IDs is very simple.

for flag in -r -c -a; do
  bugzilla query $flag $email -i
done | sort -n -u

(replacing $email with your email address, or even mine if you so care).

That pulls out 781 unique bug IDs, the oldest being one I don’t remember reporting and the most recent being a virt-v2v bug. (Note this includes CLOSED bugs which in the script are ignored).

Now I take the list of bug IDs and pull out the other fields I want from the Bugzilla database. The command below is just an illustrative example of what the script does:

$ bugzilla query -b "86308,601092" \
  --outputformat="%{bug_id}|%{bug_status}|%{product}|%{component}|%{short_desc}" |
  sort -n -t"|"
86308|CLOSED|Red Hat Web Site|Other|Red Hat Command Centre site is down
601092|NEW|Red Hat Enterprise Linux 5|libguestfs|[RFE]Incorrect error msg popped up when missing "-f" in v2v command

As far as I know you have to guess what the %-fields are called. I also had to choose a separator character which wouldn’t appear in any field except the short_desc (which is always the last field), since fields like the product name can and do contain spaces.

The rest of the script is “merely” formatting this whole lot into a nice looking email report:

Bugs in OPEN states:

*** coccinelle ***

  In Fedora:
    502185 coccinelle segfaults on ppc64
    579457 coccinelle-0.2.3rc1 is available

*** e2fsprogs ***

  In Red Hat Enterprise Linux 5:
    518190 mke2fs -t option in RHEL5 differs vs upstream, leading to confusion

[etc]

You can download the script here: http://oirase.annexia.org/tmp/bugs-report.ml.txt

14 Comments

Filed under Uncategorized

Alix 2D3 board, part 1


Left-to-right: serial, 3xnetwork, USB, power. Note no graphics.

I bought an Alix 2D3 from LinITX. These are interesting little single board computers, featuring a slow AMD Geode processor, VIA chipset, no graphics, three 100 Mbps network ports, and a serial port. They boot from a CF card and cost around £100 including VAT and delivery.

I’ve installed Debian on the CF card, and had to deal with a Connector Conspiracy which has prevented me from mating my laptop with the serial console. Why Debian? Fedora is too bloated because of all the desktop/Python/*Kit crap that comes by default, whereas Debian can boot comfortably in 256 MB of RAM. Or indeed much less — in my previous job I used to install Debian virtual servers by default with 64 MB of RAM, and sometimes 32 MB. That’s more than an order of magnitude smaller than the minimal Fedora install. Second choice was FreeBSD which has a much more streamlined and visibly faster kernel than Linux, but Debian seems to be working out so far.

More to follow …

7 Comments

Filed under Uncategorized