Tag Archives: xml

Decoding the Windows Event Log using guestfish

The Windows Event Log system is Windows’ centralized way of capturing log messages from the operating system and a wide variety of applications.

In all versions of Windows the messages are stored in binary files and normally you can only read these using Microsoft’s proprietary Event Viewer program. In Windows Vista, Microsoft overhauled the entire messaging system and changed the binary format. Luckily a German computer forensics researcher named Andreas Schuster reverse engineered the format [PDF] and wrote a small GPL’d program called EvtxParser [download dir] which can decode it.

We can use guestfish (or libguestfs) along with EvtxParser to easily look at the events in any Windows Vista / 2008 / 7 virtual machine.

Firstly download EvtxParser. You don’t need to install it (indeed, it doesn’t come with any build system so you can’t install it without some effort). You do need to install a few supporting Perl modules though:

# yum install perl-Digest-CRC perl-DateTime \
    perl-Carp-Assert perl-CPAN tidy
# cpan install Data::Hexify

Also the Perl scripts in the EvtxParser zip file aren’t all executable, so chmod them:

$ chmod +x *.pl

Now grab some *.evtx files from your Windows Vista (or later) system. They are stored in the /Windows/System32/winevt/Logs/ directory:

# guestfish --ro -i -d WindowsGuest
><fs> ll win:/Windows/System32/winevt/Logs
total 10540
drwxrwxrwx 1 root root   28672 Oct  1  2010 .
drwxrwxrwx 1 root root       0 Jul 14  2009 ..
-rwxrwxrwx 2 root root 1118208 Dec 23 18:22 Application.evtx
-rwxrwxrwx 2 root root   69632 Sep 19  2010 HardwareEvents.evtx
-rwxrwxrwx 2 root root   69632 Sep 19  2010 Internet Explorer.evtx
-rwxrwxrwx 2 root root   69632 Sep 19  2010 Key Management Service.evtx
-rwxrwxrwx 2 root root   69632 Sep 19  2010 Media Center.evtx
[and many more]
><fs> download win:/Windows/System32/winevt/Logs/System.evtx /tmp/System.evtx
><fs> exit

You can directly dump the files you have downloaded as XML to reveal the events inside them.

$ ./evtxdump.pl /tmp/System.evtx | tidy -xml -indent -quiet | less

This gives me a 40,000 line XML document(!) As a representative sample, the last event is the shutdown event from when I shut the VM off last time:

  <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
    <System>
      <Provider Name="Service Control Manager"
      Guid="{555908d1-a6d7-4695-8e1e-26931d2012f4}"
      EventSourceName="Service Control Manager" />
      <EventID Qualifiers="16384">7036</EventID>
      <Version>0</Version>
      <Level>4</Level>
      <Task>0</Task>
      <Opcode>0</Opcode>
      <Keywords>0x8080000000000000</Keywords>
      <TimeCreated SystemTime="2010-12-23T18:22:58.4980Z" />
      <EventRecordID>1452</EventRecordID>
      <Correlation />
      <Execution ProcessID="456" ThreadID="1748" />
      <Channel>System</Channel>
      <Computer>win7x32</Computer>
      <Security />
    </System>
    <EventData>
      <Data Name="param1">Power</Data>
      <Data Name="param2">stopped</Data>
      <Binary>50006F007700650072002F0031000000</Binary>
    </EventData>
  </Event>

A tip for reading these: the key field is the EventID. For example, EventID 1074 is a user-initiated clean shutdown.

2 Comments

Filed under Uncategorized

Stay classy, Microsoft

I thought when I was looking at the Windows Registry I’d seen it all … until today when I found that the Windows 7 installation CD contains what seems like a registry that directly encodes an XML schema document.

WHY!!!

If you have a Win7 install CD, it is /sources/schema.dat. After passing it through hivexregedit –export you get:

Windows Registry Editor Version 5.00

[\]

[\wcm://Microsoft-Windows-DNS-Client?version=6.1.7600.16385&language=neutral&processorArchitecture=amd64&publicKeyToken=31bf3856ad364e35&versionScope=nonSxS&scope=allUsers]

[\wcm://Microsoft-Windows-DNS-Client?version=6.1.7600.16385&language=neutral&processorArchitecture=amd64&publicKeyToken=31bf3856ad364e35&versionScope=nonSxS&scope=allUsers\metadata]
"@_legacyHandler"=hex(10000005):04,00,00,00
"@_targetNamespace"=hex(1000000c):4d,00,69,00,63,00,72,00,6f,00,73,00,6f,00,66,00,74,00,2d,00,57,00,69,00,6e,00,64,00,6f,00,77,00,73,00,2d,00,44,00,4e,00,53,00,2d,00,43,00,6c,00,69,00,65,00,6e,00,74,00,00,00
"@language"=hex(1000000c):6e,00,65,00,75,00,74,00,72,00,61,00,6c,00,00,00
"@processorArchitecture"=hex(1000000c):61,00,6d,00,64,00,36,00,34,00,00,00
"@publicKeyToken"=hex(1000000c):33,00,31,00,62,00,66,00,33,00,38,00,35,00,36,00,61,00,64,00,33,00,36,00,34,00,65,00,33,00,35,00,00,00
"@version"=hex(1000000c):36,00,2e,00,31,00,2e,00,37,00,36,00,30,00,30,00,2e,00,31,00,36,00,33,00,38,00,35,00,00,00
"@versionScope"=hex(1000000c):6e,00,6f,00,6e,00,53,00,78,00,53,00,00,00
"@xmlns:asmv3"=hex(1000000c):75,00,72,00,6e,00,3a,00,73,00,63,00,68,00,65,00,6d,00,61,00,73,00,2d,00,6d,00,69,00,63,00,72,00,6f,00,73,00,6f,00,66,00,74,00,2d,00,63,00,6f,00,6d,00,3a,00,61,00,73,00,6d,00,2e,00,76,00,33,00,00,00
"@xmlns:wcm"=hex(1000000c):68,00,74,00,74,00,70,00,3a,00,2f,00,2f,00,73,00,63,00,68,00,65,00,6d,00,61,00,73,00,2e,00,6d,00,69,00,63,00,72,00,6f,00,73,00,6f,00,66,00,74,00,2e,00,63,00,6f,00,6d,00,2f,00,57,00,4d,00,49,00,43,00,6f,00,6e,00,66,00,69,00,67,00,2f,00,32,00,30,00,30,00,32,00,2f,00,53,00,74,00,61,00,74,00,65,00,00,00
"@xmlns:xmlns"=hex(1000000c):68,00,74,00,74,00,70,00,3a,00,2f,00,2f,00,77,00,77,00,77,00,2e,00,77,00,33,00,2e,00,6f,00,72,00,67,00,2f,00,32,00,30,00,30,00,30,00,2f,00,78,00,6d,00,6c,00,6e,00,73,00,2f,00,00,00
"@xmlns:xsd"=hex(1000000c):68,00,74,00,74,00,70,00,3a,00,2f,00,2f,00,77,00,77,00,77,00,2e,00,77,00,33,00,2e,00,6f,00,72,00,67,00,2f,00,32,00,30,00,30,00,31,00,2f,00,58,00,4d,00,4c,00,53,00,63,00,68,00,65,00,6d,00,61,00,00,00

[\wcm://Microsoft-Windows-DNS-Client?version=6.1.7600.16385&language=neutral&processorArchitecture=amd64&publicKeyToken=31bf3856ad364e35&versionScope=nonSxS&scope=allUsers\metadata\complexTypes]

[\wcm://Microsoft-Windows-DNS-Client?version=6.1.7600.16385&language=neutral&processorArchitecture=amd64&publicKeyToken=31bf3856ad364e35&versionScope=nonSxS&scope=allUsers\metadata\complexTypes\DomainNameCollectionType]
"@_MemberList"=hex(1000800c):44,00,6f,00,6d,00,61,00,69,00,6e,00,4e,00,61,00,6d,00,65,00,00,00,00,00
"@_valid"=hex(10000001):00

[\wcm://Microsoft-Windows-DNS-Client?version=6.1.7600.16385&language=neutral&processorArchitecture=amd64&publicKeyToken=31bf3856ad364e35&versionScope=nonSxS&scope=allUsers\metadata\complexTypes\DomainNameCollectionType\DomainName]
"@_hint"=hex(10000006):01,00,00,00
"@_type"=hex(10000005):0c,60,00,00
"@xsd:maxOccurs"=hex(10000006):ff,ff,ff,ff
"@xsd:minOccurs"=hex(10000006):00,00,00,00
"@xsd:type"=hex(1000000c):44,00,6f,00,6d,00,61,00,69,00,6e,00,4e,00,61,00,6d,00,65,00,54,00,79,00,70,00,65,00,00,00

[\wcm://Microsoft-Windows-DNS-Client?version=6.1.7600.16385&language=neutral&processorArchitecture=amd64&publicKeyToken=31bf3856ad364e35&versionScope=nonSxS&scope=allUsers\metadata\complexTypes\InterfaceCollectionType]
"@_MemberList"=hex(1000800c):49,00,6e,00,74,00,65,00,72,00,66,00,61,00,63,00,65,00,00,00,00,00
"@_valid"=hex(10000001):00

[\wcm://Microsoft-Windows-DNS-Client?version=6.1.7600.16385&language=neutral&processorArchitecture=amd64&publicKeyToken=31bf3856ad364e35&versionScope=nonSxS&scope=allUsers\metadata\complexTypes\InterfaceCollectionType\Interface]
"@_type"=hex(10000005):10,40,00,00
"@key"=hex(1000000c):49,00,64,00,65,00,6e,00,74,00,69,00,66,00,69,00,65,00,72,00,00,00
"@xsd:maxOccurs"=hex(10000006):ff,ff,ff,ff
"@xsd:minOccurs"=hex(10000006):00,00,00,00
"@xsd:type"=hex(1000000c):49,00,6e,00,74,00,65,00,72,00,66,00,61,00,63,00,65,00,54,00,79,00,70,00,65,00,00,00

(and on for 1,355 lines)

Leave a comment

Filed under Uncategorized

Display an XML doc from the command line

Leading me down the garden path today, how to quickly display an XML document … graphically, from the command line?

This doesn’t work:

$ virt-inspector --xml RHEL54.img | firefox -

Creating a temporary file is possible, but ugly.

Then I was tipped off that you can create and pass a data: URI to Firefox.

There is no existing command line tool to generate data URIs, but we can write one in 3 lines of shell script:

#!/bin/sh -
echo -n data:$1\;
uuencode -m notused | tail -n +2 | tr -d '\n'

Example:

$ cat > /tmp/test.html
<b>Hello,</b>
<i>world!</i>
$ datauri text/html < /tmp/test.html
data:text/html;PGI+SGVsbG8sPC9iPgo8aT53b3JsZCE8L2k+Cg======

This also doesn’t work. There are two problems: the XML generated by virt-inspector is too long for a data URI, and in any case Firefox seems to ignore the data URI although I’m sure I’m constructing it correctly. Maybe it’s a security or configuration issue?

Well, good idea, but let’s go back to the temporary file idea. Bash process substitution might have worked:

$ firefox <(virt-inspector --xml RHEL54.img)

but Firefox’s frankly stupid session management crap gets in the way because this command expands to something like:

$ firefox /proc/self/fd/123

and the new firefox process passes the non-portable /proc/self path to the currently running instance of Firefox which doesn’t have the same view of /proc/self.

So we are finally left with:

$ firefox $(f=`mktemp -u`;
            virt-inspector --xml RHEL54.img > $f.xml;
            echo $f.xml)

which is fugly and unsafe.

If only there was a less insane tool to display XML, but being XML I guess insane goes with the territory.

3 Comments

Filed under Uncategorized

XML or S-expressions?

I was writing a little program to track monthly outgoings. “Only” £30/month for internet access or whatever can quickly add up …

But what format should I save the data in? XML is heavyweight and redundant compared to S-expressions, compare:

<outgoing rate="monthly">
  <price>30.</price>
  <name>Internet</name>
</outgoing>
(outgoing
  (rate monthly)
  (price 30.)
  (name "Internet"))

(Update: fixed XML x 2)

One difference I always notice is the redundancy of attributes like rate=”monthly”. S-expressions let you decide to make the attribute structured, but with XML you’re stuck with a simple string unless you make an incompatible change to the schema.

Another difference is that S-expressions are typed. 30 is a float and “Internet” is a string. XML is all just strings, which sucks when your language is typed.

On the other hand this article makes a good argument that XML is not (and is better than) S-expressions. More debate here.

A killer feature of OCaml is the sexplib syntax extension which makes S-expressions really easy. You just define any OCaml type in the usual way, and add with sexp after it, and that magically generates serializer and deserializer functions for your type, so you can slurp your data into and out of S-expression files effortlessly. A page of boilerplate disappears in just two words. That’s probably the reason why I’ll go with S-expressions for this.

14 Comments

Filed under Uncategorized

Use libguestfs to view devices and filesystems in a VM

Here is a small C program (examples/to-xml.c) which uses libguestfs to examine a virtual machine and produce a little XML display of the partitions, LVM data and filesystems within the virtual machine.

This XML is fanciful. Is there a standard XML description for this sort of thing?

Example:

$ examples/to-xml RHEL52PV32-tmp.img | tidy -xml -i -q
<guestfs-system>
  <devices>
    <device dev="/dev/sda">
      <partitions>
        <partition dev="/dev/sda1">
          <fs type="ext3">
            <uuid>780a9d90-e0e4-45a4-9012-3e58eedb25be</uuid>
            <blocksize>1024</blocksize>
          </fs>
        </partition>
        <partition dev="/dev/sda2">
          <physvol />
        </partition>
      </partitions>
    </device>
  </devices>
  <volgroups>
    <volgroup name="VolGroup00">
      <logvol name="/dev/VolGroup00/LogVol00">
        <fs type="ext3">
          <uuid>2d3af696-15af-4f32-97ae-eeb0870c5a94</uuid>
          <blocksize>4096</blocksize>
        </fs>
      </logvol>
      <logvol name="/dev/VolGroup00/LogVol01">
        <linux-swap />
      </logvol>
    </volgroup>
  </volgroups>
</guestfs-system>

Leave a comment

Filed under Uncategorized