Tip: use SystemTap to monitor SELinux changes to files

Something unknown was changing the labels on certain devices behind my back. We couldn’t find out what it was using ordinary diagnostics, so I decided to investigate if we could do this with SystemTap. I quickly found an existing script to monitor changes in ordinary file attributes. This won’t work for SELinux labels though because those are stored in ext2/3/4 extended attributes (xattrs).

Basically I had to modify that script to monitor calls to setxattr instead.

Using the LXR I found that the call is implemented in Linux in fs/xattr.c, function vfs_setxattr. I had to modify the script to probe that kernel function, and the parameters are slightly different too.

I also had to install the correct kernel-{,PAE-}debuginfo package corresponding to my installed kernel. This is how SystemTap is able to resolve symbols in the current kernel.

/* Watch changes to xattrs on an inode.
 * http://rwmj.wordpress.com/2010/03/09/tip-use-systemtap-to-monitor-selinux-changes-to-files/
 */

probe kernel.function("vfs_setxattr")
{
  dev_nr = $dentry->d_inode->i_sb->s_dev
  inode_nr = $dentry->d_inode->i_ino

  if (inode_nr == $1)
    printf ("%s(%d) %s 0x%x/%u %d %s %s\n",
      execname(), pid(), probefunc(), dev_nr, inode_nr, uid(),
      kernel_string ($name), kernel_string_n ($value, $size))
}

Then run it with:

# stap -v /tmp/inodewatchxattr.stp inodenum

The bug turned out to be udevd, which I don’t think anyone was expecting …

libvirtd(4338) vfs_setxattr 0x5/166267 0 security.selinux system_u:object_r:svirt_image_t:s0:c177,c272
udevd(28299) vfs_setxattr 0x5/166267 0 security.selinux system_u:object_r:fixed_disk_device_t:s0
udevd(28299) vfs_setxattr 0x5/166267 0 security.selinux system_u:object_r:fixed_disk_device_t:s0
udevd(28299) vfs_setxattr 0x5/166267 0 security.selinux system_u:object_r:fixed_disk_device_t:s0

All in all, I’m impressed with SystemTap. It’s a simple, strongly-typed, sane programming language with type inference. Thankfully Python was not an influence on it.