Use guestfish and nbdkit to examine physical disk locations

This question came up: LVM makes sure that the logical volumes it creates are properly aligned to a generous block size, to ensure most efficient access. However what happens if the partition underneath an LVM physical volume isn’t aligned? Is LVM smart enough to move the LV around so it is still aligned, relative to the physical disk underneath?

This is easily answered using guestfish and captive ndkit. (Since captive nbdkit is a new feature, you’ll need Rawhide this Fedora 20 update, or to compile nbdkit from source if you want to run the commands below.)

First we create a badly aligned partition, and put an LVM physical volume inside it, and an LVM logical volume in the physical volume. This kind of disk construction is trivial with guestfish:

$ guestfish -N disk <<EOF
  part-init /dev/sda mbr
  # Unaligned partition, starts on sector 63
  part-add /dev/sda p 63 -100

  pvcreate /dev/sda1
  vgcreate VG /dev/sda1
  lvcreate LV VG 32

  # You could also create a filesystem here:
  #mkfs ext2 /dev/VG/LV
EOF

Use virt-filesystems to verify the layout of the disk:

$ virt-filesystems -a test1.img --all --long -h
Name        Type        VFS      Label  MBR  Size  Parent
/dev/VG/LV  filesystem  unknown  -      -    32M   -
/dev/VG/LV  lv          -        -      -    32M   /dev/VG
/dev/VG     vg          -        -      -    96M   /dev/sda1
/dev/sda1   pv          -        -      -    96M   -
/dev/sda1   partition   -        -      83   100M  /dev/sda
/dev/sda    device      -        -      -    100M  -

Now using nbdkit we can try writing to the logical volume, while observing the actual read and write operations that happen on the underlying file:

$ nbdkit -f -v file file=test1.img \
  --run '
    guestfish -x --format=raw -a $nbd \
      run : \
      pwrite-device /dev/VG/LV " " 0
  '

This command prints out a lot of debugging. At the end we see:

[...]
libguestfs: trace: pwrite_device "/dev/VG/LV" " " 0
nbdkit: file[1]: debug: acquire per-connection request lock
nbdkit: file[1]: debug: pread count=4096 offset=1080832
nbdkit: file[1]: debug: release per-connection request lock
nbdkit: file[1]: debug: acquire per-connection request lock
nbdkit: file[1]: debug: pwrite count=4096 offset=1080832
nbdkit: file[1]: debug: release per-connection request lock
libguestfs: trace: pwrite_device = 1
[...]

It looks like writing to the first few bytes of /dev/VG/LV resulted in a read and a write of a 4K block starting at byte offset 1080832 (relative to the start of the physical disk).

1080832 = 0x107E00 which is 512 byte aligned. That is not aligned for performance, showing that LVM does not do any magic to adjust LVs relative to the underlying disk when the partition it is on is not aligned properly.

Notice that I created the partition starting on sector 63. If you add 1 sector (512 bytes, 0x200) to 0x107E00 you get 0x108000 which is aligned to 32K.

Leave a comment

Filed under Uncategorized

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s