Parallel xzcat .. causes virt-builder to slow down

Eric Sandeen solved the mystery. See the end.

If you look at the ordinary output of virt-builder:

$ virt-builder ubuntu-12.04 
[   1.0] Downloading: http://libguestfs.org/download/builder/ubuntu-12.04.xz
[   3.0] Creating disk image: ubuntu-12.04.img
[   3.0] Uncompressing: http://libguestfs.org/download/builder/ubuntu-12.04.xz
[  29.0] Running virt-resize to expand the disk to 4.2G
[  86.0] Opening the new disk
[  89.0] Setting a random seed
[  89.0] Random root password: 2oMCHc3GaqoPKpXf [did you mean to use --root-password?]
[  89.0] Finishing off
Output: ubuntu-12.04.img
Total usable space: 3.1G
Free space: 2.4G (75%)

The (in bold) uncompressing step takes 26 seconds or so. This is running xzcat which is single threaded.

This afternoon I wrote a parallel pxzcat tool. On its own, it’s about 3½ times faster on 4 cores, so that’s pretty scalable.

Unfortunately, although the compression step goes faster, virt-builder actually slows down:

$ virt-builder ubuntu-12.04 
[   1.0] Downloading: file:///home/rjones/d/libguestfs/builder/website/ubuntu-12.04.xz
[   1.0] Creating disk image: ubuntu-12.04.img
[   2.0] Uncompressing: file:///home/rjones/d/libguestfs/builder/website/ubuntu-12.04.xz
[  12.0] Running virt-resize to expand the disk to 4.2G
[  86.0] Opening the new disk
[ 100.0] Setting a random seed
[ 100.0] Random root password: KQIhD8gKxksbC36k [did you mean to use --root-password?]
[ 100.0] Finishing off
Output: ubuntu-12.04.img
Total usable space: 3.1G
Free space: 2.4G (75%)

This result is quite reproducible.

(Also we should run virt-resize at all if the user didn’t request a --size argument).

Update:

Thanks to Eric Sandeen who solved the problem. It turns out to be a heuristic in ext4 (auto_da_alloc), which causes a file that has been truncated to be flushed out on close.

2 Comments

Filed under Uncategorized

2 responses to “Parallel xzcat .. causes virt-builder to slow down

  1. Gabriel

    Is that a delalloc fail, which filefrag will confirm?
    Could you use fallocate in your decompressor?

    • rich

      No, I’m using ftruncate because I want to preserve sparseness in the output image.

      Also I’m using posix_fadvise, but that makes no difference.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s