Tag Archives: guestfish

Testing exabyte-sized filesystems using qcow2 and guestfish

You can use qcow2 backing files as a convenient way to test what happens when you try to create exabyte-sized filesystems. Just to remind you, 1 exabyte is a million terabytes, or a pile of ordinary hard disks stacked 8 miles high.

There is a bug in qemu that prevents you from creating very large disks unless you adjust the cluster_size option (thanks Kevin Wolf):

$ qemu-img create -f qcow2 huge.qcow2 \
      $((1024*1024))T -o cluster_size=2M
Formatting 'huge.qcow2', fmt=qcow2 size=1152921504606846976 encryption=off cluster_size=2097152 lazy_refcounts=off 

After that you can just attach the disk to guestfish and start playing with huge filesystems.

[I should note that virt-rescue is probably a better choice of tool here, especially for people who need to experiment with unusual filesystem or LVM options]

$ guestfish -a huge.qcow2

Welcome to guestfish, the guest filesystem shell for
editing virtual machine filesystems and disk images.

Type: 'help' for help on commands
      'man' to read the manual
      'quit' to quit the shell

><fs> run
><fs> blockdev-getsize64 /dev/sda
1152921504606846976
><fs> part-disk /dev/sda gpt

Ext4 (according to Wikipedia) is supposed to support 1 exabyte disks, but I couldn’t get that to work, possibly because there was not enough RAM:

><fs> mkfs ext4 /dev/sda1
libguestfs: error: mkfs: ext4: /dev/sda1: mke2fs 1.42.5 (29-Jul-2012)
/dev/sda1: Not enough space to build proposed filesystem while setting up superblock

XFS could create a filesystem, but I didn’t let it run to completion because it would need about 5 petabytes to store the filesystem metadata:

><fs> mkfs xfs /dev/sda1
[ disks churn for many minutes while qcow2 file grows
and grows and grows ... ]

LVM2 PVs are possible, but creating a VG requires us to adjust the extent size:

><fs> pvcreate /dev/sda1
><fs> vgcreate VG /dev/sda1
libguestfs: error: vgcreate:   PV /dev/sda1 too large for extent size 4.00 MiB.
  Format-specific setup of physical volume '/dev/sda1' failed.
  Unable to add physical volume '/dev/sda1' to volume group 'VG'.
><fs> debug sh "vgcreate -s 1G VG /dev/sda1"
  Volume group "VG" successfully created
><fs> lvcreate LV VG 1000000000
><fs> lvs-full
[0] = {
  lv_name: LV
[...]
  lv_size: 1048576536870912
}

Previously …

Leave a comment

Filed under Uncategorized

Using libguestfs to access iSCSI disks

In libguestfs ≥ 1.21.38 you can access at least some iSCSI disks.

On my server (RHEL 6 in this case) I create an iSCSI target backed by a Windows XP disk image:

# service tgtd start
Starting SCSI target daemon:                       [  OK  ]
# tgtadm --lld iscsi --op new --mode target --tid 1 \
      -T iqn.1994-05.com.redhat
# chcon system_u:object_r:tgtd_var_lib_t:s0 /tmp/winxp.img
# tgtadm --lld iscsi --op new --mode logicalunit --tid 1 \
      --lun 1 -b /tmp/winxp.img
# tgt-admin -s
...

Previously I opened port 3250 on the server. Because libguestfs doesn’t yet support authentication against the iSCSI server, I had to bypass that:

# tgtadm --lld iscsi --mode target --op bind --tid 1 -I ALL

Now on the client, I can connect to the iSCSI target using libguestfs like this:

$ export LIBGUESTFS_BACKEND=direct
$ guestfish --format=raw -a iscsi://x.x.x.x/iqn.1994-05.com.redhat/1 -i

Welcome to guestfish, the guest filesystem shell for
editing virtual machine filesystems and disk images.

Type: 'help' for help on commands
      'man' to read the manual
      'quit' to quit the shell

Operating system: Microsoft Windows XP
/dev/sda1 mounted on /

><fs> ll /
total 1573209
drwxrwxrwx  1 root root       4096 Apr 16  2012 .
drwxr-xr-x 23 1000 1000       4096 May 11 17:16 ..
-rwxrwxrwx  1 root root          0 Oct 11  2011 AUTOEXEC.BAT
-rwxrwxrwx  1 root root          0 Oct 11  2011 CONFIG.SYS
drwxrwxrwx  1 root root       4096 Oct 11  2011 Documents and Settings
-rwxrwxrwx  1 root root          0 Oct 11  2011 IO.SYS
-rwxrwxrwx  1 root root          0 Oct 11  2011 MSDOS.SYS
-rwxrwxrwx  1 root root      47564 Apr 14  2008 NTDETECT.COM
drwxrwxrwx  1 root root       4096 Oct 11  2011 Program Files
drwxrwxrwx  1 root root       4096 Oct 11  2011 System Volume Information
drwxrwxrwx  1 root root      28672 Oct 11  2011 WINDOWS
-rwxrwxrwx  1 root root        211 Oct 11  2011 boot.ini
-rwxrwxrwx  1 root root     250048 Apr 14  2008 ntldr
-rwxrwxrwx  1 root root 1610612736 Oct 11  2011 pagefile.sys

Leave a comment

Filed under Uncategorized

Using libguestfs to read Hyper-V VHDX disk images

(Thanks to Jeff Cody for both writing the code and helping me to get it to work).

New in qemu 1.5 is read-only support for Hyper-V‘s native disk format, vhdx. You can now open vhdx files in libguestfs just like any other:

$ guestfish --ro --format=vhdx -a /tmp/f18x64.vhdx -i

Welcome to guestfish, the guest filesystem shell for
editing virtual machine filesystems and disk images.

Type: 'help' for help on commands
      'man' to read the manual
      'quit' to quit the shell

Operating system: Fedora release 18 (Spherical Cow)
/dev/fedora/root mounted on /
/dev/sda1 mounted on /boot

Notes:

  1. As stated above, you will need the very latest qemu for this to work.
  2. You must specify --format=vhdx at the moment because of a missing feature in libvirt.
  3. You must specify --ro because writing is not supported (by the qemu driver).

Leave a comment

Filed under Uncategorized

Tip: Convert a Windows DVD ISO to a bootable USB key using guestfish

Situation: You have a Windows DVD (or ISO), but like any sane person in 2013 you don’t have a DVD drive on the computer. You want to convert the Windows DVD into a bootable USB key. There are many recipes for this online, but they all require another Windows machine and of course cannot be automated.

However with guestfish (and the always brilliant SYSLINUX doing most of the heavy lifting), this script will unpack the ISO and turn it into a bootable USB key.

Notes:

  1. I am not going to support this script. You will need to read the script, look up the commands in the guestfish man page, and understand what it does. Any requests for help will be deleted unread.
  2. You need to edit the USB key device before using the script.
  3. You need libguestfs ≥ 1.21 with SYSLINUX support compiled in.
#!/bin/bash -

guestfish <<'EOF'
trace on

add-ro en_microsoft_hyper-v_server_2012_x64_dvd_915600.iso

# NB: The next line MUST be changed to your USB drive.
# ANYTHING ON THIS DRIVE WILL BE OVERWRITTEN WITHOUT WARNING.
add /dev/sdX

run

# Inside the appliance, /dev/sda = DVD, /dev/sdb = USB.
# THESE ARE NOT RELATED TO HOST DISK NAMES.

echo "Partitioning the USB disk ..."
part-init /dev/sdb mbr
part-add /dev/sdb p 63 -1
part-set-mbr-id /dev/sdb 1 0xb
part-set-bootable /dev/sdb 1 true
mkfs vfat /dev/sdb1

echo "Copying the contents of the DVD to the USB key ..."
mkmountpoint /cd
mkmountpoint /usb
mount /dev/sda /cd
mount /dev/sdb1 /usb
# XXX We should add cp-r command XXX
debug sh "cp -rP /sysroot/cd/* /sysroot/usb"
#glob cp-a /cd/* /usb
umount /cd
umount /usb
rmmountpoint /cd
rmmountpoint /usb

echo "Making the USB key bootable using SYSLINUX ..."
syslinux /dev/sdb1
mount /dev/sdb1 /
upload /usr/share/syslinux/chain.c32 /chain.c32
write /syslinux.cfg "DEFAULT windows\n\nLABEL windows\nCOM32 chain.c32\nAPPEND fs ntldr=/bootmgr\n"
umount /dev/sdb1
upload /usr/share/syslinux/mbr.bin /dev/sdb

echo "Finished."

EOF

2 Comments

Filed under Uncategorized

Using rsync with libguestfs

Since libguestfs 1.20 it has been possible to use rsync to upload or download files to a disk image incrementally. This is one way to do backups, but note that it won’t work on live guests unless you take a snapshot.

rsync involves using a network connection into or out of the appliance, and is therefore a lot more involved to set up. The script below shows one way to do this, by running an rsync daemon on the host, and letting the libguestfs appliance connect to it.

The script runs rsync inside the appliance, copying /home from the attached disk image out to /tmp/backup on the host. If the operation is repeated, then only incrementally changed files will be copied out. (To incrementally delete files on the target, add the deletedest:true flag).

Note you will need to open port 2999 on your host’s firewall for this to work.

#!/bin/bash -

set -x

# The target directory.
mkdir -p /tmp/backup

# Create the daemon.
rm -f /tmp/rsyncd.pid

cat <<EOF > /tmp/rsyncd.conf
port = 2999
pid file = /tmp/rsyncd.pid

[backup]
  path = /tmp/backup
  use chroot = false
  read only = false
EOF

rsync --daemon --config=/tmp/rsyncd.conf

# Run guestfish and attach to the guest.
guestfish --ro --network -a /dev/fedora/f19rawhidex32 -i <<EOF
trace on
rsync-out /home rsync://rjones@192.168.122.1:2999/backup archive:true
EOF

# Kill the rsync daemon.
kill `cat /tmp/rsyncd.pid`

2 Comments

Filed under Uncategorized

libguestfs and qemu ssh driver

The qemu ssh block device is now up to version 7 … although sadly not upstream yet.

Nevertheless by applying this patch to libguestfs you can use libguestfs to access remote disks over ssh:

$ export LIBGUESTFS_QEMU=~/d/qemu/qemu.wrapper
$ export LIBGUESTFS_BACKEND=direct
$ ./run ./fish/guestfish

Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.

Type: 'help' for help on commands
      'man' to read the manual
      'quit' to quit the shell

><fs> add /tmp/f17x64.img readonly:true format:raw \
        protocol:ssh server:onuma
><fs> run
 100% ⟦▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒⟧ 00:00
><fs> inspect-os
/dev/vg_f17x64/lv_root
><fs> inspect-get-product-name /dev/vg_f17x64/lv_root
Fedora release 17 (Beefy Miracle)
><fs> list-filesystems
/dev/sda1: ext4
/dev/vg_f17x64/lv_root: ext4
/dev/vg_f17x64/lv_swap: swap
><fs> mount /dev/vg_f17x64/lv_root /
><fs> cat /etc/redhat-release
Fedora release 17 (Beefy Miracle)

Everything just works as if this were a local disk.

There are a couple of minor caveats (the major caveat being none of this is upstream): Firstly you have to have ssh-agent set up. Secondly the remote host must be in your known_hosts file (if not, do ssh remotehost first to add it).

Leave a comment

Filed under Uncategorized

Cool new bash-completions of libguestfs tools

Starting in libguestfs ≥ 1.21.23-2, bash tab completions of guestfish, guestmount and virt-* tools have been rewritten and greatly improved.

Note you will need to install the libguestfs-bash-completion package to enable this feature.

You can now tab complete all long options on most tools:

$ virt-df --[tab]
--add             --domain          --human-readable  --uuid
--connect         --format          --inodes          --verbose
--csv             --help            --one-per-guest   --version
$ virt-resize --[tab]
--align-first          --help                 --no-extra-partition
--alignment            --ignore               --ntfsresize-force
--debug                --lvexpand             --output-format
--debug-gc             --lv-expand            --quiet
--delete               --LVexpand             --resize
--dryrun               --LV-expand            --resize-force
--dry-run              --machine-readable     --shrink
--expand               --no-copy-boot-loader  --version
--format               --no-expand-content    

Where appropriate, the -d option will now expand to the list of libvirt domains:

# virt-df -d [tab]
archlinux20121201x64  f19rawhidex32
f18x64                f19rawhidex64

Finally, guestfish commands are expanded on the command line:

$ guestfish add /tmp/disk : run : list-[tab]
list-9p              list-events          list-md-devices
list-devices         list-filesystems     list-partitions
list-disk-labels     list-ldm-partitions  
list-dm-devices      list-ldm-volumes     

To make this less intrusive, so you can really use it daily, I left the default readline expansions enabled. This means that filenames and so on can continue to be used in every position on the command line, and should mean that bash completions won’t try to be cleverer than the user.

Libguestfs bash completions are also demand-loaded now, so that if you’re not using them, they don’t consume any resources in the shell.

Leave a comment

Filed under Uncategorized

Accessing NBD disks using libguestfs

It’s always been possible, but clumsy, to access Network Block Device (NBD) disks from libguestfs, but starting in libguestfs 1.22 we hope to make this (and Gluster, Ceph and Sheepdog access) much simpler.

The first change is upstream in libguestfs 1.21.21. You can add an NBD disk directly.

To show this using guestfish, I’ll start an NBD server. This could be started on another machine, but to make things simple I’ll start this server on the same machine:

$ qemu-nbd f18x64.img -t

f18x64.img is the disk image that I want to export. The -t option makes the qemu-nbd server persistent (ie. it doesn’t just exit after serving the first client).

Now we can connect to this server using guestfish as follows:

$ guestfish

Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.

Type: 'help' for help on commands
      'man' to read the manual
      'quit' to quit the shell

><fs> add-drive "" format:raw protocol:nbd server:localhost
><fs> run

The empty "" (quotes) are the export name. Since qemu-nbd doesn’t support export names, we can leave this empty. The main change is to specify the protocol (nbd) and the server that libguestfs should connect to (localhost, but a remote host would also work). I haven’t specified a port number here because both the client and server are using the standard NBD port (10809), but you could use server:localhost:NNN to use a different port number if needed.

Ordinary guestfish commands just work:

><fs> list-filesystems
/dev/sda1: ext4
/dev/fedora/root: ext4
/dev/fedora/swap: swap
><fs> inspect-os
/dev/fedora/root
><fs> inspect-get-product-name /dev/fedora/root
Fedora release 18 (Spherical Cow)

The next steps are to:

  1. Add support for more technologies. At least: Gluster, Ceph, Sheepdog and iSCSI, since those are all supported by qemu so we can easily leverage that support in libguestfs.
  2. Change the guestfish -a option to make adding remote drives possible from the command line.

The obvious [but not yet implemented] way to change the -a option is to allow a URI to be specified. For example:

$ guestfish -a nbd://localhost/exportname

where the elements of the URI like protocol, transport, server, port number and export name translate naturally into parameters of the add-drive API.

Leave a comment

Filed under Uncategorized

guestfish now supports 502 commands

Just over 3 years ago I wrote that guestfish supported 267 commands and I listed them out.

Today, in libguestfs 1.21.19, guestfish has 502 commands, listed below.

    Command              Description
help                 display a list of commands or help on a command
quit                 quit guestfish
acl-delete-def-file  delete the default POSIX ACL of a directory
acl-get-file         get the POSIX ACL attached to a file
acl-set-file         set the POSIX ACL attached to a file
add-cdrom            add a CD-ROM disc image to examine
add-domain           add the disc(s) from a named libvirt domain
add-drive            add an image to examine or modify
add-drive-ro         add a drive in snapshot mode (read-only)
add-drive-ro-with-if add a drive read-only specifying the QEMU block emulation to use
add-drive-with-if    add a drive specifying the QEMU block emulation to use
alloc                allocate and add a disc file
aug-clear            clear Augeas path
aug-close            close the current Augeas handle
aug-defnode          define an Augeas node
aug-defvar           define an Augeas variable
aug-get              look up the value of an Augeas path
aug-init             create a new Augeas handle
aug-insert           insert a sibling Augeas node
aug-load             load files into the tree
aug-ls               list Augeas nodes under augpath
aug-match            return Augeas nodes which match augpath
aug-mv               move Augeas node
aug-rm               remove an Augeas path
aug-save             write all pending Augeas changes to disk
aug-set              set Augeas path to value
available            test availability of some parts of the API
available-all-groups return a list of all optional groups
base64-in            upload base64-encoded data to file
base64-out           download file and encode as base64
blkid                print block device attributes
blockdev-flushbufs   flush device buffers
blockdev-getbsz      get blocksize of block device
blockdev-getro       is block device set to read-only
blockdev-getsize64   get total size of device in bytes
blockdev-getss       get sectorsize of block device
blockdev-getsz       get total size of device in 512-byte sectors
blockdev-rereadpt    reread partition table
blockdev-setbsz      set blocksize of block device
blockdev-setro       set block device to read-only
blockdev-setrw       set block device to read-write
btrfs-device-add     add devices to a btrfs filesystem
btrfs-device-delete  remove devices from a btrfs filesystem
btrfs-filesystem-balance balance a btrfs filesystem
btrfs-filesystem-resize resize a btrfs filesystem
btrfs-filesystem-sync sync a btrfs filesystem
btrfs-fsck           check a btrfs filesystem
btrfs-set-seeding    enable or disable the seeding feature of device
btrfs-subvolume-create create a btrfs snapshot
btrfs-subvolume-delete delete a btrfs snapshot
btrfs-subvolume-list list btrfs snapshots and subvolumes
btrfs-subvolume-set-default set default btrfs subvolume
btrfs-subvolume-snapshot create a writable btrfs snapshot
canonical-device-name return canonical device name
cap-get-file         get the Linux capabilities attached to a file
cap-set-file         set the Linux capabilities attached to a file
case-sensitive-path  return true path on case-insensitive filesystem
cat                  list the contents of a file
checksum             compute MD5, SHAx or CRC checksum of file
checksum-device      compute MD5, SHAx or CRC checksum of the contents of a device
checksums-out        compute MD5, SHAx or CRC checksum of files in a directory
chmod                change file mode
chown                change file owner and group
command              run a command from the guest filesystem
command-lines        run a command, returning lines
compress-device-out  output compressed device
compress-out         output compressed file
config               add qemu parameters
copy-device-to-device copy from source device to destination device
copy-device-to-file  copy from source device to destination file
copy-file-to-device  copy from source file to destination device
copy-file-to-file    copy from source file to destination file
copy-in              copy local files or directories into an image
copy-out             copy remote files or directories out of an image
copy-size            copy size bytes from source to destination using dd
cp                   copy a file
cp-a                 copy a file or directory recursively
dd                   copy from source to destination using dd
debug                debugging and internals
debug-drives         debug the drives (internal use only)
debug-upload         upload a file to the appliance (internal use only)
delete-event         delete a previously registered event handler
device-index         convert device to index
df                   report file system disk space usage
df-h                 report file system disk space usage (human readable)
disk-format          detect the disk format of a disk image
disk-has-backing-file return whether disk has a backing file
disk-virtual-size    return virtual size of a disk
display              display an image
dmesg                return kernel messages
download             download a file to the local machine
download-offset      download a file to the local machine with offset and size
drop-caches          drop kernel page cache, dentries and inodes
du                   estimate file space usage
e2fsck               check an ext2/ext3 filesystem
e2fsck-f             check an ext2/ext3 filesystem
echo                 display a line of text
echo-daemon          echo arguments back to the client
edit                 edit a file
egrep                return lines matching a pattern
egrepi               return lines matching a pattern
equal                test if two files have equal contents
event                register a handler for an event or events
exists               test if file or directory exists
fallocate            preallocate a file in the guest filesystem
fallocate64          preallocate a file in the guest filesystem
fgrep                return lines matching a pattern
fgrepi               return lines matching a pattern
file                 determine file type
file-architecture    detect the architecture of a binary file
filesize             return the size of the file in bytes
filesystem-available check if filesystem is available
fill                 fill a file with octets
fill-dir             fill a directory with empty files
fill-pattern         fill a file with a repeating pattern of bytes
find                 find all files and directories
find0                find all files and directories, returning NUL-separated list
findfs-label         find a filesystem by label
findfs-uuid          find a filesystem by UUID
fsck                 run the filesystem checker
fstrim               trim free space in a filesystem
get-append           get the additional kernel options
get-attach-method    get the attach method
get-autosync         get autosync mode
get-cachedir         get the appliance cache directory
get-direct           get direct appliance mode flag
get-e2attrs          get ext2 file attributes of a file
get-e2generation     get ext2 file generation of a file
get-e2label          get the ext2/3/4 filesystem label
get-e2uuid           get the ext2/3/4 filesystem UUID
get-libvirt-requested-credential-challenge challenge of i'th requested credential
get-libvirt-requested-credential-defresult default result of i'th requested credential
get-libvirt-requested-credential-prompt prompt of i'th requested credential
get-libvirt-requested-credentials get list of credentials requested by libvirt
get-memsize          get memory allocated to the qemu subprocess
get-network          get enable network flag
get-path             get the search path
get-pgroup           get process group flag
get-pid              get PID of qemu subprocess
get-qemu             get the qemu binary
get-recovery-proc    get recovery process enabled flag
get-selinux          get SELinux enabled flag
get-smp              get number of virtual CPUs in appliance
get-tmpdir           get the temporary directory
get-trace            get command trace enabled flag
get-umask            get the current umask
get-verbose          get verbose mode
getcon               get SELinux security context
getxattr             get a single extended attribute
getxattrs            list extended attributes of a file or directory
glob                 expand wildcards in command
glob-expand          expand a wildcard path
grep                 return lines matching a pattern
grepi                return lines matching a pattern
grub-install         install GRUB 1
head                 return first 10 lines of a file
head-n               return first N lines of a file
hexdump              dump a file in hexadecimal
hexedit              edit with a hex editor
hivex-close          close the current hivex handle
hivex-commit         commit (write) changes back to the hive
hivex-node-add-child add a child node
hivex-node-children  return list of nodes which are subkeys of node
hivex-node-delete-child delete a node (recursively)
hivex-node-get-child return the named child of node
hivex-node-get-value return the named value
hivex-node-name      return the name of the node
hivex-node-parent    return the parent of node
hivex-node-set-value set or replace a single value in a node
hivex-node-values    return list of values attached to node
hivex-open           open a Windows Registry hive file
hivex-root           return the root node of the hive
hivex-value-key      return the key field from the (key, datatype, data) tuple
hivex-value-type     return the data type from the (key, datatype, data) tuple
hivex-value-utf8     return the data field from the (key, datatype, data) tuple
hivex-value-value    return the data field from the (key, datatype, data) tuple
initrd-cat           list the contents of a single file in an initrd
initrd-list          list files in an initrd
inotify-add-watch    add an inotify watch
inotify-close        close the inotify handle
inotify-files        return list of watched files that had events
inotify-init         create an inotify handle
inotify-read         return list of inotify events
inotify-rm-watch     remove an inotify watch
inspect-get-arch     get architecture of inspected operating system
inspect-get-distro   get distro of inspected operating system
inspect-get-drive-mappings get drive letter mappings
inspect-get-filesystems get filesystems associated with inspected operating system
inspect-get-format   get format of inspected operating system
inspect-get-hostname get hostname of the operating system
inspect-get-icon     get the icon corresponding to this operating system
inspect-get-major-version get major version of inspected operating system
inspect-get-minor-version get minor version of inspected operating system
inspect-get-mountpoints get mountpoints of inspected operating system
inspect-get-package-format get package format used by the operating system
inspect-get-package-management get package management tool used by the operating system
inspect-get-product-name get product name of inspected operating system
inspect-get-product-variant get product variant of inspected operating system
inspect-get-roots    return list of operating systems found by last inspection
inspect-get-type     get type of inspected operating system
inspect-get-windows-current-control-set get Windows CurrentControlSet of inspected operating system
inspect-get-windows-systemroot get Windows systemroot of inspected operating system
inspect-is-live      get live flag for install disk
inspect-is-multipart get multipart flag for install disk
inspect-is-netinst   get netinst (network installer) flag for install disk
inspect-list-applications get list of applications installed in the operating system
inspect-list-applications2 get list of applications installed in the operating system
inspect-os           inspect disk and return list of operating systems found
is-blockdev          test if block device
is-chardev           test if character device
is-config            is in configuration state
is-dir               test if a directory
is-fifo              test if FIFO (named pipe)
is-file              test if a regular file
is-lv                test if device is a logical volume
is-socket            test if socket
is-symlink           test if symbolic link
is-whole-device      test if a device is a whole device
is-zero              test if a file contains all zero bytes
is-zero-device       test if a device contains all zero bytes
isoinfo              get ISO information from primary volume descriptor of ISO file
isoinfo-device       get ISO information from primary volume descriptor of device
kill-subprocess      kill the qemu subprocess
launch               launch the qemu subprocess
lcd                  change working directory
lchown               change file owner and group
ldmtool-create-all   scan and create Windows dynamic disk volumes
ldmtool-diskgroup-disks return the disks in a Windows dynamic disk group
ldmtool-diskgroup-name return the name of a Windows dynamic disk group
ldmtool-diskgroup-volumes return the volumes in a Windows dynamic disk group
ldmtool-remove-all   remove all Windows dynamic disk volumes
ldmtool-scan         scan for Windows dynamic disks
ldmtool-scan-devices scan for Windows dynamic disks
ldmtool-volume-hint  return the hint field of a Windows dynamic disk volume
ldmtool-volume-partitions return the partitions in a Windows dynamic disk volume
ldmtool-volume-type  return the type of a Windows dynamic disk volume
lgetxattr            get a single extended attribute
lgetxattrs           list extended attributes of a file or directory
list-9p              list 9p filesystems
list-devices         list the block devices
list-disk-labels     mapping of disk labels to devices
list-dm-devices      list device mapper devices
list-events          list event handlers
list-filesystems     list filesystems
list-ldm-partitions  list all Windows dynamic disk partitions
list-ldm-volumes     list all Windows dynamic disk volumes
list-md-devices      list Linux md (RAID) devices
list-partitions      list the partitions
ll                   list the files in a directory (long format)
llz                  list the files in a directory (long format with SELinux contexts)
ln                   create a hard link
ln-f                 create a hard link
ln-s                 create a symbolic link
ln-sf                create a symbolic link
lremovexattr         remove extended attribute of a file or directory
ls                   list the files in a directory
ls0                  get list of files in a directory
lsetxattr            set extended attribute of a file or directory
lstat                get file information for a symbolic link
lstatlist            lstat on multiple files
luks-add-key         add a key on a LUKS encrypted device
luks-close           close a LUKS device
luks-format          format a block device as a LUKS encrypted device
luks-format-cipher   format a block device as a LUKS encrypted device
luks-kill-slot       remove a key from a LUKS encrypted device
luks-open            open a LUKS-encrypted block device
luks-open-ro         open a LUKS-encrypted block device read-only
lvcreate             create an LVM logical volume
lvcreate-free        create an LVM logical volume in % remaining free space
lvm-canonical-lv-name get canonical name of an LV
lvm-clear-filter     clear LVM device filter
lvm-remove-all       remove all LVM LVs, VGs and PVs
lvm-set-filter       set LVM device filter
lvremove             remove an LVM logical volume
lvrename             rename an LVM logical volume
lvresize             resize an LVM logical volume
lvresize-free        expand an LV to fill free space
lvs                  list the LVM logical volumes (LVs)
lvs-full             list the LVM logical volumes (LVs)
lvuuid               get the UUID of a logical volume
lxattrlist           lgetxattr on multiple files
man                  open the manual
max-disks            maximum number of disks that may be added
md-create            create a Linux md (RAID) device
md-detail            obtain metadata for an MD device
md-stat              get underlying devices from an MD device
md-stop              stop a Linux md (RAID) device
mkdir                create a directory
mkdir-mode           create a directory with a particular mode
mkdir-p              create a directory and parents
mkdtemp              create a temporary directory
mke2fs               create an ext2/ext3/ext4 filesystem on device
mke2fs-J             make ext2/3/4 filesystem with external journal
mke2fs-JL            make ext2/3/4 filesystem with external journal
mke2fs-JU            make ext2/3/4 filesystem with external journal
mke2journal          make ext2/3/4 external journal
mke2journal-L        make ext2/3/4 external journal with label
mke2journal-U        make ext2/3/4 external journal with UUID
mkfifo               make FIFO (named pipe)
mkfs                 make a filesystem
mkfs-b               make a filesystem with block size
mkfs-btrfs           create a btrfs filesystem
mklost-and-found     make lost+found directory on an ext2/3/4 filesystem
mkmountpoint         create a mountpoint
mknod                make block, character or FIFO devices
mknod-b              make block device node
mknod-c              make char device node
mkswap               create a swap partition
mkswap-L             create a swap partition with a label
mkswap-U             create a swap partition with an explicit UUID
mkswap-file          create a swap file
mktemp               create a temporary file
modprobe             load a kernel module
more                 view a file
mount                mount a guest disk at a position in the filesystem
mount-9p             mount 9p filesystem
mount-local          mount on the local filesystem
mount-local-run      run main loop of mount on the local filesystem
mount-loop           mount a file using the loop device
mount-options        mount a guest disk with mount options
mount-ro             mount a guest disk, read-only
mount-vfs            mount a guest disk with mount options and vfstype
mountpoints          show mountpoints
mounts               show mounted filesystems
mv                   move a file
nr-devices           return number of whole block devices (disks) added
ntfs-3g-probe        probe NTFS volume
ntfsclone-in         restore NTFS from backup file
ntfsclone-out        save NTFS to backup file
ntfsfix              fix common errors and force Windows to check NTFS
ntfsresize           resize an NTFS filesystem
ntfsresize-size      resize an NTFS filesystem (with size)
parse-environment    parse the environment and set handle flags accordingly
parse-environment-list parse the environment and set handle flags accordingly
part-add             add a partition to the device
part-del             delete a partition
part-disk            partition whole disk with a single primary partition
part-get-bootable    return true if a partition is bootable
part-get-gpt-type    get the type GUID of a GPT partition
part-get-mbr-id      get the MBR type byte (ID byte) from a partition
part-get-parttype    get the partition table type
part-init            create an empty partition table
part-list            list partitions on a device
part-set-bootable    make a partition bootable
part-set-gpt-type    set the type GUID of a GPT partition
part-set-mbr-id      set the MBR type byte (ID byte) of a partition
part-set-name        set partition name
part-to-dev          convert partition name to device name
part-to-partnum      convert partition name to partition number
ping-daemon          ping the guest daemon
pread                read part of a file
pread-device         read part of a device
pvchange-uuid        generate a new random UUID for a physical volume
pvchange-uuid-all    generate new random UUIDs for all physical volumes
pvcreate             create an LVM physical volume
pvremove             remove an LVM physical volume
pvresize             resize an LVM physical volume
pvresize-size        resize an LVM physical volume (with size)
pvs                  list the LVM physical volumes (PVs)
pvs-full             list the LVM physical volumes (PVs)
pvuuid               get the UUID of a physical volume
pwrite               write to part of a file
pwrite-device        write to part of a device
read-file            read a file
read-lines           read file as lines
readdir              read directories entries
readlink             read the target of a symbolic link
readlinklist         readlink on multiple files
realpath             canonicalized absolute pathname
remove-drive         remove a disk image
removexattr          remove extended attribute of a file or directory
rename               rename a file on the same filesystem
reopen               close and reopen libguestfs handle
resize2fs            resize an ext2, ext3 or ext4 filesystem
resize2fs-M          resize an ext2, ext3 or ext4 filesystem to the minimum size
resize2fs-size       resize an ext2, ext3 or ext4 filesystem (with size)
rm                   remove a file
rm-f                 remove a file ignoring errors
rm-rf                remove a file or directory recursively
rmdir                remove a directory
rmmountpoint         remove a mountpoint
rsync                synchronize the contents of two directories
rsync-in             synchronize host or remote filesystem with filesystem
rsync-out            synchronize filesystem with host or remote filesystem
scrub-device         scrub (securely wipe) a device
scrub-file           scrub (securely wipe) a file
scrub-freespace      scrub (securely wipe) free space
set-append           add options to kernel command line
set-attach-method    set the attach method
set-autosync         set autosync mode
set-cachedir         set the appliance cache directory
set-direct           enable or disable direct appliance mode
set-e2attrs          set ext2 file attributes of a file
set-e2generation     set ext2 file generation of a file
set-e2label          set the ext2/3/4 filesystem label
set-e2uuid           set the ext2/3/4 filesystem UUID
set-label            set filesystem label
set-libvirt-requested-credential pass requested credential back to libvirt
set-libvirt-supported-credentials set libvirt credentials supported by calling program
set-memsize          set memory allocated to the qemu subprocess
set-network          set enable network flag
set-path             set the search path
set-pgroup           set process group flag
set-qemu             set the qemu binary
set-recovery-proc    enable or disable the recovery process
set-selinux          set SELinux enabled or disabled at appliance boot
set-smp              set number of virtual CPUs in appliance
set-tmpdir           set the temporary directory
set-trace            enable or disable command traces
set-verbose          set verbose mode
setcon               set SELinux security context
setenv               set an environment variable
setxattr             set extended attribute of a file or directory
sfdisk               create partitions on a block device
sfdiskM              create partitions on a block device
sfdisk-N             modify a single partition on a block device
sfdisk-disk-geometry display the disk geometry from the partition table
sfdisk-kernel-geometry display the kernel geometry
sfdisk-l             display the partition table
sh                   run a command via the shell
sh-lines             run a command via the shell returning lines
shutdown             shutdown the qemu subprocess
sleep                sleep for some seconds
sparse               create a sparse disk image and add
stat                 get file information
statvfs              get file system statistics
strings              print the printable strings in a file
strings-e            print the printable strings in a file
supported            list supported groups of commands
swapoff-device       disable swap on device
swapoff-file         disable swap on file
swapoff-label        disable swap on labeled swap partition
swapoff-uuid         disable swap on swap partition by UUID
swapon-device        enable swap on device
swapon-file          enable swap on file
swapon-label         enable swap on labeled swap partition
swapon-uuid          enable swap on swap partition by UUID
sync                 sync disks, writes are flushed through to the disk image
tail                 return last 10 lines of a file
tail-n               return last N lines of a file
tar-in               unpack tarfile to directory
tar-out              pack directory into tarfile
tgz-in               unpack compressed tarball to directory
tgz-out              pack directory into compressed tarball
time                 print elapsed time taken to run a command
touch                update file timestamps or create a new file
truncate             truncate a file to zero size
truncate-size        truncate a file to a particular size
tune2fs              adjust ext2/ext3/ext4 filesystem parameters
tune2fs-l            get ext2/ext3/ext4 superblock details
txz-in               unpack compressed tarball to directory
txz-out              pack directory into compressed tarball
umask                set file mode creation mask (umask)
umount               unmount a filesystem
umount-all           unmount all filesystems
umount-local         unmount a locally mounted filesystem
unsetenv             unset an environment variable
upload               upload a file from the local machine
upload-offset        upload a file from the local machine with offset
user-cancel          cancel the current upload or download operation
utimens              set timestamp of a file with nanosecond precision
utsname              appliance kernel version
version              get the library version number
vfs-label            get the filesystem label
vfs-type             get the Linux VFS type corresponding to a mounted device
vfs-uuid             get the filesystem UUID
vg-activate          activate or deactivate some volume groups
vg-activate-all      activate or deactivate all volume groups
vgchange-uuid        generate a new random UUID for a volume group
vgchange-uuid-all    generate new random UUIDs for all volume groups
vgcreate             create an LVM volume group
vglvuuids            get the LV UUIDs of all LVs in the volume group
vgmeta               get volume group metadata
vgpvuuids            get the PV UUIDs containing the volume group
vgremove             remove an LVM volume group
vgrename             rename an LVM volume group
vgs                  list the LVM volume groups (VGs)
vgs-full             list the LVM volume groups (VGs)
vgscan               rescan for LVM physical volumes, volume groups and logical volumes
vguuid               get the UUID of a volume group
wc-c                 count characters in a file
wc-l                 count lines in a file
wc-w                 count words in a file
wipefs               wipe a filesystem signature from a device
write                create a new file
write-append         append content to end of file
write-file           create a file
xfs-admin            change parameters of an XFS filesystem
xfs-growfs           expand an existing XFS filesystem
xfs-info             get geometry of XFS filesystem
xfs-repair           repair an XFS filesystem
zegrep               return lines matching a pattern
zegrepi              return lines matching a pattern
zero                 write zeroes to the device
zero-device          write zeroes to an entire device
zero-free-space      zero free space in a filesystem
zerofree             zero unused inodes and disk blocks on ext2/3 filesystem
zfgrep               return lines matching a pattern
zfgrepi              return lines matching a pattern
zfile                determine file type inside a compressed file
zgrep                return lines matching a pattern
zgrepi               return lines matching a pattern
    Use -h  / help  to show detailed help for a command.

2 Comments

Filed under Uncategorized

Accessing Ceph (rbd), sheepdog, etc using libguestfs

Thanks to infernix who contributed this tip on how to use libguestfs to access Ceph (and in theory, sheepdog, gluster, iscsi and more) devices.

If you apply this small patch to libguestfs you can use these distributed filesystems straight away by doing:

$ guestfish
><fs> set-attach-method appliance
><fs> add-drive /dev/null
><fs> config -set drive.hd0.file=rbd:pool/volume
><fs> run

… followed by usual guestfish commands.

This is a temporary hack, until we properly model Ceph (etc) through the libguestfs stable API. Nevertheless it works as follows:

  1. The add-drive /dev/null adds a drive, known to libguestfs.
  2. Implicitly this means that libguestfs adds a -drive option when it runs qemu.
  3. The custom qemu -set drive.hd0.file=... parameter modifies the preceding -drive option added by libguestfs so that the file is changed from /dev/null to whatever you want. In this case, to a Ceph rbd:... path.

3 Comments

Filed under Uncategorized