We ported nbdkit to Windows. That port is now upstream and should appear in the next stable release (1.24). There is also a new native file plugin for Windows which supports Windows files and volumes, hole punching for sparse files, querying file sparseness, and efficient zeroing.
Tag Archives: windows
nbdkit Windows port contd.
nbdkit now ported to Windows
This week I ported nbdkit, our high performance plugin-based Network Block Device server, to Windows. Currently it’s not upstream but you can download the Windows branch from here.
There were several possible ways we could have done this including Cygwin which might have been easier, but in the end I did a port to the raw Win32/Winsock APIs. You can compile it on Fedora using the mingw-w64-based Fedora Windows Cross Compiler and run it using Wine. Familiar commands like this work:
$ ./nbdkit.exe -f -v memory 1G
Windows is such a trash pile of awful APIs it’s a wonder how anyone can use it. Like what, and huh and WTF having to split a 64 bit int across two fields in a struct?? Not to mention the whole mess which is HANDLEs vs SOCKETs vs file descriptors and errno handling in Winsock. But I got there in the end.
I got many existing plugins and filters compiled. Not all of those listed will be working, but the main features are fine. You can also write your own plugins to the same API as Linux ones. I’m hoping that someone can write a Windows block device plugin (especially one which integrates with features like VSS).
$ find \( -name '*.exe' -o -name '*.dll' \) -a -printf "%f\n" | sort -u nbdkit-blocksize-filter.dll nbdkit-cacheextents-filter.dll nbdkit-cow-filter.dll nbdkit-data-plugin.dll nbdkit-ddrescue-filter.dll nbdkit-delay-filter.dll nbdkit-error-filter.dll nbdkit-example1-plugin.dll nbdkit.exe nbdkit-exitlast-filter.dll nbdkit-extentlist-filter.dll nbdkit-file-plugin.dll nbdkit-fua-filter.dll nbdkit-full-plugin.dll nbdkit-gzip-filter.dll nbdkit-gzip-plugin.dll nbdkit-info-plugin.dll nbdkit-ip-filter.dll nbdkit-limit-filter.dll nbdkit-memory-plugin.dll nbdkit-nocache-filter.dll nbdkit-noextents-filter.dll nbdkit-nofilter-filter.dll nbdkit-noparallel-filter.dll nbdkit-nozero-filter.dll nbdkit-null-plugin.dll nbdkit-offset-filter.dll nbdkit-partition-filter.dll nbdkit-partitioning-plugin.dll nbdkit-pattern-plugin.dll nbdkit-random-plugin.dll nbdkit-rate-filter.dll nbdkit-readahead-filter.dll nbdkit-retry-filter.dll nbdkit-split-plugin.dll nbdkit-stats-filter.dll nbdkit-swab-filter.dll nbdkit-tls-fallback-filter.dll nbdkit-truncate-filter.dll nbdkit-zero-plugin.dll
Filed under Uncategorized
Creating Windows templates for virt-builder
virt-builder is a tool for rapidly creating customized Linux images. Recently I’ve added support for Windows although for rather obvious licensing reasons we cannot distribute the Windows templates which would be needed to provide Windows support for everyone. However you can build your own Windows templates as described here and then:
$ virt-builder -l | grep windows windows-10.0-server x86_64 Windows Server 2016 (x86_64) windows-6.2-server x86_64 Windows Server 2012 (x86_64) windows-6.3-server x86_64 Windows Server 2012 R2 (x86_64) $ virt-builder windows-6.3-server [ 0.6] Downloading: http://xx/builder/windows-6.3-server.xz [ 5.1] Planning how to build this image [ 5.1] Uncompressing [ 60.1] Opening the new disk [ 77.6] Setting a random seed virt-builder: warning: random seed could not be set for this type of guest virt-builder: warning: passwords could not be set for this type of guest [ 77.6] Finishing off Output file: windows-6.3-server.img Output size: 10.0G Output format: raw Total usable space: 9.7G Free space: 3.5G (36%)
To build a Windows template repository you will need the latest libguestfs sources checked out from https://github.com/libguestfs/libguestfs and you will also need a suitable Windows Volume License, KMS or MSDN developer subscription. Also the final Windows templates are at least ten times larger than Linux templates, so virt-builder operations take correspondingly longer and use lots more disk space.
First download install ISOs for the Windows guests you want to use.
After cloning the latest libguestfs sources, go into the
builder/templates subdirectory. Edit the top of the
make-template.ml script to set the path which contains the Windows ISOs. You will also possibly need to edit the names of the ISOs later in the script.
Build a template, eg:
$ ../../run ./make-template.ml windows 2k12 x86_64
You’ll need to read the script to understand what the arguments do. The script will ask you for the product key, where you should enter the volume license key or your MSDN key.
Each time you run the script successfully you’ll end up with two files called something like:
The version numbers are Windows internal version numbers.
After you’ve created templates for all the Windows guest types you need, copy them to any (private) web server, and concatenate all the index fragments into the final index file:
$ cat *.index-fragment > index
Finally create a virt-builder repo file pointing to this index file:
# cat /etc/virt-builder/repos.d/windows.conf [windows] uri=http://xx/builder/index
You can now create Windows guests in virt-builder. However note they are not sysprepped. We can’t do this because it requires some Windows tooling. So while these guests are good for small tests and similar, they’re not suitable for creating actual Windows long-lived VMs. To do that you will need to add a sysprep.exe step somewhere in the template creation process.
Tip: virt-install Windows with virtio device drivers
You have to unset these variables because of a long-standing bug in SPICE:
# unset http_proxy # unset https_proxy
You can’t use virt-install’s
--cdrom option twice, because virt-install ignores the second use of the option and only adds a single CD-ROM to the guest. Instead, use
# virt-install --name=w81-virtio --ram=4096 \ --cpu=host --vcpus=2 \ --os-type=windows --os-variant=win8.1 \ --disk /dev/VG/w81-virtio,bus=virtio \ --disk en-gb_windows_8.1_pro_n_vl_with_update_x64_dvd_6050975.iso,device=cdrom,bus=ide \ --disk /usr/share/virtio-win/virtio-win.iso,device=cdrom,bus=ide
During the install you’ll have to select the “Load driver” option and load the right viostor driver from the second CD-ROM (E:).
Filed under Uncategorized
New(ish) in libguestfs 1.27.23 — add firstboot batch files to Windows guests
You’ve been able to do this for a while by hand but now virt-sysprep & virt-customize ≥ 1.27.23 let you easily install firstboot scripts into Windows guests:
$ cat /tmp/test.bat echo Hello I am a batch file $ virt-customize -a win7.qcow2 --firstboot /tmp/test.bat
Next time the guest boots, check the log file in
C:\Program Files\Red Hat\Firstboot\log.txt
This works well for me in Windows 7 guests. It ought to work in other Windows guests too. So far the only other Windows flavour I tested was W2K3 where the service crashed for some unfathomable reason (I’m not very patient with debugging Windows problems).
So let us know how it goes and we’ll try to fix the bugs as we go along.
virt-log now supports the Windows Event Log
New virt tool virt-log now supports the Windows Event Log. If you have a recent Windows guest you can display the System event log by doing:
$ virt-log -d Win8 | less
What you will see is a very long XML file.
This requires an Evtx parser. I have now chosen this library for Fedora (it needs a reviewer, as you can see). The code is sensible and maintained.
It also only works for Windows ≥ Vista, because Microsoft completely rewrote the way that log files are stored, from one strange binary format to another strange binary format [so a little different from the systemd journal …].
As usual, patches to virt-log to support other guest operating systems are welcome.
Tip: Convert a Windows DVD ISO to a bootable USB key using guestfish
Situation: You have a Windows DVD (or ISO), but like any sane person in 2013 you don’t have a DVD drive on the computer. You want to convert the Windows DVD into a bootable USB key. There are many recipes for this online, but they all require another Windows machine and of course cannot be automated.
However with guestfish (and the always brilliant SYSLINUX doing most of the heavy lifting), this script will unpack the ISO and turn it into a bootable USB key.
- I am not going to support this script. You will need to read the script, look up the commands in the guestfish man page, and understand what it does. Any requests for help will be deleted unread.
- You need to edit the USB key device before using the script.
- You need libguestfs ≥ 1.21 with SYSLINUX support compiled in.
#!/bin/bash - guestfish <<'EOF' trace on add-ro en_microsoft_hyper-v_server_2012_x64_dvd_915600.iso # NB: The next line MUST be changed to your USB drive. # ANYTHING ON THIS DRIVE WILL BE OVERWRITTEN WITHOUT WARNING. add /dev/sdX run # Inside the appliance, /dev/sda = DVD, /dev/sdb = USB. # THESE ARE NOT RELATED TO HOST DISK NAMES. echo "Partitioning the USB disk ..." part-init /dev/sdb mbr part-add /dev/sdb p 63 -1 part-set-mbr-id /dev/sdb 1 0xb part-set-bootable /dev/sdb 1 true mkfs vfat /dev/sdb1 echo "Copying the contents of the DVD to the USB key ..." mkmountpoint /cd mkmountpoint /usb mount /dev/sda /cd mount /dev/sdb1 /usb # XXX We should add cp-r command XXX debug sh "cp -rP /sysroot/cd/* /sysroot/usb" #glob cp-a /cd/* /usb umount /cd umount /usb rmmountpoint /cd rmmountpoint /usb echo "Making the USB key bootable using SYSLINUX ..." syslinux /dev/sdb1 mount /dev/sdb1 / upload /usr/share/syslinux/chain.c32 /chain.c32 write /syslinux.cfg "DEFAULT windows\n\nLABEL windows\nCOM32 chain.c32\nAPPEND fs ntldr=/bootmgr\n" umount /dev/sdb1 upload /usr/share/syslinux/mbr.bin /dev/sdb echo "Finished." EOF
A quick note about 64 bit Fedora Windows cross-compiler
I don’t often talk about the Fedora Windows cross-compiler project which I guess I helped to start back in 2008.
However last week some legal uncertainty about the 64 bit compiler was lifted and Erik van Pienbroek and Kalev Lember have been doing an amazing job getting a full 32 + 64 bit cross-compiler chain into Fedora 17.
The upshot of this is incredible: From a single code-base, you can build, on Fedora, a Linux binary, 32- and 64- bit Windows binaries, and (with a tiny bit of proprietary code) Mac OS X binaries. This includes cross-platform graphical programs (using Gtk), and over a hundred libraries for graphics, sound, games, networking, databases and much more. It’s not just plain binaries either, mingw-w64 includes a credible DDK replacement so you can build Windows device drivers too.
Android is the new Windows …
You want to know if Android or iPhone is going to “win” in the end? It’s obviously going to be Android, because it’s the new Windows.
Exhibit (a): My Google Nexus S phone: requires rebooting every 3 days otherwise it just stops being able to receive/send calls and/or texts. Data over 3g still works during these outages.
Exhibit (b): My new Samsung Galaxy tab, a Christmas present of sorts from Red Hat: Fresh out of the box(!), this cannot connect to my wifi (absolutely everything else can connect fine to this wifi). Support forums are full of people reporting the same thing, and everyone suggesting you reboot/reinstall/press some magic key sequence/etc. Where have I heard that story before?
Filed under Uncategorized
libguestfs 1.12.6 for Debian
Thanks to the tireless work of Hilko Bengen, libguestfs 1.12.6 is now available as an official Debian package.
Also, you can compile hivex on Mac OS X and Windows, thanks to Alex Nelson and Gillen Daniel respectively.