After someone asked me a question about “balloons” (in the virtualization sense) today, I noticed that there is not very much documentation around. This post explains what the KVM virtio_balloon driver is all about.
First of all, what is a balloon driver if you’ve never even heard of the concept? It’s a way to give or take RAM from a guest. (In theory at least), if your guest needs more RAM, you can use the balloon driver to give it more RAM. Or if the host needs to take RAM away from guests, it can do so. All of this is done without needing to pause or reboot the guest.
You might think that this would work as a RAM “hot add” feature, rather like hot adding disks to a guest. Although RAM hot add would (IMHO) be much better, currently this is not how ballooning works.
What we have is a kernel driver inside the guest called virtio_balloon. This driver acts like a kind of weird process, either expanding its own memory usage or shrinking down to nearly nothing, as in the diagrams below:
When the balloon driver expands, normal applications running in the guest suddenly have a lot less memory and the guest does the usual things it does when there’s not much memory, including swapping stuff out and starting up the OOM killer. (The balloon itself is non-swappable and un-killable in case you were wondering).
So what’s the point of a kernel driver which wastes memory? There are two points: Firstly, the driver communicates with the host (over the virtio channel), and the host gives it instructions (“expand to this size”, “shrink down now”). The guest cooperates, but doesn’t directly control the balloon.
Secondly, memory pages in the balloon are unmapped from the guest and handed back to the host, so the host can hand them out to other guests. It’s like the guest’s memory has a chunk missing from it:
Libvirt has two settings you can control called currentMemory and maxMemory (“memory” in the libvirt XML):
maxMemory (or just <memory>) is the memory allocated at boot time to a guest. KVM and Xen guests currently cannot exceed this. currentMemory controls what memory you’re requesting to give to the guest’s applications. The balloon fills the rest of the memory and gives it back to the host for the host to use elsewhere.
You can adjust this manually for your guests, either by editing the XML, or by using the
virsh setmem command.
22 responses to “Virtio balloon”
Thanks for the explanation.
in currentMemory definition: “If this is omitted, it defaults to the same value as the memory element”
Does this mean that by default (memory=currentMemory) there is no balloon at all for a VM?
Does this mean that if I want for example to “reserve” (or grant) at least 512MB but not all of my 2Gb ram defined VM for its applications, I can set currentMemory to 512Mb and so give more memory available for other VMs (1,5Gb in this case)?
Is the currentMemory parameter dynamically adjustable?
The memory ballooning driver for Windows 7/2008 may not work because Windows automatically recognizes it as “PCI standard RAM Controller”. You can look at the properties of this system device and under details make sure that “Location information” shows the same PCI device that kvm/qemu set for the memory ballooning driver (in my case, it is PCI bus 0, device 5, function 0).
Be sure to go back to the Driver tab and update your driver to the actual balloon driver.
Thanks for a great explanation, Richard.
Thank ya Richard! Excellent explanation.
Pingback: Virtio Balloon in action — with native linux kvm tool | Kashyap Chamarthy
Hi can you please tell me the correct procedure to use a virtioballoon
I in Linux . Is there any command that need to be executed between guest or host in order for it to work .
Does this need something special running *on the host* to function?
Pingback: KVM/Xen and libvirt: currentMemory, memory and ballooning. Where did my memory go? | golb
Pingback: Windows 7 in QEMU/KVM « Just moozing
Pingback: Adeus chroot… seja bem vindo LXC… | Vinipsmaker labs
Pingback: Wywiad z Pawłem Dziepakiem – deweloperem Haiku | OSWorld.pl
Pingback: Drivers VirtIO para guests Windows en KVM | Pollo sin Cabeza
Pingback: Jugar en linux con Steam en el mejor escenario (Parte 2) – mi linux blog
Excellent explanation. What happens to the processes that were using the memory when the hypervisor decides to steal memory from the guest operating system?
For example, let’s say a guest OS has 4gb of RAM and MySQL is using half of the available memory 2gb and the hypervisor decides its going to reclaim 3gb of memory from the guest OS? Does the OOM killer kill the MySQL process?
The hypervisor doesn’t just take the memory away, it has to cooperate with the guest to expand the balloon which “pushes” all the running processes into the remaining space. Of course if, as a result, there’s not enough room to run whatever jobs were running before then something has to give, and for Linux guests that would probably involve the OOM killer.
If the balloon is enabled, will it ever do anything if the total of memory configured for all VMs is less than the total ram on the physical host? i.e is memory overcommit a requirement for the balloon to be activated? I have it configured, but I am not overcommitted, I am wondering if it will noop.
I also noticed that there is a vballoon kernel thread on the CentOS6 guests, but not on the CentOS7 guest despite having the same VM params passed to KVM.. What gives?
it really depends on the target use case / how it is being used in the hypervisor. If memory ballooning is used to optimize memory overcommit in the hypervisor (e.g., avoid swapping), then you would expect ballooning activity only get active when there is real memory pressure in the hypervisor.
If memory ballooning is used to speed up (live) migration of VMs, then you could see some temporary ballooning activity.
If memory ballooning is used for dynamic resizing of VMs (e.g., cloud user wants to optimize costs by requesting more/less memory for a VM), then you can expect memory ballooning even without memory overcommit.
So, it pretty much boils down to the target use case.
Pingback: Virtual DragonFly: memory balloon – DragonFly BSD Digest
i just stumbled across this. Nice job. One question: is this “balooning” done automatically by the guest/hypervisor/host ? Or has it to be adjusted manually ? Great if someone may clarify that.
in most hypervisors, including QEMU/KVM with virtio-balloon, the hypervisor specifies the target size of the balloon for the guest. The guest will try to fulfill this request from the hypervisor by inflating or deflating the balloon. Depending on the hypervisor, the target balloon size is either configured manually by a user, or automatically by some monitoring mechanism. In case of QEMU/KVM, it’s usually configured manually by the user, however I remember there are options in openstack to monitor the VMs and adjust the balloon size accordingly automatically.
Pingback: virtio-mem goes to Windows – Our Windows Man