Boost performance of KVM guests
Download this article as a single PDF document
Contents
Software
Software used to write this howto:
- Qemu 0.14.1
- Linux 3.0.0
- KVM Host: - Debian Wheezy
- KVM Guests: Debian Squeeze
Introduction
In this post I would like to share two tips that will greatly improve the performance of KVM guests.
I was testing I/O performance of my KVM guests performing a simple task of scping a large file from the host to the KVM guests.
I was really disappointed discovering the throughput was less than satisfactory with speed dropping down to 3MB/s.
The guests are installed directly to LVM logical volumes configured on an RAID 6 array of speedy Serial ATA-600 drives which should give me way more than the lousy result I was experiencing.
The tests I performed on the KVM host itself showed a much higher transfer speed.
With these tips you should notice an immediate improvement in performance including boot times, general performance, and responsiveness to inputs (mouse / keyboard).
Tweaking
Change the KVM guest configuration
One of the first things I checked was the definition of the disk for my KVM guests.
Running
# virsh edit kvm_guest
would show the XML configuration.
It turned out disk caching was enabled, which is the default setting.
From man qemu :
By default, writethrough caching is used for all block device. This means that the host page cache will be used to read and write data but write notification will be sent to the guest only when the data has been reported as written by the storage subsystem. Writeback caching will report data writes as completed as soon as the data is present in the host page cache. This is safe as long as you trust your host. If your host crashes or loses power, then the guest may experience data corruption. When using the -snapshot option, writeback caching is used by default. The host page can be avoided entirely with cache=none. This will attempt to do disk IO directly to the guests memory. QEMU may still perform an internal copy of the data. Some block drivers perform badly with cache=writethrough, most notably, qcow2. If performance is more important than correctness, cache=writeback should be used with qcow2. By default, if no explicit caching is specified for a qcow2 disk image, cache=writeback will be used. For all other disk types, cache=writethrough is the default.
So either using writeback or disabling the cache will improve performance.
This is the relevant part of my config file:
<disk type='block' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source dev='/dev/kvm/host01'/>
<target dev='vda' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</disk>
Changing the scheduler
By default the disk scheduler is set to cfq. You can check that value running following command:
# for f in /sys/block/sd*/queue/scheduler; do cat $f; done
To change the scheduler to noop run following command:
# for f in /sys/block/sd*/queue/scheduler; do echo "noop" > $f; done
As a permanent solution you need to pass what scheduler to use to the Linux kernel. Using any editor, open
/etc/default/grub
Look for the line :
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
Add elevator=noop at the end of the line
GRUB_CMDLINE_LINUX_DEFAULT=”quiet splash elevator=noop”
Save your changes and exit your editor and update grub configuration running:
# update-grub
The scheduler will be now automatically set to noop after each reboot.
Enable KSM - Kernel SamePage Merging
Kernel Samepage Merging (KSM) is a feature of the Linux kernel introduced in the 2.6.32 kernel.
KSM allows for an application to register with the kernel to have its pages merged with other processes that also register to have their pages merged.
For KVM, the KSM mechanism allows for guest virtual machines to share pages with each other.
In an environment where many of the guest operating systems are similar, this can result in significant memory savings.
You can turn KSM on or off by echoing a 1 or 0 to /sys/kernel/mm/ksm/run.
# echo 1 > /sys/kernel/mm/ksm/run
If KSM is running, and there are pages to be merged (i.e. more than one similar VM is running), then /sys/kernel/mm/ksm/pages_shared should be non-zero.
From the kernel documentation in Documentation/vm/ksm.txt
The effectiveness of KSM and MADV_MERGEABLE is shown in /sys/kernel/mm/ksm/: pages_shared - how many shared unswappable kernel pages KSM is using pages_sharing - how many more sites are sharing them i.e. how much saved pages_unshared - how many pages unique but repeatedly checked for merging pages_volatile - how many pages changing too fast to be placed in a tree full_scans - how many times all mergeable areas have been scanned A high ratio of pages_sharing to pages_shared indicates good sharing, but a high ratio of pages_unshared to pages_sharing indicates wasted effort. pages_volatile embraces several different kinds of activity, but a high proportion there would also indicate poor use of madvise MADV_MERGEABLE.
An easy way to see how well KSM is performing is to simply print the contents of all the files in that directory.
# for ii in /sys/kernel/mm/ksm/* ; do echo -n "$ii: " ; cat $ii ; done
/sys/kernel/mm/ksm/full_scans: 1 /sys/kernel/mm/ksm/pages_shared: 2568 /sys/kernel/mm/ksm/pages_sharing: 24136 /sys/kernel/mm/ksm/pages_to_scan: 100 /sys/kernel/mm/ksm/pages_unshared: 71010 /sys/kernel/mm/ksm/pages_volatile: 732105 /sys/kernel/mm/ksm/run: 1 /sys/kernel/mm/ksm/sleep_millisecs: 20
That's all folks.
Marcin
<comments />
Ansari said ...
<comment date="2012-04-17T02:36:24Z" name="Ansari">
Hooooo! yes, it looks like my Hackintosh is booting up etfasr. It takes now 40sec from Chimera to Lion, before it was like around 1m4sec.Thanks so much for sharing this knowledge. bless.by the way, i have a question maybe someone can explane me. in Multibeast 4.1 there is a option about "Generale CPU States", what does this mean? please i realy would like to know.. cheers Intel i7 960 Quad Core 3.2GHzGigabyte x58a-udr3 rev.2Gigabyte GTS 450 OC nVidia6GB Corsair Ram
</comment>
