Brussels / 31 January & 1 February 2015


Observability in KVM

How to troubleshoot virtual machines

KVM is the most popular hypervisor deployed with OpenStack and is also often used with libvirt in non-cloud scenarios. With multiple layers of software between the user and the virtual machine, how does one troubleshoot failures and performance issues?

This presentation covers tools and techniques for observing virtual machines with the KVM hypervisor. It gives you a mental model of KVM's architecture so you'll know how to get to the bottom of questions about virtual machine behavior and performance.

Areas covered include guest CPU activity, RAM, disk I/O, network traffic, and the QEMU monitor. Examples are based on real-life scenarios often encountered by KVM users.

The KVM hypervisor is part of the Linux kernel, making it an easy and popular way to run virtual machines on Linux hosts. KVM is also the most well supported and popular hypervisor deployed with OpenStack, typically through the libvirt library.

When something goes wrong with a virtual machine there are often questions like "how do I see the network packets that the host is receiving for this virtual machine?" or "can I find out which virtual machine is causing excessive disk I/O?".

Luckily KVM is not a black box, it's an open source hypervisor that runs together with the userspace QEMU emulator. There are a wealth of tools for observing virtual machine activity and they can be used to troubleshoot failures and performance issues. From packet sniffing to CPU profilers, tracing to debuggers, this talk covers the arsenal of tools and how to apply them.

This presentation arms you with knowledge of KVM's architecture and how to observe virtual machine activity. Monitoring and debugging virtual machines is actually much like (and sometimes even easier than) on physical machines so you can reuse many familiar tools.

Once you understand how virtual machines interact with the outside world, how CPU, RAM, disk I/O, and network resources are connected, investigating issues with virtual machines becomes a systematic process. This presentation includes examples of common questions you can ask with the help of performance monitoring and debugging tools.

This talk is suitable for both users of KVM virtual machines and developers of management software like libvirt, OpenStack, or oVirt. The aim is share knowledge gained from real-world cases so that virtual machines are not just a joy to use, but also a powerful vehicle for observability.


Stefan Hajnoczi