[lttng-dev] View for virtual machine monitoring

[Mohamad Gebai]

Hello,
We are currently working on a new view in Eclipse's TMF plugin (Tracing and
Monitoring Framework) specific to virtual machine analysis. This view requires
kernel traces from the host and from each guest with a set of specific
tracepoints activated. The traces are then merged together and analysed in a
way that the real state of each system can be rebuilt, while taking into
account all the interactions between the different systems.

The main purpose of this view is to easily point out latency problems due to
resource sharing. For now, we only consider CPU time, but more resources (such
as memory allocation, disks...) will be added.

Two screenshots are attached. The first one shows the virtual machines and the
state of their respective virtual CPUs. The second screenshot gives in-depth
information about one of the virtual CPUs, showing only the threads that
interacted with this vCPU and their state during the time of the trace. We
think that this approach of showing information across the layers (OS, KVM,
guest OS, and eventually JVM...) can be helpful to investigate latency-related
problems specific to virtual machines.

Legend:
Green: user mode
Blue: kernel mode
Yellow: process blocked
Purple: vCPU preempted
Grey: vCPU idle

For the sake of our experience, we pinned vCPU0 of VM1 and vCPU0 of VM2 on the
same physical CPU, and ran a CPU-intensive workload for one second one each one
of them. We generated our traces using the low-overhead LTTng tracer. We can
clearly see that during that second, both of the virtual CPUs are fighting over
the same physical CPU.

We seek any thoughts or suggestions on the effectiveness of this view or on our
approach. Any real life problems waiting for investigation are also welcome.

Mohamad Gebai
-------------- next part --------------
A non-text attachment was scrubbed...
Name: screenshot_1.png
Type: image/png
Size: 36963 bytes
Desc: not available
URL: <http://lists.lttng.org/pipermail/lttng-dev/attachments/20130709/043f82ca/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: screenshot_2.png
Type: image/png
Size: 86040 bytes
Desc: not available
URL: <http://lists.lttng.org/pipermail/lttng-dev/attachments/20130709/043f82ca/attachment-0003.png>