[lttng-dev] [Qemu-devel] [PATCH 0/6] hypertrace: Lightweight guest-to-QEMU trace channel

[Lluís Vilanova]

Stefan Hajnoczi writes:

> On Fri, Aug 05, 2016 at 06:59:23PM +0200, Lluís Vilanova wrote:
>> The hypertrace channel allows guest code to emit events in QEMU (the host) using
>> its tracing infrastructure (see "docs/trace.txt"). This works in both 'system'
>> and 'user' modes. That is, hypertrace is to tracing, what hypercalls are to
>> system calls.
>> 
>> You can use this to emit an event on both guest and QEMU (host) traces to easily
>> synchronize or correlate them. You could also modify you guest's tracing system
>> to emit all events through the hypertrace channel, providing a unified and fully
>> synchronized trace log. Another use case is timing the performance of guest code
>> when optimizing TCG (QEMU traces have a timestamp).
>> 
>> See first commit for a full description.

> This tracing approach has a high performance overhead, particularly for
> SMP guests where each trace event requires writing to the global control
> register.  All CPUs will be hammering this register (heavyweight vmexit)
> for each trace event.

> I think the folks CCed on this email all take an asynchronous approach
> to avoid this performance overhead.  Synchronous means taking a VM exit
> for every event.  Asynchronous means writing trace data to a buffer and
> later interleaving guest data with host trace data.

> LTTng Userspace Tracer is an example of the asynchronous approach.  The
> trace data buffers are in shared memory.  The LTTng process can grab
> buffers at appropriate times.

> The ftrace virtio-serial approach has been to splice() the ftrace
> buffers, resulting in efficient I/O.

> Steven is working on a host/guest solution for trace-cmd.  It is also
> asynchronous.  No new paravirt hardware is needed and it makes me wonder
> whether the hypertrace PCI device is trying to solve the problem at the
> wrong layer.

> If you want to play around with asynchronous tracing, you could start
> with trace/simple.c.  It has a trace buffer that is asynchronously
> written out to file by a dedicated "writer" thread.

> The one case where hypertrace makes sense to me is for -user tracing.
> There QEMU can efficiently interleave guest and QEMU traces, although as
> mentioned in the patch, I don't think the SIGSEGV approach should be
> used.

> I suggest stripping this series down to focus on -user.  Synchronous
> tracing is not a good approach for -system emulation.

As I said, I wanted to implement a simple way to provide common reference points
in guest and host traces. For that, a *synchronous* guest-to-host channel is the
simplest way to do it (otherwise you're back to square one).

Another reason to have this is that I wanted to attach a dtrace/systemtap hook
to the hypertrace event, and from there dynamically control QEMU's tracing. Then
you can insert simple calls to hypertrace on your guest code to tell your dtrace
hooks when to enable/disable tracing of certain QEMU events (like guest instr
traces).

Unfortuntely, I've been unable to to make dtrace recognise QEMU's events (I'm
only able to see the host kernel events). If someone with more experience on it
can help me use dtrace with QEMU's events, I'll also add the supporting library
to let dtrace do the callout to QEMU's moitor interface and control the events,
and add a prperly useful example of that on the hypertrace docs (which was my
original intention).


Thanks,
  Lluis