[ltt-dev] LTTng specialized probes

[Mathieu Desnoyers]

The idea is that someone who want to add a new instrumentation site in
the Linux kernel does not have to write a specialized probe up front.
The format string parser will take care of writing the typed data into
the buffers (default behavior), but can still overridden by a
specialized function which will expect the format string arguments and
serialize those into the buffers.

About what we discussed in Portland and where Steven is currently going:
it does not provide any kind of binary standard to export the data
between different platforms or even from kernel 64-bits kernel to
32-bits userland. Steven also cleary states that he doesn't care about
exporting this data to userspace in binary format. He wants a
supplementary layer to do this formatting, which I don't think will
produce the performance results we are looking for. Plus, I think
feeding the data through the kernel which recorded the information to
decode it is the wrong approach, especially when the system which
recorded such information is a small embedded device, where getting the
data _out_ is already non-trivial. Feeding it back in seems a bit crazy.

Mathieu


* Martin Bligh (mbligh at google.com) wrote:
> Question ... It seems that strings are mandatory for markers at the moment.
> I don't see why this is, and it seems significantly less efficient than what
> we discussed in Portland recently?
> 
> On Mon, Oct 6, 2008 at 7:11 AM, Mathieu Desnoyers
> <compudj at krystal.dyndns.org> wrote:
> > Hi,
> >
> > I'm currently working towards getting LTTng in shape for what is
> > required for mainline. I got the "TLB-less" buffers and splice()
> > working last week. I then did some performance testing on the flight
> > recorder mode and noticed an optimization that's really worth doing :
> >
> > LTTng "ltt-serialize.c", which parses the format strings and formats
> > data into the trace buffers takes a lot of CPU time. I tried only
> > keeping the size calculation (first pass on the format string) and
> > disabling the real data write and basically got something like :
> >
> > (default LTTng instrumentation, very approximate numbers)
> >
> > tbench no tracing : ~1900MB/s
> >       Markers enabled : ~1800MB/s
> >       with size calculation : ~1400MB/s
> >       size calc + data write : ~950MB/s
> >
> > I then remembered I've done ltt-serialize in such a way that it can be
> > easily overridden by per-format string specialized callbacks.
> >
> > Therefore, it would be worthwhile to create such specialized serializers
> > so the common cases can be made much faster. I think it will have a very
> > significant impact on performance.
> >
> > It's simply a matter of creating a new .c kernel module in ltt/ and to
> > create structures similar to :
> >
> > ltt-serialize.c :
> >
> > struct ltt_available_probe default_probe = {
> >        .name = "default",
> >        .format = NULL,
> >        .probe_func = ltt_vtrace,
> >        .callbacks[0] = ltt_serialize_data,
> > };
> >
> > Give it a non-null format string (just giving the types expected by the
> > callback), a good name, and a callback function, which implements the
> > specialized serialization. Note that kernel/marker.c currently expects
> > the format string to match exactly the marker format string, including
> > the type names, which should be changed. The type verification should
> > only check that the %X parameters are the same (and that there are the
> > same amount of arguments expected).
> >
> > That should not be hard, but it's not what I plan to focus on next.
> > Anyone is willing to work on this ?
> >
> > Mathieu
> >
> > --
> > Mathieu Desnoyers
> > OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
> >
> 

-- 
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68