[lttng-dev] Question about filtering bytecode

Mon Jul 22 02:14:47 EDT 2013

Thanks for the explanations.

I currently feel that the effort to get such a thing working is not going 
to materialize in the timeframe required for my current work, and 
therefore, regrettably, I will have to work around this somehow. I am 
currently thinking of using a combination of Python and Cython to generate 
a wrapper for the babeltrace iterator. We'll see.

Thanks,

Amit Margalit
IBM XIV - Storage Reinvented
XIV-NAS Development Team
Tel. 03-689-7774
Fax. 03-689-7230

From:   Jérémie Galarneau <jeremie.galarneau at efficios.com>
To:     Amit Margalit/Israel/IBM at IBMIL
Cc:     lttng-dev at lists.lttng.org
Date:   07/21/2013 10:07 PM
Subject:        Re: [lttng-dev] Question about filtering bytecode
Sent by:        jeremie.galarneau at gmail.com

On Sun, Jul 21, 2013 at 1:51 AM, Amit Margalit <AMITM at il.ibm.com> wrote:
> Hi,
>
> I was thinking more about a programmer's guide to using this mechanism.
>

Unfortunately there is no programmer's guide on filters, but feel free
to ask about specifics.

> What I'd like to do is to have Babeltrace filter events it is showing me
> from an existing trace.
>
> Consider the following example:
>
> There are 3 types of tracepoints being captured (without filtering):
>
> A debug log message (occasionally) which includes a "debug_level" value
> (1..10 with 1 being highest importance).
> Function entry (via gcc -finstrument-functions or other 
instrumentation).
> Function exit (similarly).
>
> During a 1-hour period, I have 2-3 GB of trace data captured.
> The trace includes events from 4 different processes.
>
>
> If I use Babeltrace to read the trace I will get an enormously big 
result in
> text (~30-40GB at least).
>
> I would like to be able to do something like this:
>
> $ babeltrace --filter '$ctx.procname == "smbd" || debug_level < 3'
> /local/lttng-traces/ust/uid/2140/64-bit
>
> This would then let me see any events from the "smbd" process (since i 
added
> procname to the context using lttng add-context) or debug messages from 
all
> processes where the debug level is 1 or 2.
>

That would be a neat feature! There are currently no plans to support
filtering in Babeltrace, but that contribution would be welcome.
lttng-ust's filtering code is licensed under LGPL v2.1 so it could be
library-fied and used by the ctf-text plug-in. In the meantime, I
guess you could work around this limitation with a combination of AWK
and shell scripting...

> Regarding linking - I am referring to the function
> lttng_enabler_event_link_bytecode(), called from
> lttng-ust/liblttng-ust/lttng-events.c around line 628.
>

Thanks for clarifying!

Keep in mind that the filter's byte code is generated by the lttng
binary without knowing the events' declarations. Thus, the byte code
must go through a linking stage not completely unlike what is done by
a regular linker. One of the objectives of the linking stage is
setting the appropriate offsets to load an event's fields. The byte
code is also "specialized" by resolving each field's type and making
sure the appropriate implementation of each operator is used. For
instance, the FILTER_OP_EQ (check for equality) op-code is not
implemented the same way for integers, doubles and strings (see
lttng_filter_specialize_bytecode in lttng-filter-specialize.c).

You may want to have a look at _lttng_filter_event_link_bytecode
(lttng-ust/liblttng-ust/lttng-filter.c, line 318) to see how each step
of the linking process is implemented.

Regards,
Jérémie

> Thanks,
>
> Amit Margalit
> IBM XIV - Storage Reinvented
> XIV-NAS Development Team
> Tel. 03-689-7774
> Fax. 03-689-7230
>
>
>
> From:        Jérémie Galarneau <jeremie.galarneau at efficios.com>
> To:        Amit Margalit/Israel/IBM at IBMIL
> Cc:        lttng-dev at lists.lttng.org
> Date:        07/18/2013 07:10 PM
> Subject:        Re: [lttng-dev] Question about filtering bytecode
> Sent by:        jeremie.galarneau at gmail.com
> ________________________________
>
>
>
> Hi Amit,
>
> You may find information on the Babeltrace API in babeltrace/doc/API.txt
> The filtering mechanism used by the user space tracer is unrelated to
> Babeltrace.
>
> As for the sequence of events, this is how I understand it.
>
> When invoking the "lttng" binary with --enable-event and a filter
> option, a byte code is generated from the filtering rules
> (lttng-tools/src/lib/lttng-ctl/lttng-ctl.c:704). This byte code is
> then handed off to the session-daemon.
>
> Then, when an instrumented application is launched and connects to the
> session-daemon, it receives the byte code while its events are being
> enabled. To answer your question directly, that byte code will be
> executed by the tracee when the event is "triggered"
> (lttng-ust/liblttng-ust/lttng-filter-interpreter.c:175).
>
> I'm not sure I understand your question about filter/event linking. A
> filter has to be linked to an event since its rules (expressions) are
> applied on that event's values. Could you clarify if that's not what
> you meant?
>
> Regards,
> Jérémie
>
> On Thu, Jul 18, 2013 at 3:12 AM, Amit Margalit <AMITM at il.ibm.com> wrote:
>> Hi,
>>
>> I'm considering whether I can write a wrapper for the libbabeltrace ctf
>> iterator which provides a filtered "next()" method.
>>
>> I've started reading the code in the hopes of understanding how to do
>> this,
>> and yet, I'd really appreciate a link to some ordered documentation, or 
-
>> any other kind of explanation.
>>
>> For example, one thing I still don't understand is the exact sequence, 
and
>> where the bytecode is actually executed? What is the meaning and 
purpose
>> of
>> "linking" the filter to an event.
>>
>> Thanks,
>>
>> Amit Margalit
>> IBM XIV - Storage Reinvented
>> XIV-NAS Development Team
>> Tel. 03-689-7774
>> Fax. 03-689-7230
>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev at lists.lttng.org
>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>
>
>
>
> --
> Jérémie Galarneau
> EfficiOS Inc.
> http://www.efficios.com
>
>

-- 
Jérémie Galarneau
EfficiOS Inc.
http://www.efficios.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lttng.org/pipermail/lttng-dev/attachments/20130722/ae5c495e/attachment.html>