[lttng-dev] sdt.h tracepoints with unicode data and/or structs

[Mathieu Desnoyers]

----- On Sep 20, 2016, at 4:34 PM, Milian Wolff milian.wolff at kdab.com wrote:

> On Montag, 12. September 2016 16:24:04 CEST Mathieu Desnoyers wrote:
>> ----- On Sep 12, 2016, at 11:40 AM, Milian Wolff milian.wolff at kdab.com
> wrote:
>> > On Monday, September 12, 2016 3:03:04 PM CEST Mathieu Desnoyers wrote:
>> >> ----- On Sep 6, 2016, at 3:00 PM, Milian Wolff milian.wolff at kdab.com
> wrote:
>> >> > Hey all,
>> >> > 
>> >> > where can I find more documentation on how to use sdt.h to add static
>> >> > tracepoints to user-land applications? If this is not the right place
>> >> > to
>> >> > ask, please refer to to elsewhere.
>> >> 
>> >> Hi Milian,
>> > 
>> > Hey Mathieu,
>> > 
>> >> LTTng-UST offers a "tracepoint" instrumentation facility, which can
>> >> optionally emit "sdt.h" probe points too for compatibility with
>> >> SystemTAP.
>> >> 
>> >> > I plan to upstream a collection of tracepoints to Qt, and possibly
>> >> > elsewhere. One problem I'm having right now is figuring out how to
>> >> > "design" the tracepoints such that they have minimal overhead.
>> >> 
>> >> The main question here would be: do you want your instrumentation to be
>> >> usable with LTTng-UST ? If yes, then you want the tracepoint
>> >> instrumentation facility. Else, if you only plan on using SystemTAP, you
>> >> can use sdt.h instrumentation.
>> >> 
>> >> > So my questions:
>> >> My answers will be about lttng-ust tracepoints.
>> > 
>> > Thanks for the clarifications! Are the LTTNG-UST tracepoints also "zero-
>> > overhead" like the SystemTap once, i.e. put into a separate section and
>> > "only" one nop is added when tracing is disabled? I could not find such
>> > an information anywhere yet.
>> 
>> The lttng-ust tracepoint mechanism comes from the design of the
>> Linux kernel tracepoints, which have proven to be unnoticeable
>> when disabled. There is one slight technicality though:
>> 
>> The linux kernel tracepoints use the asm goto mechanism and code patching
>> to flip between a no-op and a jump to dynamically enable each tracepoint.
>> This is all very fine for the kernel.
>> 
>> Now for user-space, the lttng-ust tracepoints rather use a conditional
>> branch to skip over the entire stack setup and the function call. There
>> are a few reasons for using a old-fashioned conditional branch: first,
>> there are no widely adopted multi-core, live code patching library
>> available in user-space that am I aware of. Second, even if we would
>> have one, we may not want to send SIGSTOP/SIGCONT to the traced process,
>> due to debugger interactions. Thirdly, doing code patching may not
>> interact will security features such as read-only code sections and
>> code checksumming. Finally, doing code patching of library and
>> executable code will trigger copy-on-write of the touched pages, which
>> will prevent sharing those pages between processes running the same
>> apps/libraries. All in all, the conditional branch does not seem like
>> a too bad alternative after all.
>> 
>> If we look at the code generated by SystemTAP sdt.h probes (as well
>> as DTrace), they are actually more heavyweight than what has been
>> claimed by Sun's marketing department back in the days: it does
>> cost a stack setup of all the variables passed to the the probe,
>> and indeed the function call is no-op'd.
>> 
>> The downside here is that all side-effects, and layout of the arguments
>> for the no-op'd call, are taken by the instrumented application, even
>> though tracing is not dynamically enabled.
>> 
>> Therefore, the SDT mechanism is not a lightweight as is generally
>> claimed. It is not "just a single no-op per site", but rather a
>> function call stack setup and a no-op.
> 
> Thanks a lot for the in-depth explanation Mathieu, much appreciated! From what
> I gather, sdt.h does allow to check whether a given trace point is enabled or
> not, no? I.e. via the semaphores. Is there any performance difference between
> checking the semaphore of a sdt.h tracepoint vs. doing the analogous check for
> lttng-ust tracepoints?
> 

Not sure what you refer to about "semaphores". And it's unclear how you
define "checking if active": is it to decide in the fast path whether to
trace or not, or in a slow path to query what is currently active ?

Thanks,

Mathieu


-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com