[lttng-dev] sdt.h tracepoints with unicode data and/or structs
Mathieu Desnoyers
mathieu.desnoyers at efficios.com
Thu Sep 22 16:21:48 UTC 2016
----- On Sep 22, 2016, at 8:47 AM, Milian Wolff milian.wolff at kdab.com wrote:
> On Wednesday, September 21, 2016 6:09:15 PM CEST Mathieu Desnoyers wrote:
>> ----- On Sep 20, 2016, at 4:34 PM, Milian Wolff milian.wolff at kdab.com wrote:
>> > On Montag, 12. September 2016 16:24:04 CEST Mathieu Desnoyers wrote:
>> >> ----- On Sep 12, 2016, at 11:40 AM, Milian Wolff milian.wolff at kdab.com
>> >
>> > wrote:
>> >> > On Monday, September 12, 2016 3:03:04 PM CEST Mathieu Desnoyers wrote:
>> >> >> ----- On Sep 6, 2016, at 3:00 PM, Milian Wolff milian.wolff at kdab.com
>> >
>> > wrote:
>> >> >> > Hey all,
>> >> >> >
>> >> >> > where can I find more documentation on how to use sdt.h to add
>> >> >> > static
>> >> >> > tracepoints to user-land applications? If this is not the right
>> >> >> > place
>> >> >> > to
>> >> >> > ask, please refer to to elsewhere.
>> >> >>
>> >> >> Hi Milian,
>> >> >
>> >> > Hey Mathieu,
>> >> >
>> >> >> LTTng-UST offers a "tracepoint" instrumentation facility, which can
>> >> >> optionally emit "sdt.h" probe points too for compatibility with
>> >> >> SystemTAP.
>> >> >>
>> >> >> > I plan to upstream a collection of tracepoints to Qt, and possibly
>> >> >> > elsewhere. One problem I'm having right now is figuring out how to
>> >> >> > "design" the tracepoints such that they have minimal overhead.
>> >> >>
>> >> >> The main question here would be: do you want your instrumentation to
>> >> >> be
>> >> >> usable with LTTng-UST ? If yes, then you want the tracepoint
>> >> >> instrumentation facility. Else, if you only plan on using SystemTAP,
>> >> >> you
>> >> >> can use sdt.h instrumentation.
>> >> >>
>> >> >> > So my questions:
>> >> >> My answers will be about lttng-ust tracepoints.
>> >> >
>> >> > Thanks for the clarifications! Are the LTTNG-UST tracepoints also
>> >> > "zero-
>> >> > overhead" like the SystemTap once, i.e. put into a separate section and
>> >> > "only" one nop is added when tracing is disabled? I could not find such
>> >> > an information anywhere yet.
>> >>
>> >> The lttng-ust tracepoint mechanism comes from the design of the
>> >> Linux kernel tracepoints, which have proven to be unnoticeable
>> >> when disabled. There is one slight technicality though:
>> >>
>> >> The linux kernel tracepoints use the asm goto mechanism and code patching
>> >> to flip between a no-op and a jump to dynamically enable each tracepoint.
>> >> This is all very fine for the kernel.
>> >>
>> >> Now for user-space, the lttng-ust tracepoints rather use a conditional
>> >> branch to skip over the entire stack setup and the function call. There
>> >> are a few reasons for using a old-fashioned conditional branch: first,
>> >> there are no widely adopted multi-core, live code patching library
>> >> available in user-space that am I aware of. Second, even if we would
>> >> have one, we may not want to send SIGSTOP/SIGCONT to the traced process,
>> >> due to debugger interactions. Thirdly, doing code patching may not
>> >> interact will security features such as read-only code sections and
>> >> code checksumming. Finally, doing code patching of library and
>> >> executable code will trigger copy-on-write of the touched pages, which
>> >> will prevent sharing those pages between processes running the same
>> >> apps/libraries. All in all, the conditional branch does not seem like
>> >> a too bad alternative after all.
>> >>
>> >> If we look at the code generated by SystemTAP sdt.h probes (as well
>> >> as DTrace), they are actually more heavyweight than what has been
>> >> claimed by Sun's marketing department back in the days: it does
>> >> cost a stack setup of all the variables passed to the the probe,
>> >> and indeed the function call is no-op'd.
>> >>
>> >> The downside here is that all side-effects, and layout of the arguments
>> >> for the no-op'd call, are taken by the instrumented application, even
>> >> though tracing is not dynamically enabled.
>> >>
>> >> Therefore, the SDT mechanism is not a lightweight as is generally
>> >> claimed. It is not "just a single no-op per site", but rather a
>> >> function call stack setup and a no-op.
>> >
>> > Thanks a lot for the in-depth explanation Mathieu, much appreciated! From
>> > what I gather, sdt.h does allow to check whether a given trace point is
>> > enabled or not, no? I.e. via the semaphores. Is there any performance
>> > difference between checking the semaphore of a sdt.h tracepoint vs. doing
>> > the analogous check for lttng-ust tracepoints?
>>
>> Not sure what you refer to about "semaphores". And it's unclear how you
>> define "checking if active": is it to decide in the fast path whether to
>> trace or not, or in a slow path to query what is currently active ?
>
> The fast path to check whether to trace or not:
>
> $ cat probes.d
> provider milian {
> probe test();
> };
> $ dtrace -C -h -s probes.d -o probes.h
> $ cat probes.h
> /* Generated by the Systemtap dtrace wrapper */
>
>
> #define _SDT_HAS_SEMAPHORES 1
>
>
> #define STAP_HAS_SEMAPHORES 1 /* deprecated */
>
>
> #include <sys/sdt.h>
>
> /* MILIAN_TEST ( ) */
> #if defined STAP_SDT_V1
> #define MILIAN_TEST_ENABLED() __builtin_expect (test_semaphore, 0)
> #define milian_test_semaphore test_semaphore
> #else
> #define MILIAN_TEST_ENABLED() __builtin_expect (milian_test_semaphore, 0)
> #endif
> __extension__ extern unsigned short milian_test_semaphore __attribute__
> ((unused)) __attribute__ ((section (".probes")));
> #define MILIAN_TEST() \
> DTRACE_PROBE (milian, test)
Got it. The performance difference between lttng-ust tracepoint_enabled() check
and the sdt.h semaphore check will be none, because both perform an unlikely
branch (using builtin expect 0).
Thanks,
Mathieu
>
>
> --
> Milian Wolff | milian.wolff at kdab.com | Software Engineer
> KDAB (Deutschland) GmbH&Co KG, a KDAB Group company
> Tel: +49-30-521325470
> KDAB - The Qt Experts
> _______________________________________________
> lttng-dev mailing list
> lttng-dev at lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
More information about the lttng-dev
mailing list