[lttng-dev] sdt.h tracepoints with unicode data and/or structs

[Mathieu Desnoyers]

----- On Sep 12, 2016, at 11:40 AM, Milian Wolff milian.wolff at kdab.com wrote:

> On Monday, September 12, 2016 3:03:04 PM CEST Mathieu Desnoyers wrote:
>> ----- On Sep 6, 2016, at 3:00 PM, Milian Wolff milian.wolff at kdab.com wrote:
>> > Hey all,
>> > 
>> > where can I find more documentation on how to use sdt.h to add static
>> > tracepoints to user-land applications? If this is not the right place to
>> > ask, please refer to to elsewhere.
>> 
>> Hi Milian,
> 
> Hey Mathieu,
> 
>> LTTng-UST offers a "tracepoint" instrumentation facility, which can
>> optionally emit "sdt.h" probe points too for compatibility with SystemTAP.
>> 
>> > I plan to upstream a collection of tracepoints to Qt, and possibly
>> > elsewhere. One problem I'm having right now is figuring out how to
>> > "design" the tracepoints such that they have minimal overhead.
>> 
>> The main question here would be: do you want your instrumentation to be
>> usable with LTTng-UST ? If yes, then you want the tracepoint instrumentation
>> facility. Else, if you only plan on using SystemTAP, you can use sdt.h
>> instrumentation.
>> 
>> > So my questions:
>> My answers will be about lttng-ust tracepoints.
> 
> Thanks for the clarifications! Are the LTTNG-UST tracepoints also "zero-
> overhead" like the SystemTap once, i.e. put into a separate section and "only"
> one nop is added when tracing is disabled? I could not find such an
> information anywhere yet.

The lttng-ust tracepoint mechanism comes from the design of the
Linux kernel tracepoints, which have proven to be unnoticeable
when disabled. There is one slight technicality though:

The linux kernel tracepoints use the asm goto mechanism and code patching
to flip between a no-op and a jump to dynamically enable each tracepoint.
This is all very fine for the kernel.

Now for user-space, the lttng-ust tracepoints rather use a conditional
branch to skip over the entire stack setup and the function call. There
are a few reasons for using a old-fashioned conditional branch: first,
there are no widely adopted multi-core, live code patching library
available in user-space that am I aware of. Second, even if we would
have one, we may not want to send SIGSTOP/SIGCONT to the traced process,
due to debugger interactions. Thirdly, doing code patching may not
interact will security features such as read-only code sections and
code checksumming. Finally, doing code patching of library and
executable code will trigger copy-on-write of the touched pages, which
will prevent sharing those pages between processes running the same
apps/libraries. All in all, the conditional branch does not seem like
a too bad alternative after all.

If we look at the code generated by SystemTAP sdt.h probes (as well
as DTrace), they are actually more heavyweight than what has been
claimed by Sun's marketing department back in the days: it does
cost a stack setup of all the variables passed to the the probe,
and indeed the function call is no-op'd.

The downside here is that all side-effects, and layout of the arguments
for the no-op'd call, are taken by the instrumented application, even
though tracing is not dynamically enabled.

Therefore, the SDT mechanism is not a lightweight as is generally
claimed. It is not "just a single no-op per site", but rather a
function call stack setup and a no-op.

> 
>> > What data types can I use for trace point arguments?
>> 
>> See http://lttng.org/docs/#doc-c-application
>> 
>> > Can I pass UTF16 strings? Do they need to be null-terminated?
>> 
>> You should convert them to UTF8.
>> 
>> A ctf_string() needs to be null-terminated. A ctf_sequence_text() is
>> not required to be null-terminated. See
>> http://lttng.org/docs/#doc-liblttng-ust-tp-fields
> 
> This sounds like tracing would then incur a huge runtime overhead, when I need
> to convert all my UTF-16 strings to UTF-8. How does one deal with that?

Are you concerned about the runtime overhead when tracing is disabled, or
enabled ? It would be good to gather some metrics on the overhead of this
conversion.

> 
> Looking at the API documentation, using ctf_array instead with ushort as C
> type sounds like a better approach. In a custom babeltrace consumer script, I
> could then convert it to UTF8 offline, if needed.
> 
> The advantage here is that I have "zero" overhead while tracing, which is
> crucial for me to get the tracepoints accepted upstream in Qt, without hiding
> them behind a compiler switch.

As a short term solution this appears to be fine.

In the long run, we're currently designing CTF 2.0, and we're been wondering
whether we keep it limited to UTF8, or extend it somehow. I'm CCing
Philippe Proulx who is driving this effort.

Thanks!

Mathieu

> 
>> > Can I pass structs? Or do I need to pass each member as an individual
>> > argument?
>> 
>> You can pass pointers to structures to the tracepoint() call from your
>> application source code, and then you need to express each individual
>> field you want to serialize into the trace within the TP_FIELDS() macro.
>> Ref. http://lttng.org/docs/#doc-liblttng-ust-tp-fields
> 
> Thank you.
> 
> Bye
> 
> --
> Milian Wolff | milian.wolff at kdab.com | Software Engineer
> KDAB (Deutschland) GmbH&Co KG, a KDAB Group company
> Tel: +49-30-521325470
> KDAB - The Qt Experts

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com