[lttng-dev] sdt.h tracepoints with unicode data and/or structs

[Mathieu Desnoyers]

----- On Sep 20, 2016, at 7:22 PM, Philippe Proulx eeppeliteloop at gmail.com wrote:

>>> >> > Can I pass UTF16 strings? Do they need to be null-terminated?
>>> >>
>>> >> You should convert them to UTF8.
>>> >>
>>> >> A ctf_string() needs to be null-terminated. A ctf_sequence_text() is
>>> >> not required to be null-terminated. See
>>> >> http://lttng.org/docs/#doc-liblttng-ust-tp-fields
>>> >
>>> > This sounds like tracing would then incur a huge runtime overhead, when I
>>> > need to convert all my UTF-16 strings to UTF-8. How does one deal with
>>> > that?
>>>
>>> Are you concerned about the runtime overhead when tracing is disabled, or
>>> enabled ? It would be good to gather some metrics on the overhead of this
>>> conversion.
>>
>> I'm mostly concerned about the overhead when tracing is disabled. Ideally, I
>> would like to see these tracepoints unconditionally compiled into Qt. But if
>> the overhead is noticeable when they are not in use, I may have a hard time
>> achieving this goal. Instead, one will then probably need to recompile Qt with
>> the tracepoints enabled.
> 
> As Mathieu wrote, the overhead of disabled LTTng-UST tracepoints is pretty
> much unnoticeable. You can use the tracepoint_enabled() and do_tracepoint()
> macros to perform some computation, conversions, and prepare stuff specifically
> for the payload of the event.
> 
>>
>> When in use, the overhead of the tracepoints should also be minimal.
>> Allocating memory for a temporary UTF-16 to UTF-8 conversion alone has a large
>> overhead, and then converting the data to UTF-8 also adds on top of that.
>> Thus, if at all possible, I would like to prevent that.
> 
> There's no way to support UTF-16 as of the current versions of LTTng and
> CTF. Even if you find a way to store the string as is, for example using a
> sequence of bytes, the existing CTF viewers and analyzers won't recognize the
> field as a string.
> 
> However we could think about a way to support UTF-16 in the future, and
> possibly other Unicode encodings too.
> 
>>
>> Similarly, I am looking for a way to put URLs into tracepoints, and converting
>> a QUrl to string data is far from cheap.
> 
> Yes, I see that this code is executed:
> <https://github.com/qt/qtbase/blob/dev/src/corelib/io/qurl.cpp#L3279>.
> 
> You could allocate one tracepoint field for each individual attribute of the
> QUrl object, for example the scheme, the path, the query string, the
> fragment, etc. But I guess you'd still have the UTF-16 issue.
> 
>>
>> At this point, I don't see a way around only enabling tracepoints optionally.
>> Independent of the mechanism in use for the actual tracepoints (i.e. lttng-ust
>> or sdt). The conditional to check whether tracing is enabled may not be too
>> bad. But then in the conditional it looks like $some code will be required to
>> massage the data into a form that the tracepoints accept them. This increase
>> in code size negatively influences code caches and I don't see any way around
>> that.
> 
> I leave this part for Mathieu ;-).

This "extra code" can be implemented within the tracepoint provider,
which is a cache cold function, not used at all when tracing is disabled.

Thanks,

Mathieu

> 
> Phil
> 
>>
>>> > Looking at the API documentation, using ctf_array instead with ushort as C
>>> > type sounds like a better approach. In a custom babeltrace consumer
>>> > script, I could then convert it to UTF8 offline, if needed.
>>> >
>>> > The advantage here is that I have "zero" overhead while tracing, which is
>>> > crucial for me to get the tracepoints accepted upstream in Qt, without
>>> > hiding them behind a compiler switch.
>>>
>>> As a short term solution this appears to be fine.
>>>
>>> In the long run, we're currently designing CTF 2.0, and we're been wondering
>>> whether we keep it limited to UTF8, or extend it somehow. I'm CCing
>>> Philippe Proulx who is driving this effort.
>>
>> Thanks again!
>>
>> Bye
>>
>> --
>> Milian Wolff | milian.wolff at kdab.com | Software Engineer
>> KDAB (Deutschland) GmbH&Co KG, a KDAB Group company
>> Tel: +49-30-521325470
>> KDAB - The Qt Experts
>>
>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev at lists.lttng.org
>> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>
> _______________________________________________
> lttng-dev mailing list
> lttng-dev at lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com