[lttng-dev] sdt.h tracepoints with unicode data and/or structs

[Philippe Proulx]

>> >> > Can I pass UTF16 strings? Do they need to be null-terminated?
>> >>
>> >> You should convert them to UTF8.
>> >>
>> >> A ctf_string() needs to be null-terminated. A ctf_sequence_text() is
>> >> not required to be null-terminated. See
>> >> http://lttng.org/docs/#doc-liblttng-ust-tp-fields
>> >
>> > This sounds like tracing would then incur a huge runtime overhead, when I
>> > need to convert all my UTF-16 strings to UTF-8. How does one deal with
>> > that?
>>
>> Are you concerned about the runtime overhead when tracing is disabled, or
>> enabled ? It would be good to gather some metrics on the overhead of this
>> conversion.
>
> I'm mostly concerned about the overhead when tracing is disabled. Ideally, I
> would like to see these tracepoints unconditionally compiled into Qt. But if
> the overhead is noticeable when they are not in use, I may have a hard time
> achieving this goal. Instead, one will then probably need to recompile Qt with
> the tracepoints enabled.

As Mathieu wrote, the overhead of disabled LTTng-UST tracepoints is pretty
much unnoticeable. You can use the tracepoint_enabled() and do_tracepoint()
macros to perform some computation, conversions, and prepare stuff specifically
for the payload of the event.

>
> When in use, the overhead of the tracepoints should also be minimal.
> Allocating memory for a temporary UTF-16 to UTF-8 conversion alone has a large
> overhead, and then converting the data to UTF-8 also adds on top of that.
> Thus, if at all possible, I would like to prevent that.

There's no way to support UTF-16 as of the current versions of LTTng and
CTF. Even if you find a way to store the string as is, for example using a
sequence of bytes, the existing CTF viewers and analyzers won't recognize the
field as a string.

However we could think about a way to support UTF-16 in the future, and
possibly other Unicode encodings too.

>
> Similarly, I am looking for a way to put URLs into tracepoints, and converting
> a QUrl to string data is far from cheap.

Yes, I see that this code is executed:
<https://github.com/qt/qtbase/blob/dev/src/corelib/io/qurl.cpp#L3279>.

You could allocate one tracepoint field for each individual attribute of the
QUrl object, for example the scheme, the path, the query string, the
fragment, etc. But I guess you'd still have the UTF-16 issue.

>
> At this point, I don't see a way around only enabling tracepoints optionally.
> Independent of the mechanism in use for the actual tracepoints (i.e. lttng-ust
> or sdt). The conditional to check whether tracing is enabled may not be too
> bad. But then in the conditional it looks like $some code will be required to
> massage the data into a form that the tracepoints accept them. This increase
> in code size negatively influences code caches and I don't see any way around
> that.

I leave this part for Mathieu ;-).

Phil

>
>> > Looking at the API documentation, using ctf_array instead with ushort as C
>> > type sounds like a better approach. In a custom babeltrace consumer
>> > script, I could then convert it to UTF8 offline, if needed.
>> >
>> > The advantage here is that I have "zero" overhead while tracing, which is
>> > crucial for me to get the tracepoints accepted upstream in Qt, without
>> > hiding them behind a compiler switch.
>>
>> As a short term solution this appears to be fine.
>>
>> In the long run, we're currently designing CTF 2.0, and we're been wondering
>> whether we keep it limited to UTF8, or extend it somehow. I'm CCing
>> Philippe Proulx who is driving this effort.
>
> Thanks again!
>
> Bye
>
> --
> Milian Wolff | milian.wolff at kdab.com | Software Engineer
> KDAB (Deutschland) GmbH&Co KG, a KDAB Group company
> Tel: +49-30-521325470
> KDAB - The Qt Experts
>
> _______________________________________________
> lttng-dev mailing list
> lttng-dev at lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>