[lttng-dev] sdt.h tracepoints with unicode data and/or structs

[Milian Wolff]

On Wednesday, September 21, 2016 5:22:35 PM CEST Mathieu Desnoyers wrote:
> ----- On Sep 20, 2016, at 7:22 PM, Philippe Proulx eeppeliteloop at gmail.com 
wrote:
> >>> >> > Can I pass UTF16 strings? Do they need to be null-terminated?
> >>> >> 
> >>> >> You should convert them to UTF8.
> >>> >> 
> >>> >> A ctf_string() needs to be null-terminated. A ctf_sequence_text() is
> >>> >> not required to be null-terminated. See
> >>> >> http://lttng.org/docs/#doc-liblttng-ust-tp-fields
> >>> > 
> >>> > This sounds like tracing would then incur a huge runtime overhead,
> >>> > when I
> >>> > need to convert all my UTF-16 strings to UTF-8. How does one deal with
> >>> > that?
> >>> 
> >>> Are you concerned about the runtime overhead when tracing is disabled,
> >>> or
> >>> enabled ? It would be good to gather some metrics on the overhead of
> >>> this
> >>> conversion.
> >> 
> >> I'm mostly concerned about the overhead when tracing is disabled.
> >> Ideally, I would like to see these tracepoints unconditionally compiled
> >> into Qt. But if the overhead is noticeable when they are not in use, I
> >> may have a hard time achieving this goal. Instead, one will then
> >> probably need to recompile Qt with the tracepoints enabled.
> > 
> > As Mathieu wrote, the overhead of disabled LTTng-UST tracepoints is pretty
> > much unnoticeable. You can use the tracepoint_enabled() and
> > do_tracepoint()
> > macros to perform some computation, conversions, and prepare stuff
> > specifically for the payload of the event.
> > 
> >> When in use, the overhead of the tracepoints should also be minimal.
> >> Allocating memory for a temporary UTF-16 to UTF-8 conversion alone has a
> >> large overhead, and then converting the data to UTF-8 also adds on top
> >> of that. Thus, if at all possible, I would like to prevent that.
> > 
> > There's no way to support UTF-16 as of the current versions of LTTng and
> > CTF. Even if you find a way to store the string as is, for example using a
> > sequence of bytes, the existing CTF viewers and analyzers won't recognize
> > the field as a string.
> > 
> > However we could think about a way to support UTF-16 in the future, and
> > possibly other Unicode encodings too.
> > 
> >> Similarly, I am looking for a way to put URLs into tracepoints, and
> >> converting a QUrl to string data is far from cheap.
> > 
> > Yes, I see that this code is executed:
> > <https://github.com/qt/qtbase/blob/dev/src/corelib/io/qurl.cpp#L3279>.
> > 
> > You could allocate one tracepoint field for each individual attribute of
> > the QUrl object, for example the scheme, the path, the query string, the
> > fragment, etc. But I guess you'd still have the UTF-16 issue.
> > 
> >> At this point, I don't see a way around only enabling tracepoints
> >> optionally. Independent of the mechanism in use for the actual
> >> tracepoints (i.e. lttng-ust or sdt). The conditional to check whether
> >> tracing is enabled may not be too bad. But then in the conditional it
> >> looks like $some code will be required to massage the data into a form
> >> that the tracepoints accept them. This increase in code size negatively
> >> influences code caches and I don't see any way around that.
> > 
> > I leave this part for Mathieu ;-).
> 
> This "extra code" can be implemented within the tracepoint provider,
> which is a cache cold function, not used at all when tracing is disabled.

This sounds excellent. Can you tell me how? Could you maybe add an example to 
lttng-ust. Also note how http://lttng.org/man/3/lttng-ust/v2.7 says:

	if (tracepoint_enabled(ust_tests_hello, tptest)) {
		/* prepare arguments */
		do_tracepoint(ust_tests_hello, tptest, i, netint, values,
			text, strlen(text), dbl, flt);
	}

If I understood you correctly, then I could do something like

	if (tracepoint_enabled(ust_tests_hello, tptest)) {
		/* don't prepare arguments */
		do_tracepoint(ust_tests_hello, my_complex_data);
	}

And then have the "prepare" code somewhere in my TRACEPOINT_EVENT?

Thanks
-- 
Milian Wolff | milian.wolff at kdab.com | Software Engineer
KDAB (Deutschland) GmbH&Co KG, a KDAB Group company
Tel: +49-30-521325470
KDAB - The Qt Experts
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5903 bytes
Desc: not available
URL: <https://lists.lttng.org/pipermail/lttng-dev/attachments/20160922/5e428452/attachment-0001.bin>