[lttng-dev] "Hands-free" tracepoints using LD_PRELOAD

[Francis Deslauriers]

"

Le mer. 23 janv. 2019, à 19 h 20, Brian Rossa <br at f0cal.com> a écrit :
>
> Jonathan,
>
> Responses below:
>
>> AFAIK, lttng does not have an equivalent.
>
>
> I believe my code could significantly reduce the need for hand-writing the tracepoints. But I won't likely take on a port to LTTng immediately, as a "vanilla" interposition approach seems to be meeting my requirements.
>
>>
>> > Step #2 is particularly problematic due to
>> > ambiguities in the mangling grammar, and will need support going forward to
>> > generalize well.
>>
>> What is the status of this step in your project?
>
>
> I demangle the symbol name using c++filt and then use regex to extract the list of argument types. Using something like ItaniumPartialDemangler would be better.

Hi Brian,

If you have a list of the argument types, you can programmatically
generate the tracepoint descriptions and callsites accordingly. The
main blocker I see here is tracing arguments that are pointers to
classes or containers. We need to be able to map each argument with a
CTF type [1]. It's easy enough for int, float and char * but it's
harder for complex structs and data structures.

You mentioned that you wrap structures with ostreams to output them in
text format. Can you explain this a bit more?

Here are a few ideas:

#First idea
If you are already defining the printing format and order of each of
the fields of each structures in your libfoo.so maybe you could do the
same but in LTTng-UST format. See "my-custom-structure.h" example [2].

#Second idea:
If you prefer not convert those ostreams wrappers to ctf wrapper, you
could reuse them to generate CTF_STRINGs.
1. Simple data types (int, float, char*) are mapped directly to CTF types,
2. Complex data types are wrapped with ostream function,
3. Complex data types are saved in the trace as CTF_STRING using the ostream.
All this could be done by the boilerplate scripts I mentioned earlier.
By using string format for some argument, you don't get the full power
of LTTng but it will still be faster that saving everything in text.

#Third idea:
Do you know of tracef() [3] ? Using it, you can save any string to a
UST trace. As a first step, you could directly replace your calls to
spdlog by calls to tracef. It's an highly inefficient way of using
LTTng, but it works (and probably lower overhead than writing to a
file).

[1]: https://lttng.org/man/3/lttng-ust/v2.10/
[2]: https://lttng.org/docs/v2.10/#doc-defining-tracepoints
[3]: https://lttng.org/docs/v2.10/#doc-tracef

Thank you,
Francis

>
>>
>> What are the problems that make your implementation "error-pone"?
>
>
> Use of regex as above.
>
>>
>> Would you mind linking us to said project so we can have a look?
>
>
> It's a project for my employer, and I would need to keep the source closed until I can get an OK. A signal of interest from LTTng would be helpful there. In the meantime, I think it would be fine to share as read-only with the LTTng maintainers if someone wants to send me a Github username.
>
>>
>> I would be interested in seeing at first lttng tracepoint used as Francis
>> demonstrated and see from there were this project can go.
>
> ...
>>
>> > I would be happy to contribute some or all of my implementation if it's
>> > something that the LTTng community would be interested in supporting and
>> > extending.
>>
>> We are clearly open for discussion and helping you improve the project. I am not
>> so sure on supporting and extending it. Others might have a different opinion.
>
>
> Sounds good. Looking forward to connecting via personal email.
>
> Cheers!
> ~br
>
>
> On Tue, Jan 22, 2019 at 2:17 PM Jonathan Rajotte-Julien <jonathan.rajotte-julien at efficios.com> wrote:
>>
>> Hi Brian,
>>
>> On Tue, Jan 22, 2019 at 01:30:23PM -0500, Brian Rossa wrote:
>> >    4. Boilerplate that does the typical `log(...); auto return_val =
>> >    dlsym(...); log(...); return return_val;` gets generated.
>>
>> As proposed by Francis, this is when you need to "generate" a corresponding
>> tracepoint definition and call the tracepoint() call with the appropriate
>> arguments.
>>
>> As Francis demonstrated we do not see any reason for lttng-ust not to work here
>> given that you compile the libshim object correctly.
>>
>> >    5. `log(...)` is a thin interface to spdlog
>> >    <https://github.com/gabime/spdlog> that handles `__attribute__`-based
>> >    setup and teardown of a logger.
>> >
>> > So at the end of the day, the shim developer provides:
>> >
>> >    - The whitelist of mangled names
>> >    - Implementations of struct "wrappers" that provide custom ostream
>> >    operators
>> >    - A map between type names and wrapper names
>> >
>> > The machinery here seems fairly general-purpose, but I don't presume to be
>> > an expert. My implementation is somewhat error-prone, and my main hope in
>> > reaching out to the mailing list was that LTTng already had some of these
>> > steps better-implemented.
>>
>> AFAIK, lttng does not have an equivalent.
>>
>> > Step #2 is particularly problematic due to
>> > ambiguities in the mangling grammar, and will need support going forward to
>> > generalize well.
>>
>> What is the status of this step in your project?
>>
>> What are the problems that make your implementation "error-pone"?
>>
>> Would you mind linking us to said project so we can have a look?
>>
>> I would be interested in seeing at first lttng tracepoint used as Francis
>> demonstrated and see from there were this project can go.
>>
>> >
>> > I would be happy to contribute some or all of my implementation if it's
>> > something that the LTTng community would be interested in supporting and
>> > extending.
>>
>> We are clearly open for discussion and helping you improve the project. I am not
>> so sure on supporting and extending it. Others might have a different opinion.
>>
>>
>> Cheers
>>
>> --
>> Jonathan Rajotte-Julien
>> EfficiOS



-- 
Francis Deslauriers
Computer Engineer
EfficiOS inc.