[lttng-dev] [RFC PATCH lttng-ust] Make lttng-ust aware of shared object base addresses (issue #474)

Tue Aug 27 05:53:26 EDT 2013

Absolutely wonderful idea, IMHO.

I encountered the same problem, and partially solved it, differently. I 
think your idea complements my solution, so perhaps together we can come 
up with something even better.

There are a couple of requirements that are still hard to meet with your 
solution:
If I seek to an arbitrary position in the trace, I have no easy way to 
find which shared object the address belongs to.
If I read the trace on a different system than where it was generated, I 
may not have access to the same shared objects, so "/lib64/libc-2.17.so" 
may not be valid, or worse - may be valid but different architecture and 
therefore the symbol addresses would be different.

Solving the first problem required me to emit something that would 
identify the shared object with every tracepoint that includes a function 
pointer that needs translation.

Saving the base address of the shared object that the address belongs to 
plus the path or name of the shared object was too expensive, and didn't 
solve the second problem.

What I did was this:
Make a map of the loaded shared objects and their base addresses - called 
before anything can generate any trace
For each shared object, also keep a "signature" that uniquely identifies 
this shared object
Emit the signature and the function address (with the base address 
subtracted already)
Created a script that reads the shared objects to create a map of 
signature-to-(address-to-symbol) nested map, and post-process babeltrace 
output to provide the translation

For signature, I took the first 8 bytes of the unique build-id that GCC 
stores for every executable and shared object (SHA1) in the .note.build-id 
ELF section.

The problems with my approach are:
1. I wasn't handling dlopen/dlclose calls
2. Relying on external tools and post-processing

Combining the 2 sugestions gives a good solution - i.e. track 
dlopen/dlclose to generate the shared object maps during runtime so that 
we can emit signature+function-pointer(relative to its shared object).

Moreover, I suggest further improvement - a TP_FIELDS macro "ctf_symbol" 
that will automatically do exactly this, and store the 
signature+address-to-name mapping in the metadata...

What do you think?

Amit Margalit
IBM XIV - Storage Reinvented
XIV-NAS Development Team
Tel. 03-689-7774
Fax. 03-689-7230

From:   <Paul_Woegerer at mentor.com>
To:     <lttng-dev at lists.lttng.org>, Mathieu Desnoyers 
<mathieu.desnoyers at efficios.com>
Date:   08/27/2013 09:33 AM
Subject:        [lttng-dev] [RFC PATCH lttng-ust] Make lttng-ust aware of 
shared  object base addresses (issue #474)

Hello,

To allow graphical trace viewers to provide features like 
address-to-symbol resolution or source lookup (address-to-lineinfo), 
information about the shared object base address mappings are required. 
These mappings are inherently time based and potentially change during the 
application runtime (dlopen, dlcose, dlopen, ...). It's even possible that 
at a later point in the program the same shared object gets mapped to a 
different base address (I saw that once on ARM or PPC)

The patch set below provides an initial (prof of concept) implementation 
(based on the previous discussion on https://bugs.lttng.org/issues/474). 
It's a starting point for further discussions that will hopefully help me 
to provide an implementation that is good enough for upstream inclusion.

Applying the patch provides the following additional events for userspace 
tracing:

*) ust_baddr:push with TP_ARGS(void *, baddr, const char*, sopath, 
int64_t, size, int64_t, mtime)
*) ust_baddr:pop with TP_ARGS(void *, baddr)

Tracepoints 'ust_baddr:push' and 'ust_baddr:pop' are emitted for every 
runtime object that makes use of tracing (i.e emits tracepoints/includes 
tracepoint.h). This is also true for shared objects that get loaded during 
application run time. Loading a shared object via dlopen will, for 
example, result in emitting:
[14:22:07.841547445] ust_baddr:push: { cpu_id = 4 }, { baddr = 
0x7F8D13DFD000, sopath = "/path/to/plugin1.so", size = 21968, mtime = 
1377519722 }
..likewise sometime later a corresponding dlclose in the program will 
result in emitting:
[14:22:07.841867307] ust_baddr:pop: { cpu_id = 4 }, { baddr = 
0x7F8D13DFD000 }

*) ust_baddr:init with TP_ARGS(void *, baddr, const char*, sopath, 
int64_t, size, int64_t, mtime)

In contrast, ust_baddr:init tracepoints are only emitted when a userspace 
application gets a "session enabled" sent by the sessiond (necessary if 
tracing starts in the middle of an already running application). In this 
case all the currently loaded shared objects are iterated and the base 
address mappings are recored as ust_baddr:init events with the same 
payload structure as ust_baddr:push. E.g.

[15:20:32.763114409]  ust_baddr:init: { cpu_id = 1 }, { baddr = 
0x7F5BAC083000, sopath = "/lib64/libc-2.17.so", size = 1992089, mtime = 
1359709982 }
[15:20:32.763118346]  ust_baddr:init: { cpu_id = 1 }, { baddr = 
0x7F5BAEAF0000, sopath = "/lib64/ld-2.17.so", size = 163493, mtime = 
1359709980 }
[15:20:32.763125921]  ust_baddr:init: { cpu_id = 1 }, { baddr = 
0x7F5BAAE7C000, sopath = "/other/currently/loaded/shared-object.so.0.0.0", 
size = 58359, mtime = 1377516019 }
... and so on for all other currently loaded shared objects

The known deficiencies of the current implementation are:

Non-optional 'ust_baddr:push' and 'ust_baddr:pop' events 

-- 
Paul Woegerer, SW Development Engineer
Sourcery Analyzer <http://go.mentor.com/sourceryanalyzer>
Mentor Graphics, Embedded Software Division

_______________________________________________
lttng-dev mailing list
lttng-dev at lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lttng.org/pipermail/lttng-dev/attachments/20130827/94372bbb/attachment-0001.html>