[ltt-dev] UST - Problems in recording trace on a current process

[Pierre-Marc Fournier]

On February 9, 2010 15:40:50 Naren wrote:
> Hi Pierre,
>
> >>- the program you want to trace has been linked dynamically against
>
> libust.
>
> >>You can do so with the ldd tool
>
> The program i want to trace is firefox with pid 1684.
>
> Here are the results of ldd'ing firefox
>
> *root at naren-laptop:/tmp# ldd /usr/lib/firefox-3.5.7/firefox-3.5*
> * **linux-gate.so.1 =>  (0xb7f27000)*
> * **libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xb7efc000)*
> * **libasound.so.2 => /usr/lib/libasound.so.2 (0xb7e35000)*
> * **libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7e30000)*
> * **libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb7d3e000)*
> * **libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb7d18000)*
> * **libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7cfa000)*
> * **libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7bb5000)*
> * **/lib/ld-linux.so.2 (0xb7f28000)*
> * **librt.so.1 => /lib/tls/i686/cmov/librt.so.1 (0xb7bab000)*

So we have the answer to your problem. You cannot list the markers on your 
program (firefox) because it was not linked with libust. Linking a program to 
libust is essential in order to be able to trace it or to list markers.

There are two ways of doing this. Either you can link it with the gcc 
option -lust at compile time, or you can link it at runtime using LD_PRELOAD. 
For example:

LD_PRELOAD=/path/to/libust.so ./myprogram

Without libust, the listener thread won't be created and the program will 
never create a socket in /tmp/ust-app-socks, hence your error message about 
the socket not being found.

Another way to do the LD_PRELOAD trick is to use usttrace, which will do it 
automatically with -l:

usttrace -l ./myprogram

Of course you need to add some instrumentation to firefox or to the libraries 
it's linked to, to see something interesting in the trace. Using the -m 
option to ustctl you can automatically instrument calls to malloc() and 
free() at the cost of an additional function call each time they are called.

Also note that firefox uses fork() and clone(). Because of this, you'll need 
to use libinterfork. This library intercepts calls to fork() and clone() and 
sets up the trace correctly in the new processes. You can use it with 
usttrace -f. This is a bit described in manual section "5.5 Tracing across 
fork() and clone()" (at http://lttng.org/files/ust/manual/ust.html) but it's 
still missing information... and it could probably use more testing.

>
> >>you did a "make install" when you upgraded your ust. You would get this
> >>error if ustctl and your traced program were linked against different
> >>versions of libust.so
>
> For what i infer, none of these are hooked up with libust. Have i missed
> out any because i didn't even have the opportunity of starting a trace on
> firefox, since i was not able to list markers.
>
> *Moreover, another unrelated nitpick.*
>
> In the manual http://lttng.org/files/ust/manual/ust.html, under section
> "5.2 Setting up the recording manually", the command  given to enable
> markers "$ ustctl --enable-marker 1234 ust/mymark" . However, after some
> reverse engineering, i found out that "parse_opts_long" function "ustctl.c"
> file expects pid to be the last argument and it is giving
> "*ustctl[1813/1813]: Error: The pid "ust/1" is invalid. (in
> parse_opts_long() at ustctl.c:159)*" error. Hence it would be "$ ustctl
> --enable-marker ust/mymark 1234". I am not sure of this though. Can you
> clarify?

You are absolutely right. It's now fixed. Thanks for reporting this!

pmf