[lttng-dev] Can't trace function calls

[Jonathan Rajotte-Julien]

On Wed, Mar 31, 2021 at 12:55:53PM -0500, Ramesh Errabolu wrote:
> root at RocrLnx23:~/git/compute/out/ubuntu-18.04/18.04/bin# ls ~/ | grep -i ltt
> root at RocrLnx23:~/git/compute/out/ubuntu-18.04/18.04/bin# lttng create
> my-kernel-session --output=~/my-kernel-trace
> Session my-kernel-session created.
> Traces will be output to
> /home/user1/git/compute/out/ubuntu-18.04/18.04/bin/~/my-kernel-trace
> root at RocrLnx23:~/git/compute/out/ubuntu-18.04/18.04/bin# *lttng list
> --kerne*l
> *Error: Unable to list kernel events: Kernel tracer not available*

Well this would be the first thing to look at.

First let's deactivate the lttng-sessiond.service installed by the packages.

   systemctl stop lttng-sessiond.service

You might want to reanable it later.

> 
> root at RocrLnx23:~/git/compute/out/ubuntu-18.04/18.04/bin# ps -ef | grep ltt
> root      1002     1  0 12:16 ?        00:00:00 /usr/bin/lttng-sessiond
> root      1054  1002  0 12:16 ?        00:00:00 */usr/bin/lttng-sessiond*
> root      3145  2861  0 12:51 pts/0    00:00:00 grep --color=auto ltt
> root at RocrLnx23:~/git/compute/out/ubuntu-18.04/18.04/bin#

Make sure after the systemctl call that no lttng-sessiond process is running.

Now let's launch a lttng-sessiond by hand with a bit more verbosity. For now
let's stick to the root user.

  # lttng-sessiond -vvv --verbose-consumer -b > /tmp/lttng-sessiond.log 2>&1 
  # pkill lttng-sessiond

Please share the content of /tmp/lttng-sessiond.log using a pasting service
(paste.ubuntu.com).

Also please provide the output of:

 # uname -a
 # find /lib/modules/$(uname -r)/ | grep lttng
 # dmesg | grep lttng


But again, the cyg-profile helper library is meant for Userspace tracing.

With the app running and having the LD_PRELOAD correctly set, and a sessiond
running.

 # lttng list -u 

If there is nothing, well you can start the application with the following and
share the output of it (make sure to remove any output from your application if
sensitive data is present)

 # LD_PRELOAD=.... LTTNG_UST_DEBUG=1 your_application_here

Note that debug log will be outputted on stderr.

Cheers

-- 
Jonathan Rajotte-Julien
EfficiOS