[lttng-dev] Latency tracing using LTTng

[Anup Pemmaiah]

Hi,

I am trying to find the reason of a intermittent latency that I am noticing
on my Linux system running on a RT_PREEMPT kernel. I have patched it on a
Ubuntu 16.04 LTS. Have stopped the desktop lightdm service and this is on a
Arm aarch64 platform.

At a very high level, there are two processes interacting with each other.
Process-1 does some work and sends UDP messages to Process-2 (Using the
FASTRTPS real time Publish-Subscribe protocol).
Data is sent from Process-1 to Process-2 every 100ms. I see that often
there is a delay in Process-2 receiving the message.


Trying to use LTTng and along with other supported profiling tools like
latency-tracker, lttng-profile.
Something like the description in this link
http://lttng.org/blog/2015/03/18/full-stack-latencies is what I am looking
for, but I am facing issues. The latency_tracker kernel modules and also
lttng-profile is a bit flaky as it sometimes work and sometimes panics the
kernel.  Simple test cases as provided in
https://github.com/fdoray/latency_tracker_demo works sometimes. And also
profiling Process-1 works by providing some useful traces(intermittently
kernel panics), but profiling Process-2 the process just hangs.

Not sure how updated the above pasted links are, because I notice there is
no more usec_threshold parameter and I use the threshold
in /sys/kernel/debug/latency/syscalls/default/threshold

I understand each situation is different, but was curious if  any of you
have comments/suggestions that would help in debugging latency related
issues using LTTng. I have looked many places to get required info but of
not much help.

Your comments/suggestions will be greatly appreciated.

Thanks
Anup
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.lttng.org/pipermail/lttng-dev/attachments/20171227/c32193f7/attachment.html>