[lttng-dev] Problem with perf:thread in 2.5.0

Evgeniy Ivanov i at eivanov.com
Thu Jul 31 09:55:37 EDT 2014


Hey Jonathan,

Thanks a lot for your help and clarifications. Sounds reasonable, if
someone disagrees, I'll file a bug and do additional testing.


On Tue, Jul 29, 2014 at 6:05 PM, Jonathan Rajotte-Julien <
jonathan.rajotte-julien at ericsson.com> wrote:

> Hey Evgeniy,
>
> At first I would says that this is proper behaviour. If we look into the
> lttng-module README file there is little paragraph about "Attach 'context;
> information to events in the trace".
>
> - Have the ability to attach "context" information to events in the
>   trace (e.g. any PMU counter, pid, ppid, tid, comm name, etc).
>   All the extra information fields to be collected with events are
>   optional, specified on a per-tracing-session basis (except for
>   timestamp and event id, which are mandatory).
>
> I would assume that the absence of lttng-module disable the ability to
> attach 'context'.
>
> If we read a little more down the readme file we get to this:
>
> The following kernel configuration options will affect the features
> available from LTTng:
>
> ...
>
> CONFIG_PERF_EVENTS:
>             performance counters
>                lttng add-context -t perf:*
>
> ...
>
> This seems to confirm my assumption. I won't have time today to test
> without kernel to really confirm this sorry.
>
> BUT if I'm wrong, I'm sure someone from efficiOS will jump in and correct
> me.  :P
>
> If this is actually correct behaviour, I invite you to open a bug for
> lttng-tool about it not warning the user that context attachment is not
> possible with no lttng-module (if this is possible).
>
> Cheers,
> Jonathan
>
> On 07/29/2014 08:32 AM, Evgeniy Ivanov wrote:
> > Hey Jonathan,
> >
> > On the machine with zeroes in results for perf counters, there is no
> lttng modules. Is it proper behavior then?
> >
> >
> > On Fri, Jul 25, 2014 at 11:33 PM, Evgeniy Ivanov <i at eivanov.com <mailto:
> i at eivanov.com>> wrote:
> >
> >     Hey Jonathan,
> >
> >     Thanks a lot for help! I think I've found the reason of invalid
> context messages: I reran root's lttng-sessiond, but didn't check sessions
> for my user (since I've done 'destroy -a'). Today again I got issue with
> invalid context and noticed, that there are two pairs of lttng-sessiond and
> lttng-consumerd: from old version and from new version according dates. Old
> one is from July 7, which is surprising, since I often do "destroy -a".
> After killing everything seems to work. Btw, it's Oracle linux 6.5 with
> linux 3.8.
> >
> >     I'll check that zeros after weekend: I don't have own access to that
> machine.
> >
> >
> >
> >
> >     On Fri, Jul 25, 2014 at 1:07 AM, Jonathan Rajotte-Julien <
> jonathan.rajotte-julien at ericsson.com <mailto:
> jonathan.rajotte-julien at ericsson.com>> wrote:
> >
> >
> >
> >         On 07/24/2014 04:46 PM, Jonathan Rajotte-Julien wrote:
> >         >
> >         > Hey Evgeniy,
> >         >
> >         >
> >         >> Jonathan, sorry, I've copied wrong output, original command
> was with perf:thread:cpu-cycles:
> >         >>
> >         >>> lttng add-context -u -t vpid -t perf:thread:cpu-cycles -t
> perf:thread:instructions -t perf:thread:cache-misses
> >         >> Session auto-20140724-091524 created.
> >         >> Traces will be written in
> /export/home/eiva/lttng-traces/auto-20140724-091524
> >         >> UST context vpid added to all channels
> >         >> Error: perf:thread:cpu-cycles: UST invalid context
> >         >> Error: perf:thread:instructions: UST invalid context
> >         >> Error: perf:thread:cache-misses: UST invalid context
> >         >> Warning: Some command(s) went wrong
> >         >> UST event eiva_test:* created in channel channel0
> >         >
> >         > Weird, it works on my side (c&p) with latest version witch is
> 2.5 stable :P.
> >
> >         I just tested a fresh install from stable ppa (
> https://launchpad.net/~lttng/+archive/ubuntu/ppa).
> >
> >         and everything seems to work fine. Cycle,cache etc.
> >
> >         Well, keep us up to date.
> >
> >         Cheers
> >         Jonathan
> >
> >         >
> >         >>
> >         >>> lttng --version
> >         >> lttng (LTTng Trace Control) 2.5.0 - Fumisterie
> >         >
> >         > From this output I guess you are working with the lttng
> package and not the source from git. I don't think packaging is the problem
> here but let just keep that in mind.
> >         > What dist are you on ?
> >         >
> >         > I'm sure you already done it a thousand time but could you
> just run these and send the output ?
> >         >
> >         > First make sure no lttng-sessiond is running then launch:
> >         >
> >         > lttng-sessiond -vvv
> >         >
> >         > In a separate terminal:
> >         >
> >         > lttng destroy -a  // Just in case
> >         > lttng -vvv create test
> >         >
> >         > lttng -vvv add-context -u -t vpid -t perf:thread:cpu-cycles -t
> perf:thread:instructions -t perf:thread:cache-misses
> >         >
> >         > If this work, you could test if you get the data or zero.
> >         >
> >         > lttng enable-event -u -a -s test
> >         >
> >         > lttng start
> >         > // Run the program
> >         >
> >         > lttng stop
> >         >
> >         > Make sure to send the output from lttng-sessiond -vvv.
> >         >
> >         >
> >         > Cheers,
> >         > Jonathan
> >         >
> >         >>
> >         >> Later I've tried cpu-cycles separately and then accidentally
> switched to perf:cpu. Now everything started to work, but nothing changed
> since then. Though didn't check if we still get zeroes on another machine.
> >         >>
> >         >>
> >         >> On Thu, Jul 24, 2014 at 7:01 PM, Jonathan Rajotte-Julien <
> jonathan.rajotte-julien at ericsson.com <mailto:
> jonathan.rajotte-julien at ericsson.com> <mailto:
> jonathan.rajotte-julien at ericsson.com <mailto:
> jonathan.rajotte-julien at ericsson.com>>> wrote:
> >         >>
> >         >>     Hi!
> >         >>
> >         >>     > Hi!
> >         >>     >
> >         >>     > I use lttng-2.5.0 (tools, modules, …) and linux-3.8.13.
> When I try to add context for performance counters I get the following
> error:
> >         >>     >
> >         >>     >> lttng add-context -u -t perf:cpu:cpu-cycles
> >         >>     > Error: perf:cpu:cpu-cycles: UST invalid context
> >         >>     > Warning: Some command(s) went wrong
> >         >>
> >         >>     Well i think we will both agree that the help page does
> not help very much.(Should fix that)
> >         >>
> >         >>     If we go deep down into the code we get the supported
> context for UST domain. :)
> >         >>
> >         >>     Ust domain support 6 type of context:
> >         >>     LTTNG_EVENT_CONTEXT_VTID
> >         >>     LTTNG_EVENT_CONTEXT_VPID
> >         >>     LTTNG_UST_CONTEXT_VPID
> >         >>     LTTNG_EVENT_CONTEXT_PTHREAD_ID
> >         >>     LTTNG_EVENT_CONTEXT_PROCNAME
> >         >>     LTTNG_EVENT_CONTEXT_IP
> >         >>     LTTNG_EVENT_CONTEXT_PERF_THREAD_COUNTER
> >         >>
> >         >>     Well this does help a little ... If we dig a little
> deeper and search around we get more info and see that an enum from
> "./lttng-tools/include/lttng/event.h" lttng_event_context_t maps to another
> one in the lttng-tool side ( context_type in add_context.c)
> >         >>
> >         >>     *Important Part*
> >         >>     So if we decrypt the macros and functions in
> add-context.c, we get that the following contexts are supported for UST:
> >         >>
> >         >>     vpid
> >         >>     vtid
> >         >>     pthread_id
> >         >>     procname
> >         >>     ip
> >         >>
> >         >>     perf:thread:cpu-cycles
> >         >>     perf:thread:cycles
> >         >>     perf:thread:stalled-cycles-frontend
> >         >>     perf:thread:idle-cycles-frontend
> >         >>     perf:thread:stalled-cycles-backend
> >         >>     perf:thread:idle-cycles-backend
> >         >>     perf:thread:instructions
> >         >>     perf:thread:cache-references
> >         >>     perf:thread:cache-misses
> >         >>     perf:thread:branch-instructions
> >         >>     perf:thread:branches
> >         >>     perf:thread:branch-misses
> >         >>     perf:thread:bus-cycles
> >         >>
> >         >>     perf:thread:L1-dcache
> >         >>     perf:thread:L1-icache
> >         >>     perf:thread:LLC
> >         >>     perf:thread:dTLB
> >         >>     perf:thread:iTLB-loads
> >         >>     perf:thread:iTLB-load-misses
> >         >>     perf:thread:branch-loads
> >         >>     perf:thread:branch-load-misses
> >         >>
> >         >>     perf:thread:cpu-clock
> >         >>     perf:thread:task-clock
> >         >>     perf:thread:page-fault
> >         >>     perf:thread:faults
> >         >>     perf:thread:major-faults
> >         >>     perf:thread:minor-faults
> >         >>     perf:thread:context-switches
> >         >>     perf:thread:cs
> >         >>     perf:thread:cpu-migrations
> >         >>     perf:thread:migrations
> >         >>     perf:thread:alignment-faults
> >         >>     perf:thread:emulation-faults
> >         >>
> >         >>     **(TL;DR : perf:cpu:cpu-cycles is not in the list)**
> >         >>
> >         >>     If you want perf:cpu:cpu-cycles you will need a kernel
> tracing domain.
> >         >>
> >         >>     Even if it make sense that a ust tracing session cannot
> touch hardware stuff it won’t hurt to be more explicit about it. :).
> >         >>
> >         >>     >
> >         >>     > We have another machine with 2.5.0 (modules are not
> installed though) and adding perf:cpu:cpu-cycles works fine, but it is
> always zero in results.
> >         >>
> >         >>     As for this, I cannot explain why lttng-tool does not
> complain on adding the context.
> >         >>     You can open a bug here
> http://bugs.lttng.org/projects/lttng-tools (you will need to create an
> account and sign in)
> >         >>
> >         >>     Cheers,
> >         >>     Jonathan
> >         >>
> >         >>     >
> >         >>     > --
> >         >>     > Cheers,
> >         >>     > Evgeniy
> >         >>     >
> >         >>     >
> >         >>     > _______________________________________________
> >         >>     > lttng-dev mailing list
> >         >>     > lttng-dev at lists.lttng.org <mailto:
> lttng-dev at lists.lttng.org> <mailto:lttng-dev at lists.lttng.org <mailto:
> lttng-dev at lists.lttng.org>>
> >         >>     >
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> >         >>     >
> >         >>
> >         >>
> >         >>
> >         >>
> >         >> --
> >         >> Cheers,
> >         >> Evgeniy
> >         >
> >         >
> >         > _______________________________________________
> >         > lttng-dev mailing list
> >         > lttng-dev at lists.lttng.org <mailto:lttng-dev at lists.lttng.org>
> >         > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> >         >
> >
> >
> >         _______________________________________________
> >         lttng-dev mailing list
> >         lttng-dev at lists.lttng.org <mailto:lttng-dev at lists.lttng.org>
> >         http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> >
> >
> >
> >
> >     --
> >     Cheers,
> >     Evgeniy
> >
> >
> >
> >
> > --
> > Cheers,
> > Evgeniy
>
>


-- 
Cheers,
Evgeniy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lttng.org/pipermail/lttng-dev/attachments/20140731/ff988c63/attachment-0001.html>


More information about the lttng-dev mailing list