[lttng-dev] Problem with perf:thread in 2.5.0
Jonathan Rajotte-Julien
jonathan.rajotte-julien at ericsson.com
Tue Jul 29 10:05:49 EDT 2014
Hey Evgeniy,
At first I would says that this is proper behaviour. If we look into the lttng-module README file there is little paragraph about "Attach 'context; information to events in the trace".
- Have the ability to attach "context" information to events in the
trace (e.g. any PMU counter, pid, ppid, tid, comm name, etc).
All the extra information fields to be collected with events are
optional, specified on a per-tracing-session basis (except for
timestamp and event id, which are mandatory).
I would assume that the absence of lttng-module disable the ability to attach 'context'.
If we read a little more down the readme file we get to this:
The following kernel configuration options will affect the features
available from LTTng:
...
CONFIG_PERF_EVENTS:
performance counters
lttng add-context -t perf:*
...
This seems to confirm my assumption. I won't have time today to test without kernel to really confirm this sorry.
BUT if I'm wrong, I'm sure someone from efficiOS will jump in and correct me. :P
If this is actually correct behaviour, I invite you to open a bug for lttng-tool about it not warning the user that context attachment is not possible with no lttng-module (if this is possible).
Cheers,
Jonathan
On 07/29/2014 08:32 AM, Evgeniy Ivanov wrote:
> Hey Jonathan,
>
> On the machine with zeroes in results for perf counters, there is no lttng modules. Is it proper behavior then?
>
>
> On Fri, Jul 25, 2014 at 11:33 PM, Evgeniy Ivanov <i at eivanov.com <mailto:i at eivanov.com>> wrote:
>
> Hey Jonathan,
>
> Thanks a lot for help! I think I've found the reason of invalid context messages: I reran root's lttng-sessiond, but didn't check sessions for my user (since I've done 'destroy -a'). Today again I got issue with invalid context and noticed, that there are two pairs of lttng-sessiond and lttng-consumerd: from old version and from new version according dates. Old one is from July 7, which is surprising, since I often do "destroy -a". After killing everything seems to work. Btw, it's Oracle linux 6.5 with linux 3.8.
>
> I'll check that zeros after weekend: I don't have own access to that machine.
>
>
>
>
> On Fri, Jul 25, 2014 at 1:07 AM, Jonathan Rajotte-Julien <jonathan.rajotte-julien at ericsson.com <mailto:jonathan.rajotte-julien at ericsson.com>> wrote:
>
>
>
> On 07/24/2014 04:46 PM, Jonathan Rajotte-Julien wrote:
> >
> > Hey Evgeniy,
> >
> >
> >> Jonathan, sorry, I've copied wrong output, original command was with perf:thread:cpu-cycles:
> >>
> >>> lttng add-context -u -t vpid -t perf:thread:cpu-cycles -t perf:thread:instructions -t perf:thread:cache-misses
> >> Session auto-20140724-091524 created.
> >> Traces will be written in /export/home/eiva/lttng-traces/auto-20140724-091524
> >> UST context vpid added to all channels
> >> Error: perf:thread:cpu-cycles: UST invalid context
> >> Error: perf:thread:instructions: UST invalid context
> >> Error: perf:thread:cache-misses: UST invalid context
> >> Warning: Some command(s) went wrong
> >> UST event eiva_test:* created in channel channel0
> >
> > Weird, it works on my side (c&p) with latest version witch is 2.5 stable :P.
>
> I just tested a fresh install from stable ppa (https://launchpad.net/~lttng/+archive/ubuntu/ppa).
>
> and everything seems to work fine. Cycle,cache etc.
>
> Well, keep us up to date.
>
> Cheers
> Jonathan
>
> >
> >>
> >>> lttng --version
> >> lttng (LTTng Trace Control) 2.5.0 - Fumisterie
> >
> > From this output I guess you are working with the lttng package and not the source from git. I don't think packaging is the problem here but let just keep that in mind.
> > What dist are you on ?
> >
> > I'm sure you already done it a thousand time but could you just run these and send the output ?
> >
> > First make sure no lttng-sessiond is running then launch:
> >
> > lttng-sessiond -vvv
> >
> > In a separate terminal:
> >
> > lttng destroy -a // Just in case
> > lttng -vvv create test
> >
> > lttng -vvv add-context -u -t vpid -t perf:thread:cpu-cycles -t perf:thread:instructions -t perf:thread:cache-misses
> >
> > If this work, you could test if you get the data or zero.
> >
> > lttng enable-event -u -a -s test
> >
> > lttng start
> > // Run the program
> >
> > lttng stop
> >
> > Make sure to send the output from lttng-sessiond -vvv.
> >
> >
> > Cheers,
> > Jonathan
> >
> >>
> >> Later I've tried cpu-cycles separately and then accidentally switched to perf:cpu. Now everything started to work, but nothing changed since then. Though didn't check if we still get zeroes on another machine.
> >>
> >>
> >> On Thu, Jul 24, 2014 at 7:01 PM, Jonathan Rajotte-Julien <jonathan.rajotte-julien at ericsson.com <mailto:jonathan.rajotte-julien at ericsson.com> <mailto:jonathan.rajotte-julien at ericsson.com <mailto:jonathan.rajotte-julien at ericsson.com>>> wrote:
> >>
> >> Hi!
> >>
> >> > Hi!
> >> >
> >> > I use lttng-2.5.0 (tools, modules, …) and linux-3.8.13. When I try to add context for performance counters I get the following error:
> >> >
> >> >> lttng add-context -u -t perf:cpu:cpu-cycles
> >> > Error: perf:cpu:cpu-cycles: UST invalid context
> >> > Warning: Some command(s) went wrong
> >>
> >> Well i think we will both agree that the help page does not help very much.(Should fix that)
> >>
> >> If we go deep down into the code we get the supported context for UST domain. :)
> >>
> >> Ust domain support 6 type of context:
> >> LTTNG_EVENT_CONTEXT_VTID
> >> LTTNG_EVENT_CONTEXT_VPID
> >> LTTNG_UST_CONTEXT_VPID
> >> LTTNG_EVENT_CONTEXT_PTHREAD_ID
> >> LTTNG_EVENT_CONTEXT_PROCNAME
> >> LTTNG_EVENT_CONTEXT_IP
> >> LTTNG_EVENT_CONTEXT_PERF_THREAD_COUNTER
> >>
> >> Well this does help a little ... If we dig a little deeper and search around we get more info and see that an enum from "./lttng-tools/include/lttng/event.h" lttng_event_context_t maps to another one in the lttng-tool side ( context_type in add_context.c)
> >>
> >> *Important Part*
> >> So if we decrypt the macros and functions in add-context.c, we get that the following contexts are supported for UST:
> >>
> >> vpid
> >> vtid
> >> pthread_id
> >> procname
> >> ip
> >>
> >> perf:thread:cpu-cycles
> >> perf:thread:cycles
> >> perf:thread:stalled-cycles-frontend
> >> perf:thread:idle-cycles-frontend
> >> perf:thread:stalled-cycles-backend
> >> perf:thread:idle-cycles-backend
> >> perf:thread:instructions
> >> perf:thread:cache-references
> >> perf:thread:cache-misses
> >> perf:thread:branch-instructions
> >> perf:thread:branches
> >> perf:thread:branch-misses
> >> perf:thread:bus-cycles
> >>
> >> perf:thread:L1-dcache
> >> perf:thread:L1-icache
> >> perf:thread:LLC
> >> perf:thread:dTLB
> >> perf:thread:iTLB-loads
> >> perf:thread:iTLB-load-misses
> >> perf:thread:branch-loads
> >> perf:thread:branch-load-misses
> >>
> >> perf:thread:cpu-clock
> >> perf:thread:task-clock
> >> perf:thread:page-fault
> >> perf:thread:faults
> >> perf:thread:major-faults
> >> perf:thread:minor-faults
> >> perf:thread:context-switches
> >> perf:thread:cs
> >> perf:thread:cpu-migrations
> >> perf:thread:migrations
> >> perf:thread:alignment-faults
> >> perf:thread:emulation-faults
> >>
> >> **(TL;DR : perf:cpu:cpu-cycles is not in the list)**
> >>
> >> If you want perf:cpu:cpu-cycles you will need a kernel tracing domain.
> >>
> >> Even if it make sense that a ust tracing session cannot touch hardware stuff it won’t hurt to be more explicit about it. :).
> >>
> >> >
> >> > We have another machine with 2.5.0 (modules are not installed though) and adding perf:cpu:cpu-cycles works fine, but it is always zero in results.
> >>
> >> As for this, I cannot explain why lttng-tool does not complain on adding the context.
> >> You can open a bug here http://bugs.lttng.org/projects/lttng-tools (you will need to create an account and sign in)
> >>
> >> Cheers,
> >> Jonathan
> >>
> >> >
> >> > --
> >> > Cheers,
> >> > Evgeniy
> >> >
> >> >
> >> > _______________________________________________
> >> > lttng-dev mailing list
> >> > lttng-dev at lists.lttng.org <mailto:lttng-dev at lists.lttng.org> <mailto:lttng-dev at lists.lttng.org <mailto:lttng-dev at lists.lttng.org>>
> >> > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> >> >
> >>
> >>
> >>
> >>
> >> --
> >> Cheers,
> >> Evgeniy
> >
> >
> > _______________________________________________
> > lttng-dev mailing list
> > lttng-dev at lists.lttng.org <mailto:lttng-dev at lists.lttng.org>
> > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> >
>
>
> _______________________________________________
> lttng-dev mailing list
> lttng-dev at lists.lttng.org <mailto:lttng-dev at lists.lttng.org>
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>
>
>
>
> --
> Cheers,
> Evgeniy
>
>
>
>
> --
> Cheers,
> Evgeniy
More information about the lttng-dev
mailing list