From francis.giraldeau at gmail.com Fri Jun 1 08:22:28 2012 From: francis.giraldeau at gmail.com (Francis Giraldeau) Date: Fri, 01 Jun 2012 14:22:28 +0200 Subject: [lttng-dev] Add pid context to UST events Message-ID: <4FC8B404.2080403@gmail.com> Hi, I tried to add pid context to UST events to distinguish events from multiple processes. I do have this error: $ lttng create bidon Session bidon created. Traces will be written in /home/francis/lttng-traces/bidon-20120601-142031 $ lttng enable-event -a -u All UST events are enabled in channel channel0 $ lttng add-context -u -t pid Error: pid: UST invalid context Warning: Some command(s) went wrong Am I doing something wrong? Thanks! Francis -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4476 bytes Desc: Signature cryptographique S/MIME URL: From mathieu.desnoyers at efficios.com Fri Jun 1 11:53:48 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Fri, 1 Jun 2012 11:53:48 -0400 Subject: [lttng-dev] Add pid context to UST events In-Reply-To: <4FC8B404.2080403@gmail.com> References: <4FC8B404.2080403@gmail.com> Message-ID: <20120601155348.GA13775@Krystal> * Francis Giraldeau (francis.giraldeau at gmail.com) wrote: > Hi, > > I tried to add pid context to UST events to distinguish events from > multiple processes. I do have this error: > > $ lttng create bidon > Session bidon created. > Traces will be written in /home/francis/lttng-traces/bidon-20120601-142031 > > $ lttng enable-event -a -u > All UST events are enabled in channel channel0 > > $ lttng add-context -u -t pid > Error: pid: UST invalid context > Warning: Some command(s) went wrong > > Am I doing something wrong? lttng add-context -u -t vpid should do it. (notice the "v"). Eventually we should implement something that looks like a lttng list -u --context to list the per-domain contexts available. Thanks, Mathieu > > Thanks! > > Francis > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From bhufmann at gmail.com Fri Jun 1 13:02:20 2012 From: bhufmann at gmail.com (Bernd Hufmann) Date: Fri, 1 Jun 2012 13:02:20 -0400 Subject: [lttng-dev] Add pid context to UST events In-Reply-To: <20120601155348.GA13775@Krystal> References: <4FC8B404.2080403@gmail.com> <20120601155348.GA13775@Krystal> Message-ID: Hi Mathieu This is an important information. Is this documented? The Eclipse implementation of the Trace Control doesn't support that. It also uses "-t pid" for UST. Could you please list all the differences for contexts name for UST? Maybe I can squeeze in a fix before the Eclipse Juno release. Thanks Bernd On Fri, Jun 1, 2012 at 11:53 AM, Mathieu Desnoyers wrote: > * Francis Giraldeau (francis.giraldeau at gmail.com) wrote: >> Hi, >> >> I tried to add pid context to UST events to distinguish events from >> multiple processes. I do have this error: >> >> $ lttng create bidon >> Session bidon created. >> Traces will be written in /home/francis/lttng-traces/bidon-20120601-142031 >> >> $ lttng enable-event -a -u >> All UST events are enabled in channel channel0 >> >> $ lttng add-context -u -t pid >> Error: pid: UST invalid context >> Warning: Some command(s) went wrong >> >> Am I doing something wrong? > > lttng add-context -u -t vpid > > should do it. (notice the "v"). > > Eventually we should implement something that looks like a > > lttng list -u --context > > to list the per-domain contexts available. > > Thanks, > > Mathieu > >> >> Thanks! >> >> Francis >> > > > >> _______________________________________________ >> lttng-dev mailing list >> lttng-dev at lists.lttng.org >> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > > -- > Mathieu Desnoyers > Operating System Efficiency R&D Consultant > EfficiOS Inc. > http://www.efficios.com > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev From mathieu.desnoyers at efficios.com Fri Jun 1 13:14:09 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Fri, 1 Jun 2012 13:14:09 -0400 Subject: [lttng-dev] Add pid context to UST events In-Reply-To: References: <4FC8B404.2080403@gmail.com> <20120601155348.GA13775@Krystal> Message-ID: <20120601171409.GA14574@Krystal> * Bernd Hufmann (bhufmann at gmail.com) wrote: > Hi Mathieu > > This is an important information. Is this documented? > The Eclipse implementation of the Trace Control doesn't support that. > It also uses "-t pid" for UST. > Could you please list all the differences for contexts name for UST? > Maybe I can squeeze in a fix > before the Eclipse Juno release. Sure. The contexts supported by LTTng-UST 2.0 are: procname, pthread_id, vpid, vtid Thanks, Mathieu > > Thanks > Bernd > > > On Fri, Jun 1, 2012 at 11:53 AM, Mathieu Desnoyers > wrote: > > * Francis Giraldeau (francis.giraldeau at gmail.com) wrote: > >> Hi, > >> > >> I tried to add pid context to UST events to distinguish events from > >> multiple processes. I do have this error: > >> > >> $ lttng create bidon > >> Session bidon created. > >> Traces will be written in /home/francis/lttng-traces/bidon-20120601-142031 > >> > >> $ lttng enable-event -a -u > >> All UST events are enabled in channel channel0 > >> > >> $ lttng add-context -u -t pid > >> Error: pid: UST invalid context > >> Warning: Some command(s) went wrong > >> > >> Am I doing something wrong? > > > > lttng add-context -u -t vpid > > > > should do it. (notice the "v"). > > > > Eventually we should implement something that looks like a > > > > lttng list -u --context > > > > to list the per-domain contexts available. > > > > Thanks, > > > > Mathieu > > > >> > >> Thanks! > >> > >> Francis > >> > > > > > > > >> _______________________________________________ > >> lttng-dev mailing list > >> lttng-dev at lists.lttng.org > >> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > > > > > -- > > Mathieu Desnoyers > > Operating System Efficiency R&D Consultant > > EfficiOS Inc. > > http://www.efficios.com > > > > _______________________________________________ > > lttng-dev mailing list > > lttng-dev at lists.lttng.org > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From bhufmann at gmail.com Fri Jun 1 13:18:14 2012 From: bhufmann at gmail.com (Bernd Hufmann) Date: Fri, 1 Jun 2012 13:18:14 -0400 Subject: [lttng-dev] Add pid context to UST events In-Reply-To: <20120601171409.GA14574@Krystal> References: <4FC8B404.2080403@gmail.com> <20120601155348.GA13775@Krystal> <20120601171409.GA14574@Krystal> Message-ID: Thanks On Fri, Jun 1, 2012 at 1:14 PM, Mathieu Desnoyers wrote: > * Bernd Hufmann (bhufmann at gmail.com) wrote: >> Hi Mathieu >> >> This is an important information. Is this documented? >> The Eclipse implementation of the Trace Control doesn't support that. >> It also uses "-t pid" for UST. >> Could you please list all the differences for contexts name for UST? >> Maybe I can squeeze in a fix >> before the Eclipse Juno release. > > Sure. The contexts supported by LTTng-UST 2.0 are: > > procname, pthread_id, vpid, vtid > > Thanks, > > Mathieu > >> >> Thanks >> Bernd >> >> >> On Fri, Jun 1, 2012 at 11:53 AM, Mathieu Desnoyers >> wrote: >> > * Francis Giraldeau (francis.giraldeau at gmail.com) wrote: >> >> Hi, >> >> >> >> I tried to add pid context to UST events to distinguish events from >> >> multiple processes. I do have this error: >> >> >> >> $ lttng create bidon >> >> Session bidon created. >> >> Traces will be written in /home/francis/lttng-traces/bidon-20120601-142031 >> >> >> >> $ lttng enable-event -a -u >> >> All UST events are enabled in channel channel0 >> >> >> >> $ lttng add-context -u -t pid >> >> Error: pid: UST invalid context >> >> Warning: Some command(s) went wrong >> >> >> >> Am I doing something wrong? >> > >> > lttng add-context -u -t vpid >> > >> > should do it. (notice the "v"). >> > >> > Eventually we should implement something that looks like a >> > >> > lttng list -u --context >> > >> > to list the per-domain contexts available. >> > >> > Thanks, >> > >> > Mathieu >> > >> >> >> >> Thanks! >> >> >> >> Francis >> >> >> > >> > >> > >> >> _______________________________________________ >> >> lttng-dev mailing list >> >> lttng-dev at lists.lttng.org >> >> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev >> > >> > >> > -- >> > Mathieu Desnoyers >> > Operating System Efficiency R&D Consultant >> > EfficiOS Inc. >> > http://www.efficios.com >> > >> > _______________________________________________ >> > lttng-dev mailing list >> > lttng-dev at lists.lttng.org >> > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > -- > Mathieu Desnoyers > Operating System Efficiency R&D Consultant > EfficiOS Inc. > http://www.efficios.com From mathieu.desnoyers at efficios.com Fri Jun 1 14:05:33 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Fri, 1 Jun 2012 14:05:33 -0400 Subject: [lttng-dev] [RELEASE] Userspace RCU 0.7.3 Message-ID: <20120601180533.GA15522@Krystal> liburcu is a LGPLv2.1 userspace RCU (read-copy-update) library. This data synchronization library provides read-side access which scales linearly with the number of cores. It does so by allowing multiples copies of a given data structure to live at the same time, and by monitoring the data structure accesses to detect grace periods after which memory reclamation is possible. liburcu-cds provides efficient data structures based on RCU and lock-free algorithms. Those structures include hash tables, queues, stacks, and doubly-linked lists. This is a minor compatibility-related release, fixing build issues with FreeBSD and NetBSD. On Linux, only the test_perthreadlock fix could change the result of make check (which could previously fail due to non-initialized mutexes), but it does not impact anything installed on the system. Changelog: 2012-06-01 Userspace RCU 0.7.3 * Fix tests: make dist lib dependency * Update README for OS supported, tests dependency * Add CodingStyle to tarball * Add coding style document * Test fix: test_perthreadlock uninitialized mutex * tests: support FreeBSD short "time" args * freebsd 8.2 fix: define MAP_ANONYMOUS for compatibility Project website: http://lttng.org/urcu Download link: http://lttng.org/files/urcu/ -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Fri Jun 1 14:17:26 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Fri, 1 Jun 2012 14:17:26 -0400 Subject: [lttng-dev] [RELEASE] LTTng-UST 2.0.3 Message-ID: <20120601181726.GA15890@Krystal> LTTng-UST, the Linux Trace Toolkit Next Generation Userspace Tracer, is port of the low-overhead tracing capabilities of the LTTng kernel tracer to user-space. The library "liblttng-ust" enables tracing of applications and libraries. In this release, we drop the lttng-ust-java java tracing library, since the API was not deemed ready for use at this stage (no API exposed in 2.0, anyway, this was simply a library). We will work on it for 2.1. Also noteworthy in this release, signals are now blocked in listener UST threads, and /dev/shm filesystem full conditions are now handled so it never triggers an SIGBUS in a traced application. 2.0.x users are encouraged to upgrade. Changelog: 2012-06-01 lttng-ust 2.0.3 * Fix: Block all signals in listener thread * Remove jni support for configure * Fix: don't SIGBUS when filesystem is full * tracepoint: include stdio.h for NULL definition * manpage update: document that probes need gcc * Fix: remove # in front on extern "C" { * Cleanup: don't use GNU old-style field designator extension * Fix: remove padding field after variable sized array * Fix: lttng-ust.pc needs to specify -ldl * Update changelog after liblttng-ust-java removal * Fix: drop unusable liblttng-ust-java.so Project website: http://lttng.org Download link: http://lttng.org/download (please refer to the README file for installation instructions) -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Fri Jun 1 14:50:39 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Fri, 1 Jun 2012 14:50:39 -0400 Subject: [lttng-dev] [RELEASE] LTTng modules 2.0.3 Message-ID: <20120601185039.GA17058@Krystal> The LTTng modules provide Linux kernel tracing capability to the LTTng 2.0 tracer toolset. Except for the free_event_id minor fix, upgrading is really needed if compiling against a 3.4+ kernel. Changelog: 2012-06-01 LTTng modules 2.0.3 * Fix: free_event_id check should compare unsigned int with -1U * Fix: update signal instrumentation for 3.4 kernel Project website: http://lttng.org Download link: http://lttng.org/download (please refer to the README files for installation instructions and lttng-tools doc/quickstart.txt for usage information) -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From montarcyber at gmail.com Sun Jun 3 23:22:04 2012 From: montarcyber at gmail.com (tchak adim) Date: Sun, 3 Jun 2012 23:22:04 -0400 Subject: [lttng-dev] Displaying graphical results In-Reply-To: <524C960C5DFC794E82BE548D825F05CF283435F8@EU-MBX-01.mgc.mentorg.com> References: <524C960C5DFC794E82BE548D825F05CF283435F8@EU-MBX-01.mgc.mentorg.com> Message-ID: Thanks very much for your responses :) Regards. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mathieu.desnoyers at efficios.com Mon Jun 4 11:51:43 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Mon, 4 Jun 2012 11:51:43 -0400 Subject: [lttng-dev] [RELEASE] Userspace RCU 0.7.3 In-Reply-To: References: <20120601180533.GA15522@Krystal> Message-ID: <20120604155143.GA24551@Krystal> * Gerhard Mack (gmack at innerfire.net) wrote: > > Are there any online examples of how to use this library? I can't seem to > find any. The perfbook from Paul McKenney now uses userspace RCU in its examples (http://kernel.org/pub/linux/kernel/people/paulmck/perfbook/perfbook.html) Also, you will find various small programs in the source tree of the userspace-rcu packages under tests/ that act as test programs, and also show how to use the library. (in the git tree: http://git.lttng.org/?p=userspace-rcu.git;a=tree;f=tests;hb=HEAD) I guess setting up a tutorial HTML page from the test content would be valuable, we just have not had the time to do it at this point. Maybe setting up links to that documentation on the lttng.org/urcu web page would be a good start though. Alexandre, when you find a minute, can you look into this ? Thanks! Mathieu > > Gerhard > > > > On Fri, 1 Jun 2012, Mathieu Desnoyers wrote: > > > Date: Fri, 1 Jun 2012 14:05:33 -0400 > > From: Mathieu Desnoyers > > To: linux-kernel at vger.kernel.org, lttng-dev at lists.lttng.org, > > rp at svcs.cs.pdx.edu > > Cc: Paul E. McKenney , > > Lai Jiangshan , > > Stephen Hemminger > > Subject: [RELEASE] Userspace RCU 0.7.3 > > > > liburcu is a LGPLv2.1 userspace RCU (read-copy-update) library. This > > data synchronization library provides read-side access which scales > > linearly with the number of cores. It does so by allowing multiples > > copies of a given data structure to live at the same time, and by > > monitoring the data structure accesses to detect grace periods after > > which memory reclamation is possible. > > > > liburcu-cds provides efficient data structures based on RCU and > > lock-free algorithms. Those structures include hash tables, queues, > > stacks, and doubly-linked lists. > > > > This is a minor compatibility-related release, fixing build issues with > > FreeBSD and NetBSD. On Linux, only the test_perthreadlock fix could > > change the result of make check (which could previously fail due to > > non-initialized mutexes), but it does not impact anything installed on > > the system. > > > > Changelog: > > 2012-06-01 Userspace RCU 0.7.3 > > * Fix tests: make dist lib dependency > > * Update README for OS supported, tests dependency > > * Add CodingStyle to tarball > > * Add coding style document > > * Test fix: test_perthreadlock uninitialized mutex > > * tests: support FreeBSD short "time" args > > * freebsd 8.2 fix: define MAP_ANONYMOUS for compatibility > > > > Project website: http://lttng.org/urcu > > Download link: http://lttng.org/files/urcu/ > > > > > > -- > Gerhard Mack > > gmack at innerfire.net > > <>< As a computer, I find your faith in technology amusing. -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From montarcyber at gmail.com Mon Jun 4 16:57:11 2012 From: montarcyber at gmail.com (tchak adim) Date: Mon, 4 Jun 2012 16:57:11 -0400 Subject: [lttng-dev] Displaying graphical results In-Reply-To: References: <524C960C5DFC794E82BE548D825F05CF283435F8@EU-MBX-01.mgc.mentorg.com> Message-ID: for Eclipse tool : i already follow the different steps to integrate the cft, lttng2 plugins as described in the link that u give it . i don't konw how can i visualize the graphics of lttng kernel module traces using eclipse , can u explain to me how ? For sourcery tool : i didn't find the source code, i'm using an ordinary i386 machine running ubuntu 10.04. Can u advice me and tell me what i can do ? thanks in advance ! -------------- next part -------------- An HTML attachment was scrubbed... URL: From mathieu.desnoyers at efficios.com Mon Jun 4 18:32:36 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Mon, 4 Jun 2012 18:32:36 -0400 Subject: [lttng-dev] [lttng-tools stable-2.0 merge req] Fix: close all file descriptors when executed as daemon Message-ID: <20120604223236.GA30869@Krystal> This commit from lttng-tools master should be merged into stable-2.0: commit ceed52b545103258e84f1c7040700be9dbbbaec6 Author: Mathieu Desnoyers Date: Mon Jun 4 18:08:24 2012 -0400 Fix: close all file descriptors when executed as daemon Both sessiond and consumerd support the option "-d" to run as daemon. In some specific cases, e.g. when launched from dpkg installation scripts, file descriptors 3, 4, 5 are left open and don't seem to have O_CLOEXEC flag set, so the install script hangs because the sessiond still holds a reference to them. daemon(3) only closes standard FD 0, 1, 2. Fix this issue by closing all file descriptors after calling daemon(3). Note: we make sure no file descriptor is opened before calling daemon(3) by moving the init_thread_quit_pipe() call after the FD close. Signed-off-by: Mathieu Desnoyers -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From toojays at toojays.net Mon Jun 4 19:19:53 2012 From: toojays at toojays.net (John Steele Scott) Date: Tue, 05 Jun 2012 08:49:53 +0930 Subject: [lttng-dev] UST "Tracepoint signature mismatch" with C99 bool. Message-ID: I originally sent this on the 22nd of May, but it looks like it got stuck in moderation since I'm not subscribed. Reposting via gmane.comp.sysutils.lttng.devel . . . I'm just getting started wiring my up to lttng-ust, and unfortunately ran into an issue with the first probe I tried, which used a C99 bool argument. The problem can be seen by patching the demo test from lttng-ust as follows: --- a/tests/demo/ust_tests_demo3.h +++ b/tests/demo/ust_tests_demo3.h @@ -22,12 +22,14 @@ extern "C" { * all copies or substantial portions of the Software. */ +#include + #include TRACEPOINT_EVENT(ust_tests_demo3, done, - TP_ARGS(int, value), + TP_ARGS(bool, value), TP_FIELDS( - ctf_integer(int, value, value) + ctf_integer(bool, value, value) ) ) Then when the demo is run with LTTNG_UST_DEBUG=1, a warning is shown, like: liblttng_ust_tracepoint[3315/3315]: Warning: Tracepoint signature mismatch, not enabling one or more tracepoints. Ensure that the tracepoint probes prototypes match the application. (in set_tracepoint() at tracepoint.c:310) liblttng_ust_tracepoint[3315/3315]: Warning: Tracepoint "ust_tests_demo3:done" signatures: call: "_Bool, value" vs probe: "bool, value". (in set_tracepoint() at tracepoint.c:312) It seems that TP_ARGS does not perform preprocessor expansion on the "bool" type spec, while something underneath TP_FIELDS does. And since (at least on this Centos 6.2 box) stdbool.h uses a #define rather than a typedef to make bool equivalent to _Bool, liblttng detects a mismatch. So in my tracepoint specifications, I need to remember to use the less attractive _Bool instead of bool. Perhaps it's worth having a special case for this where the trace/probe signatures are compared? On the other hand, I guess nobody complained until now? cheers, John From trost at cloud.rain.com Sat Jun 2 10:35:39 2012 From: trost at cloud.rain.com (Bill Trost) Date: Sat, 02 Jun 2012 07:35:39 -0700 Subject: [lttng-dev] using --function/--probe Message-ID: <3551.1338647739@cloud.rain.com> Hi, I'm trying to investigate a problem under Linux 2.6.39.4 involving kworker threads and am pretty much stymied. I've tried enabling various events using --function and --probe and it's been very hit or miss. Very often, "lttng enable-event" comes back with nothing more specific than "bad ioctl", but every once in a while I'll find a symbol I can add a probe point for. So: What does "--probe" mean? What does "--function" mean, and how does it differ from "--probe"? How do I determine what symbols are valid for each of these options? And, maybe as a worked example, how do I trace what work is being enqueued and run by the kworker threads? Thanks, Bill From alexandre.montplaisir at polymtl.ca Mon Jun 4 23:12:14 2012 From: alexandre.montplaisir at polymtl.ca (Alexandre Montplaisir) Date: Mon, 04 Jun 2012 23:12:14 -0400 Subject: [lttng-dev] UST "Tracepoint signature mismatch" with C99 bool. In-Reply-To: References: Message-ID: <4FCD790E.8080301@polymtl.ca> On 12-06-04 07:19 PM, John Steele Scott wrote: > I originally sent this on the 22nd of May, but it looks like it got stuck in moderation since I'm not subscribed. Reposting via gmane.comp.sysutils.lttng.devel . . . It seems the list stopped sending daily reminders again... I just flushed the moderation queue, there might be some 2-3 weeks old emails appearing. Please ignore them if they have already been replied to. Cheers, -- Alexandre Montplaisir DORSAL lab, ?cole Polytechnique de Montr?al From Fredrik_Oestman at mentor.com Tue Jun 5 03:06:37 2012 From: Fredrik_Oestman at mentor.com (Oestman, Fredrik) Date: Tue, 5 Jun 2012 07:06:37 +0000 Subject: [lttng-dev] Displaying graphical results In-Reply-To: References: <524C960C5DFC794E82BE548D825F05CF283435F8@EU-MBX-01.mgc.mentorg.com> Message-ID: <524C960C5DFC794E82BE548D825F05CF28344927@EU-MBX-01.mgc.mentorg.com> Hi Tchak, >For sourcery tool : >i didn't find the source code, i'm using an ordinary i386 machine running ubuntu 10.04. I'm afraid it's not an open-source tool. Cheers, Fredrik From Fredrik_Oestman at mentor.com Tue Jun 5 04:19:29 2012 From: Fredrik_Oestman at mentor.com (Oestman, Fredrik) Date: Tue, 5 Jun 2012 08:19:29 +0000 Subject: [lttng-dev] UST check pointer/de-reference order In-Reply-To: <524C960C5DFC794E82BE548D825F05CF28344927@EU-MBX-01.mgc.mentorg.com> References: <524C960C5DFC794E82BE548D825F05CF283435F8@EU-MBX-01.mgc.mentorg.com> <524C960C5DFC794E82BE548D825F05CF28344927@EU-MBX-01.mgc.mentorg.com> Message-ID: <524C960C5DFC794E82BE548D825F05CF28344978@EU-MBX-01.mgc.mentorg.com> I stumbled across some code where pointers are de-referenced and then checked for NULL. Cheers, Fredrik ?stman diff --git a/liblttng-ust-ctl/ustctl.c b/liblttng-ust-ctl/ustctl.c index 9789413..80aed04 100644 --- a/liblttng-ust-ctl/ustctl.c +++ b/liblttng-ust-ctl/ustctl.c @@ -732,7 +732,7 @@ void ustctl_unmap_channel(struct lttng_ust_shm_handle *handle) struct lttng_ust_lib_ring_buffer *ustctl_open_stream_read(struct lttng_ust_shm_handle *handle, int cpu) { - struct channel *chan = handle->shadow_chan; + struct channel *chan; int *shm_fd, *wait_fd; uint64_t *memory_map_size; struct lttng_ust_lib_ring_buffer *buf; @@ -741,6 +741,7 @@ struct lttng_ust_lib_ring_buffer *ustctl_open_stream_read(struct lttng_ust_shm_h if (!handle) return NULL; + chan = handle->shadow_chan; buf = channel_get_ring_buffer(&chan->backend.config, chan, cpu, handle, &shm_fd, &wait_fd, &memory_map_size); if (!buf) @@ -784,11 +785,12 @@ int ustctl_get_mmap_len(struct lttng_ust_shm_handle *handle, unsigned long *len) { unsigned long mmap_buf_len; - struct channel *chan = handle->shadow_chan; + struct channel *chan; if (!handle || !buf || !len) return -EINVAL; + chan = handle->shadow_chan; if (chan->backend.config.output != RING_BUFFER_MMAP) return -EINVAL; mmap_buf_len = chan->backend.buf_size; @@ -805,11 +807,12 @@ int ustctl_get_max_subbuf_size(struct lttng_ust_shm_handle *handle, struct lttng_ust_lib_ring_buffer *buf, unsigned long *len) { - struct channel *chan = handle->shadow_chan; + struct channel *chan; if (!handle || !buf || !len) return -EINVAL; + chan = handle->shadow_chan; *len = chan->backend.subbuf_size; return 0; } @@ -823,12 +826,13 @@ int ustctl_get_max_subbuf_size(struct lttng_ust_shm_handle *handle, int ustctl_get_mmap_read_offset(struct lttng_ust_shm_handle *handle, struct lttng_ust_lib_ring_buffer *buf, unsigned long *off) { - struct channel *chan = handle->shadow_chan; + struct channel *chan; unsigned long sb_bindex; if (!handle || !buf || !off) return -EINVAL; + chan = handle->shadow_chan; if (chan->backend.config.output != RING_BUFFER_MMAP) return -EINVAL; sb_bindex = subbuffer_id_get_index(&chan->backend.config, @@ -841,11 +845,12 @@ int ustctl_get_mmap_read_offset(struct lttng_ust_shm_handle *handle, int ustctl_get_subbuf_size(struct lttng_ust_shm_handle *handle, struct lttng_ust_lib_ring_buffer *buf, unsigned long *len) { - struct channel *chan = handle->shadow_chan; + struct channel *chan; if (!handle || !buf || !len) return -EINVAL; + chan = handle->shadow_chan; *len = lib_ring_buffer_get_read_data_size(&chan->backend.config, buf, handle); return 0; @@ -855,11 +860,12 @@ int ustctl_get_subbuf_size(struct lttng_ust_shm_handle *handle, int ustctl_get_padded_subbuf_size(struct lttng_ust_shm_handle *handle, struct lttng_ust_lib_ring_buffer *buf, unsigned long *len) { - struct channel *chan = handle->shadow_chan; + struct channel *chan; if (!handle || !buf || !len) return -EINVAL; + chan = handle->shadow_chan; *len = lib_ring_buffer_get_read_data_size(&chan->backend.config, buf, handle); *len = PAGE_ALIGN(*len); From francis.giraldeau at gmail.com Tue Jun 5 05:20:49 2012 From: francis.giraldeau at gmail.com (Francis Giraldeau) Date: Tue, 5 Jun 2012 11:20:49 +0200 Subject: [lttng-dev] Expose kernel tracer to user-space Message-ID: <1338888051-20879-1-git-send-email-francis.giraldeau@gmail.com> This series of patch follows the discussion about exposing kernel tracer to user-space. Here are few highlights: * Creates one lttng_uevent kernel event by writing to /proc/lttng * The event is a variable size byte sequence, parsed by default as UTF-8 text * Uses already existing /proc/lttng file instead of creating another * Refactor the tp_memcpy macros to support sequence and copy_from_user From francis.giraldeau at gmail.com Tue Jun 5 05:20:50 2012 From: francis.giraldeau at gmail.com (Francis Giraldeau) Date: Tue, 5 Jun 2012 11:20:50 +0200 Subject: [lttng-dev] [PATCH 1/2] Makes write operation a parameter for tp_memcpy macro In-Reply-To: <1338888051-20879-1-git-send-email-francis.giraldeau@gmail.com> References: <1338888051-20879-1-git-send-email-francis.giraldeau@gmail.com> Message-ID: <1338888051-20879-2-git-send-email-francis.giraldeau@gmail.com> Memcpy source can be either user-space or kernel-space. To avoid code duplication, this patch makes the operation a parameter to the macros. Available macros are thus: * tp_memcpy: kernel-space array copy * tp_memcpy_from_user: user-space array copy * tp_memcpy_dyn: kernel-space sequence copy * tp_memcpy_dyn_from_user: user-space sequence copy Those are TP_fast_assign macros that can be used with __dynamic_array macros in TP_STRUCT__entry in a TRACE_EVENT. Signed-off-by: Francis Giraldeau --- probes/lttng-events.h | 37 +++++++++++++++++++++++-------------- 1 file changed, 23 insertions(+), 14 deletions(-) diff --git a/probes/lttng-events.h b/probes/lttng-events.h index 05e17b9..492423a 100644 --- a/probes/lttng-events.h +++ b/probes/lttng-events.h @@ -523,17 +523,27 @@ __assign_##dest: \ } \ goto __end_field_##dest; -#undef tp_memcpy -#define tp_memcpy(dest, src, len) \ +/* fixed length array memcpy */ +#undef tp_memcpy_gen +#define tp_memcpy_gen(write_ops, dest, src, len) \ __assign_##dest: \ if (0) \ (void) __typemap.dest; \ lib_ring_buffer_align_ctx(&__ctx, lttng_alignof(__typemap.dest)); \ - __chan->ops->event_write(&__ctx, src, len); \ + __chan->ops->write_ops(&__ctx, src, len); \ goto __end_field_##dest; -#undef tp_memcpy_dyn -#define tp_memcpy_dyn(dest, src) \ +#undef tp_memcpy +#define tp_memcpy(dest, src, len) \ + tp_memcpy_gen(event_write, dest, src, len) + +#undef tp_memcpy_from_user +#define tp_memcpy_from_user(dest, src, len) \ + tp_memcpy_gen(event_write_from_user, dest, src, len) + +/* variable length sequence memcpy */ +#undef tp_memcpy_dyn_gen +#define tp_memcpy_dyn_gen(write_ops, dest, src) \ __assign_##dest##_1: \ { \ u32 __tmpl = __dynamic_len[__dynamic_len_idx]; \ @@ -543,18 +553,17 @@ __assign_##dest##_1: \ goto __end_field_##dest##_1; \ __assign_##dest##_2: \ lib_ring_buffer_align_ctx(&__ctx, lttng_alignof(__typemap.dest)); \ - __chan->ops->event_write(&__ctx, src, \ + __chan->ops->write_ops(&__ctx, src, \ sizeof(__typemap.dest) * __get_dynamic_array_len(dest));\ goto __end_field_##dest##_2; -#undef tp_memcpy_from_user -#define tp_memcpy_from_user(dest, src, len) \ - __assign_##dest: \ - if (0) \ - (void) __typemap.dest; \ - lib_ring_buffer_align_ctx(&__ctx, lttng_alignof(__typemap.dest)); \ - __chan->ops->event_write_from_user(&__ctx, src, len); \ - goto __end_field_##dest; +#undef tp_memcpy_dyn +#define tp_memcpy_dyn(dest, src) \ + tp_memcpy_dyn_gen(event_write, dest, src) + +#undef tp_memcpy_dyn_from_user +#define tp_memcpy_dyn_from_user(dest, src) \ + tp_memcpy_dyn_gen(event_write_from_user, dest, src) /* * The string length including the final \0. -- 1.7.9.5 From francis.giraldeau at gmail.com Tue Jun 5 05:20:51 2012 From: francis.giraldeau at gmail.com (Francis Giraldeau) Date: Tue, 5 Jun 2012 11:20:51 +0200 Subject: [lttng-dev] [PATCH 2/2] Expose kernel tracer to user-space In-Reply-To: <1338888051-20879-1-git-send-email-francis.giraldeau@gmail.com> References: <1338888051-20879-1-git-send-email-francis.giraldeau@gmail.com> Message-ID: <1338888051-20879-3-git-send-email-francis.giraldeau@gmail.com> By writing to the file /proc/lttng, a user-space application creates a kernel event. The event's payload is by default UTF-8 text, but any data can be written, up to 1024 bytes. Null-character is optional and is not enforced. The event uses sequence for space efficiency and to store any data as payload. Signed-off-by: Francis Giraldeau --- instrumentation/events/lttng-module/lttng.h | 21 +++++++++++++++++++++ lttng-abi.c | 20 ++++++++++++++++++++ lttng-abi.h | 1 + 3 files changed, 42 insertions(+) diff --git a/instrumentation/events/lttng-module/lttng.h b/instrumentation/events/lttng-module/lttng.h index 6f3d6d1..cc36a8b 100644 --- a/instrumentation/events/lttng-module/lttng.h +++ b/instrumentation/events/lttng-module/lttng.h @@ -28,6 +28,27 @@ TRACE_EVENT(lttng_metadata, TP_printk("") ) +TRACE_EVENT(lttng_uevent, + + TP_PROTO(const char *str, size_t len), + + TP_ARGS(str, len), + + /* + * Uses sequence to hold variable size data, by default considered + * as text. Null-terminal character is optional and is not enforced. + */ + TP_STRUCT__entry( + __dynamic_array_text(char, text, len) + ), + + TP_fast_assign( + tp_memcpy_dyn_from_user(text, str) + ), + + TP_printk("") +) + #endif /* _TRACE_LTTNG_H */ /* This part must be outside protection */ diff --git a/lttng-abi.c b/lttng-abi.c index 26a02ed..fa29e53 100644 --- a/lttng-abi.c +++ b/lttng-abi.c @@ -50,6 +50,10 @@ #include "lttng-events.h" #include "lttng-tracer.h" +/* include our own uevent tracepoint */ +#include "instrumentation/events/lttng-module/lttng.h" +DEFINE_TRACE(lttng_uevent); + /* * This is LTTng's own personal way to create a system call as an external * module. We use ioctl() on /proc/lttng. @@ -252,9 +256,25 @@ long lttng_ioctl(struct file *file, unsigned int cmd, unsigned long arg) } } +/* + * lttng_write_uevent - expose kernel tracer to user-space + */ + +static +ssize_t lttng_write_uevent(struct file *file, const char __user *user_buf, + size_t count, loff_t *fpos) +{ + if (count > LTTNG_UEVENT_SIZE) + count = LTTNG_UEVENT_SIZE; + + trace_lttng_uevent(user_buf, count); + return count; +} + static const struct file_operations lttng_fops = { .owner = THIS_MODULE, .unlocked_ioctl = lttng_ioctl, + .write = lttng_write_uevent, #ifdef CONFIG_COMPAT .compat_ioctl = lttng_ioctl, #endif diff --git a/lttng-abi.h b/lttng-abi.h index dc230d8..f99b037 100644 --- a/lttng-abi.h +++ b/lttng-abi.h @@ -26,6 +26,7 @@ #include #define LTTNG_KERNEL_SYM_NAME_LEN 256 +#define LTTNG_UEVENT_SIZE 1024 enum lttng_kernel_instrumentation { LTTNG_KERNEL_TRACEPOINT = 0, -- 1.7.9.5 From francis.giraldeau at gmail.com Tue Jun 5 05:27:14 2012 From: francis.giraldeau at gmail.com (Francis Giraldeau) Date: Tue, 05 Jun 2012 11:27:14 +0200 Subject: [lttng-dev] Displaying graphical results In-Reply-To: References: <524C960C5DFC794E82BE548D825F05CF283435F8@EU-MBX-01.mgc.mentorg.com> Message-ID: <4FCDD0F2.9020009@gmail.com> Le 2012-06-04 22:57, tchak adim a ?crit : > for Eclipse tool : > i already follow the different steps to integrate the cft, lttng2 > plugins as described in the link that u give it . > i don't konw how can i visualize the graphics of lttng kernel module > traces using eclipse , can u explain to me how ? You need to select the lttng perspective, then create an lttng project, import a trace trace and then it should show up. There are multiple views, the event view displays events in a table, and the control flow view shows process states according to time. Cheers, Francis -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4476 bytes Desc: Signature cryptographique S/MIME URL: From Bernd.Hufmann at ericsson.com Tue Jun 5 09:40:53 2012 From: Bernd.Hufmann at ericsson.com (Bernd Hufmann) Date: Tue, 5 Jun 2012 09:40:53 -0400 Subject: [lttng-dev] Displaying graphical results In-Reply-To: References: <524C960C5DFC794E82BE548D825F05CF283435F8@EU-MBX-01.mgc.mentorg.com> Message-ID: <4FCE0C65.7010906@ericsson.com> Hi We are currently finalizing the LTTng 2.0 Integration in Eclipse as part of the Eclipse Juno release. As I understood you are using the source code of Eclipse LTTng 2.0 integration. After starting the Eclipse application, switch to the perspective "LTTng Kernel". By default the following views will show: * Project Explorer This view is for handling Tracing projects, traces and experiments. To start viewing a trace, use the New Project wizard of Eclipse to create "Tracing Project". (Right mouse click in Project Explorer view -> New - Tracing Project). Then import a trace to the "Traces" sub-folder of you newly created project. (Right mouse click on folder "Traces"). You could also drag-&-drop a trace to the "Traces" sub-folder. Note that you need to select the the trace type during import (not drag&drop) or afterwards by using the context-sensitive menu on the trace (Right mouse-click on the trace -> Click on "Select Trace Type..." -> Common Trace Format -> Kernel Trace). After that you can open the trace. * Control This view is to create traces remotely using an SSH connection to a remote (or local) host. This if you want to create a session, enable/disable events or channels, start and stop tracing. You also can import create trace to your workspace * Events View This view contains a table displaying each event in a table form. From the top row you can search or filter events * Control Flow View This view shows the kernel control flow (process states over time). It uses internally a persistent state system which is create when you open the trace the first time. In this view you can zoom using cool bar buttons or mouse wheel. You can step from one event to the next/previous event per process using the relevant buttons. * Resources View This view shows you system resources such as CPUs, IRQs, Soft IRQs and their states over time. This view also uses the persistent state system I mentioned above. The buttons, zooming behaviour etc. are same as for the Control Flow View * Histogram View This view shows the event density (number of events over time). The big window shows the whole trace. The same one shows the current selected time range. * Statistics View This view shows you the total number events and number of events per event time of the whole trace * Bookmarks view Using the context-sensitive menu of the Events table you can bookmark events. In the bookmark view you can double-click on a bookmark and the event will be shown in the events table as well as in all other open LTTng views. All views are synchronized. That means when selecting an event one of the view, it will be shown also in the other views. Additionally, when changing the active time range (e.g. in the Histogram View), also the Control Flow View/Resources View will show the new time range. This was a brief introduction into what is coming up for the LTTng Juno. Keep in mind, that there might be still some bugs, so I apologizes for any glitches you may experience. These glitches will be fixed, as well as a comprehensive User Guide will be available for the Juno release. Best Regards Bernd On 06/05/2012 05:27 AM, Francis Giraldeau wrote: > Le 2012-06-04 22:57, tchak adim a ?crit : >> for Eclipse tool : >> i already follow the different steps to integrate the cft, lttng2 >> plugins as described in the link that u give it . >> i don't konw how can i visualize the graphics of lttng kernel module >> traces using eclipse , can u explain to me how ? > You need to select the lttng perspective, then create an lttng project, > import a trace trace and then it should show up. There are multiple > views, the event view displays events in a table, and the control flow > view shows process states according to time. > > Cheers, > > Francis > On 06/04/2012 04:57 PM, tchak adim wrote: > for Eclipse tool : > i already follow the different steps to integrate the cft, lttng2 > plugins as described in the link that u give it . > i don't konw how can i visualize the graphics of lttng kernel module > traces using eclipse , can u explain to me how ? > > For sourcery tool : > i didn't find the source code, i'm using an ordinary i386 machine > running ubuntu 10.04. > > Can u advice me and tell me what i can do ? > > thanks in advance ! -------------- next part -------------- An HTML attachment was scrubbed... URL: From mathieu.desnoyers at efficios.com Tue Jun 5 10:54:22 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 5 Jun 2012 10:54:22 -0400 Subject: [lttng-dev] BABELTRACE In-Reply-To: References: Message-ID: <20120605145422.GA21009@Krystal> * Vanni Genua (vannigenua at gmail.com) wrote: > Hi there, > so once I have correctly installed LTTng-modules, LTTng-ust and > LTTng-tools, I can use LTTng but not LTTng view. In order to use #lttng > view command I have to install Babeltrace, but it seems still to be a rc > version. As a matter of fact I downloaded it from > http://lttng.org/downloadand, according to README file, I launched > #./bootsrap but this file is not > in the directory: it doesn't exist! > I'm stuck here now, any idea? Just FYI, if you want to build babeltrace from its .tar.gz RC release package (tarball), you need to skip the "./bootstrap" step, which is only required if you use the git repository (IOW, master branch). Thanks, Mathieu > Vanni > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Tue Jun 5 11:14:59 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 5 Jun 2012 11:14:59 -0400 Subject: [lttng-dev] [PATCH 1/2] Makes write operation a parameter for tp_memcpy macro In-Reply-To: <1338888051-20879-2-git-send-email-francis.giraldeau@gmail.com> References: <1338888051-20879-1-git-send-email-francis.giraldeau@gmail.com> <1338888051-20879-2-git-send-email-francis.giraldeau@gmail.com> Message-ID: <20120605151459.GB21009@Krystal> * Francis Giraldeau (francis.giraldeau at gmail.com) wrote: > Memcpy source can be either user-space or kernel-space. To avoid code > duplication, this patch makes the operation a parameter to the macros. > Available macros are thus: > > * tp_memcpy: kernel-space array copy > * tp_memcpy_from_user: user-space array copy > * tp_memcpy_dyn: kernel-space sequence copy > * tp_memcpy_dyn_from_user: user-space sequence copy > > Those are TP_fast_assign macros that can be used with __dynamic_array > macros in TP_STRUCT__entry in a TRACE_EVENT. Merged, thanks ! Mathieu > > Signed-off-by: Francis Giraldeau > --- > probes/lttng-events.h | 37 +++++++++++++++++++++++-------------- > 1 file changed, 23 insertions(+), 14 deletions(-) > > diff --git a/probes/lttng-events.h b/probes/lttng-events.h > index 05e17b9..492423a 100644 > --- a/probes/lttng-events.h > +++ b/probes/lttng-events.h > @@ -523,17 +523,27 @@ __assign_##dest: \ > } \ > goto __end_field_##dest; > > -#undef tp_memcpy > -#define tp_memcpy(dest, src, len) \ > +/* fixed length array memcpy */ > +#undef tp_memcpy_gen > +#define tp_memcpy_gen(write_ops, dest, src, len) \ > __assign_##dest: \ > if (0) \ > (void) __typemap.dest; \ > lib_ring_buffer_align_ctx(&__ctx, lttng_alignof(__typemap.dest)); \ > - __chan->ops->event_write(&__ctx, src, len); \ > + __chan->ops->write_ops(&__ctx, src, len); \ > goto __end_field_##dest; > > -#undef tp_memcpy_dyn > -#define tp_memcpy_dyn(dest, src) \ > +#undef tp_memcpy > +#define tp_memcpy(dest, src, len) \ > + tp_memcpy_gen(event_write, dest, src, len) > + > +#undef tp_memcpy_from_user > +#define tp_memcpy_from_user(dest, src, len) \ > + tp_memcpy_gen(event_write_from_user, dest, src, len) > + > +/* variable length sequence memcpy */ > +#undef tp_memcpy_dyn_gen > +#define tp_memcpy_dyn_gen(write_ops, dest, src) \ > __assign_##dest##_1: \ > { \ > u32 __tmpl = __dynamic_len[__dynamic_len_idx]; \ > @@ -543,18 +553,17 @@ __assign_##dest##_1: \ > goto __end_field_##dest##_1; \ > __assign_##dest##_2: \ > lib_ring_buffer_align_ctx(&__ctx, lttng_alignof(__typemap.dest)); \ > - __chan->ops->event_write(&__ctx, src, \ > + __chan->ops->write_ops(&__ctx, src, \ > sizeof(__typemap.dest) * __get_dynamic_array_len(dest));\ > goto __end_field_##dest##_2; > > -#undef tp_memcpy_from_user > -#define tp_memcpy_from_user(dest, src, len) \ > - __assign_##dest: \ > - if (0) \ > - (void) __typemap.dest; \ > - lib_ring_buffer_align_ctx(&__ctx, lttng_alignof(__typemap.dest)); \ > - __chan->ops->event_write_from_user(&__ctx, src, len); \ > - goto __end_field_##dest; > +#undef tp_memcpy_dyn > +#define tp_memcpy_dyn(dest, src) \ > + tp_memcpy_dyn_gen(event_write, dest, src) > + > +#undef tp_memcpy_dyn_from_user > +#define tp_memcpy_dyn_from_user(dest, src) \ > + tp_memcpy_dyn_gen(event_write_from_user, dest, src) > > /* > * The string length including the final \0. > -- > 1.7.9.5 > > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Tue Jun 5 11:16:55 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 5 Jun 2012 11:16:55 -0400 Subject: [lttng-dev] [PATCH 2/2] Expose kernel tracer to user-space In-Reply-To: <1338888051-20879-3-git-send-email-francis.giraldeau@gmail.com> References: <1338888051-20879-1-git-send-email-francis.giraldeau@gmail.com> <1338888051-20879-3-git-send-email-francis.giraldeau@gmail.com> Message-ID: <20120605151655.GA21325@Krystal> * Francis Giraldeau (francis.giraldeau at gmail.com) wrote: > By writing to the file /proc/lttng, a user-space application creates a > kernel event. The event's payload is by default UTF-8 text, but any data > can be written, up to 1024 bytes. Null-character is optional and is not > enforced. The event uses sequence for space efficiency and to store any > data as payload. As discussed on IRC, please move the function to a separate module in /probes/, and add a register function into lttng-abi.c that allows that module to register its callback. Thanks, Mathieu > > Signed-off-by: Francis Giraldeau > --- > instrumentation/events/lttng-module/lttng.h | 21 +++++++++++++++++++++ > lttng-abi.c | 20 ++++++++++++++++++++ > lttng-abi.h | 1 + > 3 files changed, 42 insertions(+) > > diff --git a/instrumentation/events/lttng-module/lttng.h b/instrumentation/events/lttng-module/lttng.h > index 6f3d6d1..cc36a8b 100644 > --- a/instrumentation/events/lttng-module/lttng.h > +++ b/instrumentation/events/lttng-module/lttng.h > @@ -28,6 +28,27 @@ TRACE_EVENT(lttng_metadata, > TP_printk("") > ) > > +TRACE_EVENT(lttng_uevent, > + > + TP_PROTO(const char *str, size_t len), > + > + TP_ARGS(str, len), > + > + /* > + * Uses sequence to hold variable size data, by default considered > + * as text. Null-terminal character is optional and is not enforced. > + */ > + TP_STRUCT__entry( > + __dynamic_array_text(char, text, len) > + ), > + > + TP_fast_assign( > + tp_memcpy_dyn_from_user(text, str) > + ), > + > + TP_printk("") > +) > + > #endif /* _TRACE_LTTNG_H */ > > /* This part must be outside protection */ > diff --git a/lttng-abi.c b/lttng-abi.c > index 26a02ed..fa29e53 100644 > --- a/lttng-abi.c > +++ b/lttng-abi.c > @@ -50,6 +50,10 @@ > #include "lttng-events.h" > #include "lttng-tracer.h" > > +/* include our own uevent tracepoint */ > +#include "instrumentation/events/lttng-module/lttng.h" > +DEFINE_TRACE(lttng_uevent); > + > /* > * This is LTTng's own personal way to create a system call as an external > * module. We use ioctl() on /proc/lttng. > @@ -252,9 +256,25 @@ long lttng_ioctl(struct file *file, unsigned int cmd, unsigned long arg) > } > } > > +/* > + * lttng_write_uevent - expose kernel tracer to user-space > + */ > + > +static > +ssize_t lttng_write_uevent(struct file *file, const char __user *user_buf, > + size_t count, loff_t *fpos) > +{ > + if (count > LTTNG_UEVENT_SIZE) > + count = LTTNG_UEVENT_SIZE; > + > + trace_lttng_uevent(user_buf, count); > + return count; > +} > + > static const struct file_operations lttng_fops = { > .owner = THIS_MODULE, > .unlocked_ioctl = lttng_ioctl, > + .write = lttng_write_uevent, > #ifdef CONFIG_COMPAT > .compat_ioctl = lttng_ioctl, > #endif > diff --git a/lttng-abi.h b/lttng-abi.h > index dc230d8..f99b037 100644 > --- a/lttng-abi.h > +++ b/lttng-abi.h > @@ -26,6 +26,7 @@ > #include > > #define LTTNG_KERNEL_SYM_NAME_LEN 256 > +#define LTTNG_UEVENT_SIZE 1024 > > enum lttng_kernel_instrumentation { > LTTNG_KERNEL_TRACEPOINT = 0, > -- > 1.7.9.5 > > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Tue Jun 5 11:19:05 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 5 Jun 2012 11:19:05 -0400 Subject: [lttng-dev] UST check pointer/de-reference order In-Reply-To: <524C960C5DFC794E82BE548D825F05CF28344978@EU-MBX-01.mgc.mentorg.com> References: <524C960C5DFC794E82BE548D825F05CF283435F8@EU-MBX-01.mgc.mentorg.com> <524C960C5DFC794E82BE548D825F05CF28344927@EU-MBX-01.mgc.mentorg.com> <524C960C5DFC794E82BE548D825F05CF28344978@EU-MBX-01.mgc.mentorg.com> Message-ID: <20120605151905.GB21325@Krystal> * Oestman, Fredrik (Fredrik_Oestman at mentor.com) wrote: > I stumbled across some code where pointers are de-referenced and then checked for NULL. > > Cheers, merged into master and stable-2.0, thanks! Mathieu > > Fredrik ?stman > > > diff --git a/liblttng-ust-ctl/ustctl.c b/liblttng-ust-ctl/ustctl.c > index 9789413..80aed04 100644 > --- a/liblttng-ust-ctl/ustctl.c > +++ b/liblttng-ust-ctl/ustctl.c > @@ -732,7 +732,7 @@ void ustctl_unmap_channel(struct lttng_ust_shm_handle *handle) > struct lttng_ust_lib_ring_buffer *ustctl_open_stream_read(struct lttng_ust_shm_handle *handle, > int cpu) > { > - struct channel *chan = handle->shadow_chan; > + struct channel *chan; > int *shm_fd, *wait_fd; > uint64_t *memory_map_size; > struct lttng_ust_lib_ring_buffer *buf; > @@ -741,6 +741,7 @@ struct lttng_ust_lib_ring_buffer *ustctl_open_stream_read(struct lttng_ust_shm_h > if (!handle) > return NULL; > > + chan = handle->shadow_chan; > buf = channel_get_ring_buffer(&chan->backend.config, > chan, cpu, handle, &shm_fd, &wait_fd, &memory_map_size); > if (!buf) > @@ -784,11 +785,12 @@ int ustctl_get_mmap_len(struct lttng_ust_shm_handle *handle, > unsigned long *len) > { > unsigned long mmap_buf_len; > - struct channel *chan = handle->shadow_chan; > + struct channel *chan; > > if (!handle || !buf || !len) > return -EINVAL; > > + chan = handle->shadow_chan; > if (chan->backend.config.output != RING_BUFFER_MMAP) > return -EINVAL; > mmap_buf_len = chan->backend.buf_size; > @@ -805,11 +807,12 @@ int ustctl_get_max_subbuf_size(struct lttng_ust_shm_handle *handle, > struct lttng_ust_lib_ring_buffer *buf, > unsigned long *len) > { > - struct channel *chan = handle->shadow_chan; > + struct channel *chan; > > if (!handle || !buf || !len) > return -EINVAL; > > + chan = handle->shadow_chan; > *len = chan->backend.subbuf_size; > return 0; > } > @@ -823,12 +826,13 @@ int ustctl_get_max_subbuf_size(struct lttng_ust_shm_handle *handle, > int ustctl_get_mmap_read_offset(struct lttng_ust_shm_handle *handle, > struct lttng_ust_lib_ring_buffer *buf, unsigned long *off) > { > - struct channel *chan = handle->shadow_chan; > + struct channel *chan; > unsigned long sb_bindex; > > if (!handle || !buf || !off) > return -EINVAL; > > + chan = handle->shadow_chan; > if (chan->backend.config.output != RING_BUFFER_MMAP) > return -EINVAL; > sb_bindex = subbuffer_id_get_index(&chan->backend.config, > @@ -841,11 +845,12 @@ int ustctl_get_mmap_read_offset(struct lttng_ust_shm_handle *handle, > int ustctl_get_subbuf_size(struct lttng_ust_shm_handle *handle, > struct lttng_ust_lib_ring_buffer *buf, unsigned long *len) > { > - struct channel *chan = handle->shadow_chan; > + struct channel *chan; > > if (!handle || !buf || !len) > return -EINVAL; > > + chan = handle->shadow_chan; > *len = lib_ring_buffer_get_read_data_size(&chan->backend.config, buf, > handle); > return 0; > @@ -855,11 +860,12 @@ int ustctl_get_subbuf_size(struct lttng_ust_shm_handle *handle, > int ustctl_get_padded_subbuf_size(struct lttng_ust_shm_handle *handle, > struct lttng_ust_lib_ring_buffer *buf, unsigned long *len) > { > - struct channel *chan = handle->shadow_chan; > + struct channel *chan; > > if (!handle || !buf || !len) > return -EINVAL; > > + chan = handle->shadow_chan; > *len = lib_ring_buffer_get_read_data_size(&chan->backend.config, buf, > handle); > *len = PAGE_ALIGN(*len); > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Tue Jun 5 11:49:50 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 5 Jun 2012 11:49:50 -0400 Subject: [lttng-dev] using --function/--probe In-Reply-To: <3551.1338647739@cloud.rain.com> References: <3551.1338647739@cloud.rain.com> Message-ID: <20120605154950.GA21955@Krystal> * Bill Trost (trost at cloud.rain.com) wrote: > Hi, > > I'm trying to investigate a problem under Linux 2.6.39.4 involving > kworker threads and am pretty much stymied. I've tried enabling > various events using --function and --probe and it's been > very hit or miss. Very often, "lttng enable-event" comes back > with nothing more specific than "bad ioctl", but every once > in a while I'll find a symbol I can add a probe point for. > > So: What does "--probe" mean? --probe allows to put a kprobe (breakpoint) at a kernel text address or symbol. > What does "--function" mean, --function allows to put a kretprobes at a function entry site (using its symbol name), which will trigger events at function entry return. > and how does > it differ from "--probe"? How do I determine what symbols are valid for > each of these options? It entirely depends on which functions are blacklisted in the kernel (this is an attribute added to the functions specifically for kprobes). The keyword is "__kprobes". > And, maybe as a worked example, how do I > trace what work is being enqueued and run by the kworker threads? For this level of details, I think kprobes/kretprobes will not currently allow you to fetch it. The two options we have are: - use static tracepoints. Is there a tracepoint that targets the information you are looking for ? Try "lttng list -k". - or extend the dynamic probing to allow fetching variables from debug information, Thanks, Mathieu > > Thanks, > Bill > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From montarcyber at gmail.com Tue Jun 5 12:56:55 2012 From: montarcyber at gmail.com (tchak adim) Date: Tue, 5 Jun 2012 12:56:55 -0400 Subject: [lttng-dev] Displaying graphical results In-Reply-To: <4FCE0C65.7010906@ericsson.com> References: <524C960C5DFC794E82BE548D825F05CF283435F8@EU-MBX-01.mgc.mentorg.com> <4FCE0C65.7010906@ericsson.com> Message-ID: Thanks for all for your detailed responses . Regards. -------------- next part -------------- An HTML attachment was scrubbed... URL: From yannick.brosseau at gmail.com Tue Jun 5 13:02:00 2012 From: yannick.brosseau at gmail.com (Brosseau, Yannick) Date: Tue, 5 Jun 2012 13:02:00 -0400 Subject: [lttng-dev] Displaying graphical results In-Reply-To: <4FC3CBF6.1060608@polymtl.ca> References: <4FC3CBF6.1060608@polymtl.ca> Message-ID: > > Support in LTTV (the other, more lightweight viewer) should be just > around the corner. Just to add that we currently have the textDump working in LTTV and we've just got the main GUI window working. We should be able to get the views really soon now. Yannick From montarcyber at gmail.com Tue Jun 5 13:12:43 2012 From: montarcyber at gmail.com (tchak adim) Date: Tue, 5 Jun 2012 13:12:43 -0400 Subject: [lttng-dev] Displaying graphical results In-Reply-To: References: <4FC3CBF6.1060608@polymtl.ca> Message-ID: Good news .. That make things much better :) Thanks for the information. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mathieu.desnoyers at efficios.com Tue Jun 5 13:14:18 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 5 Jun 2012 13:14:18 -0400 Subject: [lttng-dev] UST "Tracepoint signature mismatch" with C99 bool. In-Reply-To: References: Message-ID: <20120605171418.GA22608@Krystal> * John Steele Scott (toojays at toojays.net) wrote: > I originally sent this on the 22nd of May, but it looks like it got stuck in moderation since I'm not subscribed. Reposting via gmane.comp.sysutils.lttng.devel . . . > > I'm just getting started wiring my up to lttng-ust, and unfortunately > ran into an issue with the first probe I tried, which used a C99 bool > argument. Fixed by commit: commit 9eb061825ad8ebdf8c22d812eff0232d97d72cd2 Author: Mathieu Desnoyers Date: Tue Jun 5 13:08:54 2012 -0400 Fix: perform macro expansion on tracepoint signatures The problem can be seen by patching the demo test from lttng-ust as follows: --- a/tests/demo/ust_tests_demo3.h +++ b/tests/demo/ust_tests_demo3.h @@ -22,12 +22,14 @@ extern "C" { * all copies or substantial portions of the Software. */ +#include + #include TRACEPOINT_EVENT(ust_tests_demo3, done, - TP_ARGS(int, value), + TP_ARGS(bool, value), TP_FIELDS( - ctf_integer(int, value, value) + ctf_integer(bool, value, value) ) ) Then when the demo is run with LTTNG_UST_DEBUG=1, a warning is shown, like: liblttng_ust_tracepoint[3315/3315]: Warning: Tracepoint signature mismatch, enabling one or more tracepoints. Ensure that the tracepoint probes prototyp match the application. (in set_tracepoint() at tracepoint.c:310) liblttng_ust_tracepoint[3315/3315]: Warning: Tracepoint "ust_tests_demo3:don signatures: call: "_Bool, value" vs probe: "bool, value". (in set_tracepoint at tracepoint.c:312) It seems that TP_ARGS does not perform preprocessor expansion on the "bool" type spec, while something underneath TP_FIELDS does. And since (at least on this Centos 6.2 box) stdbool.h uses a #define rather than a typedef to make bool equivalent to _Bool, liblttng detects a mismatch. Reported-by: John Steele Scott Signed-off-by: Mathieu Desnoyers And tested by commit f121009f677db0f9444e0683e24315eed9c5973d (master branch commits). I'll immediately merge these into stable-2.0. Thanks! Mathieu > > The problem can be seen by patching the demo test from lttng-ust as > follows: > > --- a/tests/demo/ust_tests_demo3.h > +++ b/tests/demo/ust_tests_demo3.h > @@ -22,12 +22,14 @@ extern "C" { > * all copies or substantial portions of the Software. > */ > > +#include > + > #include > > TRACEPOINT_EVENT(ust_tests_demo3, done, > - TP_ARGS(int, value), > + TP_ARGS(bool, value), > TP_FIELDS( > - ctf_integer(int, value, value) > + ctf_integer(bool, value, value) > ) > ) > > Then when the demo is run with LTTNG_UST_DEBUG=1, a warning is shown, > like: > > liblttng_ust_tracepoint[3315/3315]: Warning: Tracepoint signature mismatch, not enabling one or more tracepoints. Ensure that the tracepoint probes prototypes match the application. (in set_tracepoint() at tracepoint.c:310) > liblttng_ust_tracepoint[3315/3315]: Warning: Tracepoint "ust_tests_demo3:done" signatures: call: "_Bool, value" vs probe: "bool, value". (in set_tracepoint() at tracepoint.c:312) > > It seems that TP_ARGS does not perform preprocessor expansion on the > "bool" type spec, while something underneath TP_FIELDS does. And since > (at least on this Centos 6.2 box) stdbool.h uses a #define rather than a > typedef to make bool equivalent to _Bool, liblttng detects a mismatch. > > So in my tracepoint specifications, I need to remember to use the less > attractive _Bool instead of bool. > > Perhaps it's worth having a special case for this where the trace/probe > signatures are compared? On the other hand, I guess nobody complained > until now? > > cheers, > > John > > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From francis.giraldeau at gmail.com Tue Jun 5 19:24:14 2012 From: francis.giraldeau at gmail.com (Francis Giraldeau) Date: Wed, 6 Jun 2012 01:24:14 +0200 Subject: [lttng-dev] [PATCH] Expose kernel tracer to user-space Message-ID: <1338938654-16515-1-git-send-email-francis.giraldeau@gmail.com> By writing to the file /proc/lttng, a user-space application creates a kernel event. The event's payload is by default UTF-8 text, but any data can be written, up to 1024 bytes. Null-character is optional and is not enforced. The event uses sequence for space efficiency and to store any data as payload. Update: split the probe code into it's own module and make it an optional feature of lttng-abi. The feature is enabled when the module is loaded. The lttng-abi module exports a register function and includes a wrapper for lttng_fops write. This is required since struct file_operations must be const. Since the module dependency is reversed, unloading the lttng-uevent module is done only when it's not used anymore. This is done with rwlock synchronisation. The synchronisation doesn't prevent starvation, but this situation is unlikely and can be prevented by stop active tracing sessions. Signed-off-by: Francis Giraldeau --- instrumentation/events/lttng-module/uevent.h | 33 +++++++++++++++ lttng-abi.c | 42 +++++++++++++++++++ lttng-abi.h | 3 ++ probes/Makefile | 2 + probes/lttng-probe-uevent.c | 36 ++++++++++++++++ probes/lttng-uevent.c | 58 ++++++++++++++++++++++++++ 6 files changed, 174 insertions(+) create mode 100644 instrumentation/events/lttng-module/uevent.h create mode 100644 probes/lttng-probe-uevent.c create mode 100644 probes/lttng-uevent.c diff --git a/instrumentation/events/lttng-module/uevent.h b/instrumentation/events/lttng-module/uevent.h new file mode 100644 index 0000000..f67d901 --- /dev/null +++ b/instrumentation/events/lttng-module/uevent.h @@ -0,0 +1,33 @@ +#undef TRACE_SYSTEM +#define TRACE_SYSTEM uevent + +#if !defined(UEVENT_H_) || defined(TRACE_HEADER_MULTI_READ) +#define UEVENT_H_ + +#include + +TRACE_EVENT(lttng_uevent, + + TP_PROTO(const char *str, size_t len), + + TP_ARGS(str, len), + + /* + * Uses sequence to hold variable size data, by default considered + * as text. Null-terminal character is optional and is not enforced. + */ + TP_STRUCT__entry( + __dynamic_array_text(char, text, len) + ), + + TP_fast_assign( + tp_memcpy_dyn_from_user(text, str) + ), + + TP_printk("") +) + +#endif /* UEVENT_H_ */ + +/* This part must be outside protection */ +#include "../../../probes/define_trace.h" diff --git a/lttng-abi.c b/lttng-abi.c index 26a02ed..b8e6b57 100644 --- a/lttng-abi.c +++ b/lttng-abi.c @@ -51,6 +51,12 @@ #include "lttng-tracer.h" /* + * Required data structures to support lttng-probe-uevent + */ +DEFINE_RWLOCK(uevent_rwlock); +write_ops_t lttng_uevent_handler; + +/* * This is LTTng's own personal way to create a system call as an external * module. We use ioctl() on /proc/lttng. */ @@ -252,9 +258,45 @@ long lttng_ioctl(struct file *file, unsigned int cmd, unsigned long arg) } } +/* + * lttng_uevent_set_handler - set handler functions for uevent + * + * Access to handler code is protected with rwlock in order to + * prevent the optional module to be removed while in use. + */ + +void lttng_uevent_set_handler(write_ops_t handler) +{ + write_lock(&uevent_rwlock); + lttng_uevent_handler = handler; + write_unlock(&uevent_rwlock); +} +EXPORT_SYMBOL_GPL(lttng_uevent_set_handler); + +/* + * lttng_write_uevent - expose kernel tracer to user-space + */ + +static +ssize_t lttng_write_uevent(struct file *file, const char __user *ubuf, + size_t count, loff_t *fpos) +{ + int ret; + + read_lock(&uevent_rwlock); + if (unlikely(lttng_uevent_handler == NULL)) { + read_unlock(&uevent_rwlock); + return -ENOSYS; + } + ret = (*lttng_uevent_handler)(file, ubuf, count, fpos); + read_unlock(&uevent_rwlock); + return ret; +} + static const struct file_operations lttng_fops = { .owner = THIS_MODULE, .unlocked_ioctl = lttng_ioctl, + .write = lttng_write_uevent, #ifdef CONFIG_COMPAT .compat_ioctl = lttng_ioctl, #endif diff --git a/lttng-abi.h b/lttng-abi.h index dc230d8..f4a8c0c 100644 --- a/lttng-abi.h +++ b/lttng-abi.h @@ -27,6 +27,9 @@ #define LTTNG_KERNEL_SYM_NAME_LEN 256 +typedef ssize_t (*write_ops_t) (struct file *, const char __user *, size_t, loff_t *); +void lttng_uevent_set_handler(write_ops_t handler); + enum lttng_kernel_instrumentation { LTTNG_KERNEL_TRACEPOINT = 0, LTTNG_KERNEL_KPROBE = 1, diff --git a/probes/Makefile b/probes/Makefile index 698a9c9..a895e60 100644 --- a/probes/Makefile +++ b/probes/Makefile @@ -14,6 +14,8 @@ obj-m += lttng-probe-sched.o obj-m += lttng-probe-irq.o obj-m += lttng-probe-signal.o obj-m += lttng-probe-timer.o +obj-m += lttng-probe-uevent.o +obj-m += lttng-uevent.o obj-m += lttng-probe-statedump.o diff --git a/probes/lttng-probe-uevent.c b/probes/lttng-probe-uevent.c new file mode 100644 index 0000000..90abb5e --- /dev/null +++ b/probes/lttng-probe-uevent.c @@ -0,0 +1,36 @@ +/* + * probes/lttng-probe-uevent.c + * + * Expose kernel tracer to user-space through /proc/lttng + * + * Copyright (C) 2009-2012 Mathieu Desnoyers + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; only + * version 2.1 of the License. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include + +/* + * Create lttng_uevent tracepoint probes. + */ +#define LTTNG_PACKAGE_BUILD +#define CREATE_TRACE_POINTS +#define TRACE_INCLUDE_PATH ../instrumentation/events/lttng-module + +#include "../instrumentation/events/lttng-module/uevent.h" + +MODULE_LICENSE("GPL and additional rights"); +MODULE_AUTHOR("Mathieu Desnoyers "); +MODULE_DESCRIPTION("LTTng uevent probes"); diff --git a/probes/lttng-uevent.c b/probes/lttng-uevent.c new file mode 100644 index 0000000..7b4bffc --- /dev/null +++ b/probes/lttng-uevent.c @@ -0,0 +1,58 @@ +/* + * probes/lttng-uevent.c + * + * Expose kernel tracer to user-space through /proc/lttng + * + * Copyright (C) 2012 Mathieu Desnoyers + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; only + * version 2.1 of the License. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include "../lttng-abi.h" + +/* include our own uevent tracepoint */ +#include "../instrumentation/events/lttng-module/uevent.h" +DEFINE_TRACE(lttng_uevent); + +#define LTTNG_UEVENT_SIZE 1024 + +ssize_t uevent_write_handler(struct file *file, const char __user *ubuf, + size_t count, loff_t *fpos) +{ + if (count > LTTNG_UEVENT_SIZE) + count = LTTNG_UEVENT_SIZE; + + trace_lttng_uevent(ubuf, count); + return count; +} + +static int __init lttng_probe_uevent_init(void) +{ + lttng_uevent_set_handler(uevent_write_handler); + return 0; +} + +static void __exit lttng_probe_uevent_exit(void) +{ + lttng_uevent_set_handler(NULL); +} + +module_init(lttng_probe_uevent_init); +module_exit(lttng_probe_uevent_exit); + +MODULE_LICENSE("GPL and additional rights"); +MODULE_AUTHOR("Mathieu Desnoyers "); +MODULE_DESCRIPTION("LTTng kernel event from user-space"); -- 1.7.9.5 From francis.giraldeau at gmail.com Tue Jun 5 19:40:33 2012 From: francis.giraldeau at gmail.com (Francis Giraldeau) Date: Wed, 06 Jun 2012 01:40:33 +0200 Subject: [lttng-dev] [PATCH] Expose kernel tracer to user-space In-Reply-To: <1338938654-16515-1-git-send-email-francis.giraldeau@gmail.com> References: <1338938654-16515-1-git-send-email-francis.giraldeau@gmail.com> Message-ID: <4FCE98F1.4020803@gmail.com> Le 2012-06-06 01:24, Francis Giraldeau a ?crit : > By writing to the file /proc/lttng, a user-space application creates a > kernel event. The event's payload is by default UTF-8 text, but any data > can be written, up to 1024 bytes. Null-character is optional and is not > enforced. The event uses sequence for space efficiency and to store any > data as payload. > > Update: split the probe code into it's own module and make it an optional > feature of lttng-abi. The feature is enabled when the module is loaded. The > lttng-abi module exports a register function and includes a wrapper for > lttng_fops write. This is required since struct file_operations must be const. > Since the module dependency is reversed, unloading the lttng-uevent module is > done only when it's not used anymore. This is done with rwlock synchronisation. > The synchronisation doesn't prevent starvation, but this situation is unlikely > and can be prevented by stop active tracing sessions. One little gotcha: after modprobe lttng-uevent, the module lttng-probe-uevent must be loaded manually, otherwise lttng_uevent is not listed with "lttng list -k". Is there a way to make the probe load automatically when lttng-uevent is loaded? Cheers, Francis -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4476 bytes Desc: Signature cryptographique S/MIME URL: From francis.giraldeau at gmail.com Wed Jun 6 07:08:23 2012 From: francis.giraldeau at gmail.com (Francis Giraldeau) Date: Wed, 06 Jun 2012 13:08:23 +0200 Subject: [lttng-dev] workload-kit: generating a standard trace set Message-ID: <4FCF3A27.60901@gmail.com> Hi, To validate trace analysis algorithms, some traces with known features are required. Then, a unit test can be made to check if the algorithm works well. For example, for the algorithm that recovers the CPU usage, a trace with an identifiable process that computes for a known amount of time is needed. The problem is that everyone has it's own set of traces, often made in ad-hoc fashion. In the past, such trace have been added to the source control system, but because of the large size and the binary nature of the files, it's not appropriate to do so. How can we share this trace set? How can we regenerate it on-demand? Here comes workload-kit! This project aims to be a collection of scripts and utilities to generate a comprehensive and standard trace set. Instead of storing traces in GIT, let's store the scripts and custom programs to regenerate them on-demand and automatically. We can thus generate the trace set on a daily basis for example. For those that are interested in this project, here are the URLs. Code: https://github.com/giraldeau/workload-kit Releases: http://secretaire.dorsal.polymtl.ca/~fgiraldeau/workload-kit/ Ubuntu package: https://launchpad.net/~francis-giraldeau/+archive/ppa It's pretty much the first announcement, so comments and suggestions are welcome. Contributions to expand trace use-cases that you need for your own testing are also welcome. See README for the more information. Cheers, Francis -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4476 bytes Desc: Signature cryptographique S/MIME URL: From trost at cloud.rain.com Wed Jun 6 10:48:17 2012 From: trost at cloud.rain.com (Bill Trost) Date: Wed, 06 Jun 2012 07:48:17 -0700 Subject: [lttng-dev] using --function/--probe In-Reply-To: Your message of Tue, 05 Jun 2012 11:49:50 EDT. <20120605154950.GA21955@Krystal> References: <20120605154950.GA21955@Krystal><3551.1338647739@cloud.rain.com> Message-ID: <19046.1338994097@cloud.rain.com> Mathieu wrote: * Bill Trost (trost at cloud.rain.com) wrote: > and how does it differ from "--probe"? How do I determine > what symbols are valid for each of these options? It entirely depends on which functions are blacklisted in the kernel (this is an attribute added to the functions specifically for kprobes). The keyword is "__kprobes". Is that the only basis? For example, I can add a dynamic tracepoint (of either flawor) for queue_work_on(), but not work __queue_work(), yet neither of those functions have any apparent annotation. Or does EXPORT_SYMBOL_GPL imply a probe point? > ...how do I trace what work is being > enqueued and run by the kworker threads? For this level of details, I think kprobes/kretprobes will not currently allow you to fetch it. The two options we have are: - use static tracepoints. Is there a tracepoint that targets the information you are looking for ? Try "lttng list -k". No, I've been capturing all kernel events and haven't seen anything resembling the kworker tracepoints of lttng 0.x. - or extend the dynamic probing to allow fetching variables from debug information, Hmm. Well, I did a bit more poking and got a bit closer -- at least now I know what dynamic points to use. (eg, "queue_work_on" and friends to see what is being queued). I take it that LTT can't capture function call arguments as a form of additional context? That would be an ideal bit of information in this case. Thanks again, Bill From serres at live.ca Wed Jun 6 14:54:30 2012 From: serres at live.ca (Danny Serres) Date: Wed, 6 Jun 2012 14:54:30 -0400 Subject: [lttng-dev] [lttng-tools PATCH] Move metset in channel_set_attr after the null check Message-ID: <1339008870-4891-1-git-send-email-serres@live.ca> Signed-off-by: Danny Serres --- src/lib/lttng-ctl/lttng-ctl.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/lib/lttng-ctl/lttng-ctl.c b/src/lib/lttng-ctl/lttng-ctl.c index 36069ef..356fb34 100644 --- a/src/lib/lttng-ctl/lttng-ctl.c +++ b/src/lib/lttng-ctl/lttng-ctl.c @@ -885,13 +885,13 @@ int lttng_calibrate(struct lttng_handle *handle, void lttng_channel_set_default_attr(struct lttng_domain *domain, struct lttng_channel_attr *attr) { - memset(attr, 0, sizeof(struct lttng_channel_attr)); - /* Safety check */ if (attr == NULL || domain == NULL) { return; } + memset(attr, 0, sizeof(struct lttng_channel_attr)); + switch (domain->type) { case LTTNG_DOMAIN_KERNEL: attr->overwrite = DEFAULT_CHANNEL_OVERWRITE; -- 1.7.9.5 From mathieu.desnoyers at efficios.com Wed Jun 6 15:40:09 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Wed, 6 Jun 2012 15:40:09 -0400 Subject: [lttng-dev] [lttng-tools PATCH] Move metset in channel_set_attr after the null check In-Reply-To: <1339008870-4891-1-git-send-email-serres@live.ca> References: <1339008870-4891-1-git-send-email-serres@live.ca> Message-ID: <20120606194009.GA5882@Krystal> * Danny Serres (serres at live.ca) wrote: > Signed-off-by: Danny Serres Hi Danny, Your patch looks good ! Just one detail: I forgot to create a efficios.com email for you. So I will create "danny.serres at efficios.com", and you can sign-off your patch with this email. It will be forwarded to your @live.ca email. Thanks ! Mathieu > --- > src/lib/lttng-ctl/lttng-ctl.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/src/lib/lttng-ctl/lttng-ctl.c b/src/lib/lttng-ctl/lttng-ctl.c > index 36069ef..356fb34 100644 > --- a/src/lib/lttng-ctl/lttng-ctl.c > +++ b/src/lib/lttng-ctl/lttng-ctl.c > @@ -885,13 +885,13 @@ int lttng_calibrate(struct lttng_handle *handle, > void lttng_channel_set_default_attr(struct lttng_domain *domain, > struct lttng_channel_attr *attr) > { > - memset(attr, 0, sizeof(struct lttng_channel_attr)); > - > /* Safety check */ > if (attr == NULL || domain == NULL) { > return; > } > > + memset(attr, 0, sizeof(struct lttng_channel_attr)); > + > switch (domain->type) { > case LTTNG_DOMAIN_KERNEL: > attr->overwrite = DEFAULT_CHANNEL_OVERWRITE; > -- > 1.7.9.5 > > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From dgoulet at efficios.com Thu Jun 7 13:05:36 2012 From: dgoulet at efficios.com (David Goulet) Date: Thu, 07 Jun 2012 13:05:36 -0400 Subject: [lttng-dev] [lttng-tools PATCH] Move metset in channel_set_attr after the null check In-Reply-To: <1339008870-4891-1-git-send-email-serres@live.ca> References: <1339008870-4891-1-git-send-email-serres@live.ca> Message-ID: <4FD0DF60.2080709@efficios.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Danny, I've merged your patch. Thanks! It's now upstream and in stable-2.0 :). Just a note for the next time. This is a "Fix" patch so in the commit message, start with "Fix:". I've changed it for you for this commit. Only commit with Fix: are picked for the stable-2.0 branch. commit eacaa7a62c3f2b9a019fd145d69c977fc53144bb Author: Danny Serres Date: Wed Jun 6 14:54:30 2012 -0400 Fix: move memset in channel_set_attr after NULL check Signed-off-by: Danny Serres Signed-off-by: David Goulet Cheers! David On 06/06/12 02:54 PM, Danny Serres wrote: > Signed-off-by: Danny Serres --- > src/lib/lttng-ctl/lttng-ctl.c | 4 ++-- 1 file changed, 2 insertions(+), > 2 deletions(-) > > diff --git a/src/lib/lttng-ctl/lttng-ctl.c b/src/lib/lttng-ctl/lttng-ctl.c > index 36069ef..356fb34 100644 --- a/src/lib/lttng-ctl/lttng-ctl.c +++ > b/src/lib/lttng-ctl/lttng-ctl.c @@ -885,13 +885,13 @@ int > lttng_calibrate(struct lttng_handle *handle, void > lttng_channel_set_default_attr(struct lttng_domain *domain, struct > lttng_channel_attr *attr) { - memset(attr, 0, sizeof(struct > lttng_channel_attr)); - /* Safety check */ if (attr == NULL || domain == > NULL) { return; } > > + memset(attr, 0, sizeof(struct lttng_channel_attr)); + switch > (domain->type) { case LTTNG_DOMAIN_KERNEL: attr->overwrite = > DEFAULT_CHANNEL_OVERWRITE; -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBAgAGBQJP0N9dAAoJEELoaioR9I02N3gH/3y9buoeohXbFW3YgF+ACTov Tzon+vqwglgYDBd6xCtxzNUxvMR/UdTeuOr8V6RYTApHEXzgM3viLlpLFjB5YR01 qHBCl5mpp7hRHE/VAWIS4wJfg1lBeR+m+dAg+FB0SZKxm/3FiLzkltI3IQlmi6zH qpNPXYfpEPm5Z6TmW0S+4P8qryVSkXO8OmM16NydXq3w6TOPmVkaqgYemNs2brAR pmcw9ab9OMxqhZwL3ZG39SbMxstqhZy6qlMo6OM87uoVzVh52OUFspkemmk7O2vv 2/RZVp+5b0NgA5I1QwAXB3uU6AXv+Ld+dbSg9KosppvfFAQWieCMcaElnORt63M= =LClK -----END PGP SIGNATURE----- From yannick.brosseau at gmail.com Thu Jun 7 13:43:07 2012 From: yannick.brosseau at gmail.com (Yannick Brosseau) Date: Thu, 07 Jun 2012 13:43:07 -0400 Subject: [lttng-dev] [lttng-tools PATCH] Move metset in channel_set_attr after the null check In-Reply-To: <4FD0DF60.2080709@efficios.com> References: <1339008870-4891-1-git-send-email-serres@live.ca> <4FD0DF60.2080709@efficios.com> Message-ID: <4FD0E82B.9050705@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2012-06-07 13:05, David Goulet wrote: > Hi Danny, > > I've merged your patch. Thanks! It's now upstream and in stable-2.0 :). > > Just a note for the next time. This is a "Fix" patch so in the commit message, > start with "Fix:". I've changed it for you for this commit. Only commit with > Fix: are picked for the stable-2.0 branch. That's something that need to be documented somewhere, even I was not aware of it. Yannick -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/Q6CEACgkQFQrZ7GzHX2qMdACgp+BK+/BnN5huiBwPn53TYDlS Ab8An0yG5OnKwl+Ugp/eKK2Ir3WnJoFu =7F2t -----END PGP SIGNATURE----- From mburton at ciena.com Thu Jun 7 14:16:17 2012 From: mburton at ciena.com (Burton, Michael) Date: Thu, 7 Jun 2012 14:16:17 -0400 Subject: [lttng-dev] UST Hanging at Tracepoint Message-ID: <3F56B905CF2083408E8482044F20428AEF604D22@ONWVEXCHMB01.ciena.com> I am working on getting LTTng 2.0 working in our 2.6.35 kernel. I have kernel tracing working but I'm having problems with UST. I have a program with a dynamic UST tracepoint. When I run the program with all UST events enabled (lttng enable-event -a -u) the program hangs on the tracepoint. The last line from the UST debug is: libust[6184/6185]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) Any insight into why this is happening? I've attached the UST and lttng-sessiond debug generated by the program. I am running the following: lttng-modules-2.6.32 (found through this mailing list) lttng-tools-2.0.1 lttng-ust-2.0.2 userspace-rcu-0.7.0 Thanks, Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: ust_debug.txt URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: sessiond_debug.txt URL: From hollis_blanchard at mentor.com Thu Jun 7 17:26:22 2012 From: hollis_blanchard at mentor.com (Hollis Blanchard) Date: Thu, 7 Jun 2012 14:26:22 -0700 Subject: [lttng-dev] -warn-common with lttng-ust 2.0.2 Message-ID: <4FD11C7E.7060303@mentor.com> Hi, I was adding an LTTng UST 2.0 tracepoint to an application that uses -warn-common (see http://www.math.utah.edu/docs/info/ld_2.html). I created a simple tracepoint, had lttng-gen-tp produce tracepoints.o, then linked that to the application, along with -llttng-ust. This results in some warnings: tracepoints.o: warning: common of `handle' overridden by definition /usr/local/lib/liblttng-ust.so: warning: defined here tracepoints.o: warning: common of `lttng_client_callbacks_overwrite' overridden by definition /usr/local/lib/liblttng-ust.so: warning: defined here tracepoints.o: warning: common of `lttng_client_callbacks_discard' overridden by definition /usr/local/lib/liblttng-ust.so: warning: defined here tracepoints.o: warning: common of `lttng_client_callbacks_metadata' overridden by definition /usr/local/lib/liblttng-ust.so: warning: defined here /usr/local/lib/liblttng-ust-tracepoint.so.0: warning: multiple common of `handle' tracepoints.o: warning: previous common is here This seems to be a valid warning. The LTTng UST headers contain definitions like this in include/lttng/ringbuffer-config.h: struct lttng_ust_shm_handle *handle; If two objects use that header, each will get a copy of "handle", right? -- Hollis Blanchard Mentor Graphics, Embedded Systems Division From JPelletier at analogic.com Fri Jun 8 08:46:14 2012 From: JPelletier at analogic.com (Pelletier, Joe) Date: Fri, 8 Jun 2012 08:46:14 -0400 Subject: [lttng-dev] Fedora distro Message-ID: Hello, I'm trying to find a tool chain for testing Fedora distributions and as I read through the LTTng web pages thought I had found exactly what I needed. Unfortunately for me only Ubuntu and OpenSuse distributions are available. Are there any plans for a Fedora release? If there aren't plans for a Fedora release, does anyone have any suggestions for a trace toolkit for Fedora? This sounds like a great tool! Thanks, Joe -------------- next part -------------- An HTML attachment was scrubbed... URL: From pavan.anumula at sasken.com Fri Jun 8 09:54:51 2012 From: pavan.anumula at sasken.com (Pavan Anumula) Date: Fri, 8 Jun 2012 19:24:51 +0530 Subject: [lttng-dev] Lttng start failed Message-ID: <47D610AD9C485E458630BA38C324D3B60113FB00F9EA@EXGMBX01.sasken.com> Hi , I am new to LTTng usage, I am trying to use LTTng 2.0.1 and LTTng-modules-2.0.2 for ARM(omapL138), with Linux kernel 2.6.33 wit RT-29 patch on it. After enabling all the kernel events I am facing the below errors. I loaded all the modules manually by insmod. root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng create newsessiom Session newsessiom created. Traces will be written in /home/root/lttng-traces/newsessiom-20110325-154922 root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng enable-event -a --kernel All kernel events are enabled in channel channel0 root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng start LTTng: Failure to write metadata to buffers (timeout) Error: Starting kernel trace failed Please kindly help me on this. Thanks in advance, Pavan ________________________________ SASKEN BUSINESS DISCLAIMER: This message may contain confidential, proprietary or legally privileged information. In case you are not the original intended Recipient of the message, you must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message and you are requested to delete it and inform the sender. Any views expressed in this message are those of the individual sender unless otherwise stated. Nothing contained in this message shall be construed as an offer or acceptance of any offer by Sasken Communication Technologies Limited ("Sasken") unless sent with that express intent and with due authority of Sasken. Sasken has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. Read Disclaimer at http://www.sasken.com/extras/mail_disclaimer.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From danny.serres at efficios.com Fri Jun 8 10:19:08 2012 From: danny.serres at efficios.com (Danny Serres) Date: Fri, 8 Jun 2012 10:19:08 -0400 Subject: [lttng-dev] [Fix:lttng-tools-PATCH] Fix:Change the type of enabled in lttng_event to a signed int Message-ID: <1339165148-6653-1-git-send-email-danny.serres@efficios.com> Signed-off-by: Danny Serres --- include/lttng/lttng.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/lttng/lttng.h b/include/lttng/lttng.h index 3b1be61..c80e282 100644 --- a/include/lttng/lttng.h +++ b/include/lttng/lttng.h @@ -214,7 +214,7 @@ struct lttng_event { enum lttng_loglevel_type loglevel_type; int loglevel; - uint32_t enabled; + int32_t enabled; /* Does not apply: -1 */ pid_t pid; char padding[LTTNG_EVENT_PADDING1]; -- 1.7.9.5 From alexandre.montplaisir at polymtl.ca Fri Jun 8 11:41:06 2012 From: alexandre.montplaisir at polymtl.ca (Alexandre Montplaisir) Date: Fri, 08 Jun 2012 11:41:06 -0400 Subject: [lttng-dev] Fedora distro In-Reply-To: References: Message-ID: <4FD21D12.4020907@polymtl.ca> Hi, On 12-06-08 08:46 AM, Pelletier, Joe wrote: > Hello, > I'm trying to find a tool chain for testing Fedora distributions and as I read through the LTTng web pages thought I had found exactly what I needed. Unfortunately for me only Ubuntu and OpenSuse distributions are available. Are there any plans for a Fedora release? If there aren't plans for a Fedora release, does anyone have any suggestions for a trace toolkit for Fedora? This sounds like a great tool! The OpenSuse builds also build packages for Fedora 15 and 16. If you follow that link on the download page, on the OBS page you have links to Fedora repositories on the right. The lttng-modules package however seems to have failed building, I don't know if it's temporary or if that one is simply not available on Fedora. In the meanwhile you might have to compile that part by hand. Cheers, -- Alexandre Montplaisir DORSAL lab, ?cole Polytechnique de Montr?al From yannick.brosseau at gmail.com Fri Jun 8 11:45:51 2012 From: yannick.brosseau at gmail.com (Yannick Brosseau) Date: Fri, 08 Jun 2012 11:45:51 -0400 Subject: [lttng-dev] Fedora distro In-Reply-To: References: Message-ID: <4FD21E2F.1060102@gmail.com> On 2012-06-08 08:46, Pelletier, Joe wrote: > > Hello, > > I'm trying to find a tool chain for testing Fedora distributions and > as I read through the LTTng web pages thought I had found exactly what > I needed. Unfortunately for me only Ubuntu and OpenSuse distributions > are available. Are there any plans for a Fedora release? If there > aren't plans for a Fedora release, does anyone have any suggestions > for a trace toolkit for Fedora? This sounds like a great tool! > > > Hi, I'm currently in the process of creating the packages for Fedora and uploading them to the distro. Currently waiting for reviews. I don't know how long it will take, but at least some part of LTTng should appear in Fedora in the future. Yannick Yannick -------------- next part -------------- An HTML attachment was scrubbed... URL: From mathieu.desnoyers at efficios.com Fri Jun 8 13:17:28 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Fri, 8 Jun 2012 13:17:28 -0400 Subject: [lttng-dev] -warn-common with lttng-ust 2.0.2 In-Reply-To: <4FD11C7E.7060303@mentor.com> References: <4FD11C7E.7060303@mentor.com> Message-ID: <20120608171728.GA22579@Krystal> * Hollis Blanchard (hollis_blanchard at mentor.com) wrote: > Hi, I was adding an LTTng UST 2.0 tracepoint to an application that uses > -warn-common (see http://www.math.utah.edu/docs/info/ld_2.html). I > created a simple tracepoint, had lttng-gen-tp produce tracepoints.o, > then linked that to the application, along with -llttng-ust. This > results in some warnings: > > tracepoints.o: warning: common of `handle' overridden by definition > /usr/local/lib/liblttng-ust.so: warning: defined here > tracepoints.o: warning: common of `lttng_client_callbacks_overwrite' overridden by definition > /usr/local/lib/liblttng-ust.so: warning: defined here > tracepoints.o: warning: common of `lttng_client_callbacks_discard' overridden by definition > /usr/local/lib/liblttng-ust.so: warning: defined here > tracepoints.o: warning: common of `lttng_client_callbacks_metadata' overridden by definition > /usr/local/lib/liblttng-ust.so: warning: defined here > /usr/local/lib/liblttng-ust-tracepoint.so.0: warning: multiple common of `handle' > tracepoints.o: warning: previous common is here > > This seems to be a valid warning. The LTTng UST headers contain > definitions like this in include/lttng/ringbuffer-config.h: > struct lttng_ust_shm_handle *handle; > > If two objects use that header, each will get a copy of "handle", right? Thanks for reporting, fixed by master commit: commit 5a821cd6258af4b44aac352bd89b715377cee7d2 Author: Mathieu Desnoyers Date: Fri Jun 8 13:17:05 2012 -0400 Fix: don't define variables in headers From Hollis Blanchard : > Hi, I was adding an LTTng UST 2.0 tracepoint to an application that uses > -warn-common (see http://www.math.utah.edu/docs/info/ld_2.html). I created > a simple tracepoint, had lttng-gen-tp produce tracepoints.o, then linked > that to the application, along with -llttng-ust. This results in some > warnings: > > tracepoints.o: warning: common of `handle' overridden by definition > /usr/local/lib/liblttng-ust.so: warning: defined here > tracepoints.o: warning: common of `lttng_client_callbacks_overwrite' overridden > +by definition > /usr/local/lib/liblttng-ust.so: warning: defined here > tracepoints.o: warning: common of `lttng_client_callbacks_discard' overridden by > +definition > /usr/local/lib/liblttng-ust.so: warning: defined here > tracepoints.o: warning: common of `lttng_client_callbacks_metadata' overridden > +by definition > /usr/local/lib/liblttng-ust.so: warning: defined here > /usr/local/lib/liblttng-ust-tracepoint.so.0: warning: multiple common of > +`handle' > tracepoints.o: warning: previous common is here > > This seems to be a valid warning. The LTTng UST headers contain > definitions like this in include/lttng/ringbuffer-config.h: > struct lttng_ust_shm_handle *handle; > > If two objects use that header, each will get a copy of "handle", right? handle: This was meant to be a forward declaration of struct lttng_ust_shm_handle so just removing the "*handle" part. This can be considered as a cleanup (or a fix without actual runtime effect). lttng_client_callbacks_*: if the cb values would have been used in the consumer daemon, this would have caused an issue: these would be set to NULL instead of the actual callback pointers. So in a way this is a fix, but it does not have any runtime impact at this point. Reported-by: Hollis Blanchard Signed-off-by: Mathieu Desnoyers -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Fri Jun 8 13:42:26 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Fri, 8 Jun 2012 13:42:26 -0400 Subject: [lttng-dev] Lttng start failed In-Reply-To: <47D610AD9C485E458630BA38C324D3B60113FB00F9EA@EXGMBX01.sasken.com> References: <47D610AD9C485E458630BA38C324D3B60113FB00F9EA@EXGMBX01.sasken.com> Message-ID: <20120608174226.GA23339@Krystal> * Pavan Anumula (pavan.anumula at sasken.com) wrote: > Hi , > > I am new to LTTng usage, I am trying to use LTTng 2.0.1 and LTTng-modules-2.0.2 for ARM(omapL138), with Linux kernel 2.6.33 wit RT-29 patch on it. > > After enabling all the kernel events I am facing the below errors. I loaded all the modules manually by insmod. > > > > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng create newsessiom > Session newsessiom created. > Traces will be written in /home/root/lttng-traces/newsessiom-20110325-154922 > > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng enable-event -a --kernel > All kernel events are enabled in channel channel0 > > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng start > LTTng: Failure to write metadata to buffers (timeout) > Error: Starting kernel trace failed > > Please kindly help me on this. I think you should look into the lttng-sessiond --help : --consumerd32-path PATH Specify path for the 32-bit UST consumer daemon binary --consumerd32-libdir PATH Specify path for the 32-bit UST consumer daemon libraries --consumerd64-path PATH Specify path for the 64-bit UST consumer daemon binary --consumerd64-libdir PATH Specify path for the 64-bit UST consumer daemon libraries options. My guess is that lttng-sessiond is not able to find the consumerd binary files, maybe due to a rename or because they have been moved after install. One more thing that might help is to launch the lttng-sessiond with "-vvv" : it will provide verbose output and let us know where things fall apart. Thanks, Mathieu > > > Thanks in advance, > Pavan > > ________________________________ > SASKEN BUSINESS DISCLAIMER: This message may contain confidential, proprietary or legally privileged information. In case you are not the original intended Recipient of the message, you must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message and you are requested to delete it and inform the sender. Any views expressed in this message are those of the individual sender unless otherwise stated. Nothing contained in this message shall be construed as an offer or acceptance of any offer by Sasken Communication Technologies Limited ("Sasken") unless sent with that express intent and with due authority of Sasken. Sasken has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. > Read Disclaimer at http://www.sasken.com/extras/mail_disclaimer.html > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From hollis_blanchard at mentor.com Fri Jun 8 14:26:23 2012 From: hollis_blanchard at mentor.com (Hollis Blanchard) Date: Fri, 8 Jun 2012 11:26:23 -0700 Subject: [lttng-dev] [RFC] [PATCH] -Werror=old-style-definition and UST tracepoints Message-ID: <4FD243CF.60104@mentor.com> I've defined a tracepoint without arguments: TRACEPOINT_EVENT( qemu_tb_hash, flushall, TP_ARGS(void), TP_FIELDS() ) When I build (with -Werror=old-style-definition), I get this error: In file included from /home/hollisb/work/qemu.git/exec.c:59:0: /home/hollisb/work/qemu.git/tracepoints.h:24:2: error: function declaration isn?t a prototype [-Werror=strict-prototypes] /home/hollisb/work/qemu.git/tracepoints.h: In function ?__tracepoint_cb_qemu_tb_hash___flushall?: /home/hollisb/work/qemu.git/tracepoints.h:24:2: error: old-style function definition [-Werror=old-style-definition] cc1: all warnings being treated as errors The preprocessed code looks like so: extern struct # 129 "/usr/local/include/lttng/tracepoint.h" 3 tracepoint # 24 "/home/hollisb/work/qemu.git/tracepoints.h" __tracepoint_qemu_tb_hash___flushall # 19 "/home/hollisb/work/qemu.git/tracepoints.h" ; static __attribute__ (( always_inline )) __inline__ void __tracepoint_cb_qemu_tb_hash___flushall # 19 "/home/hollisb/work/qemu.git/tracepoints.h" () { struct tracepoint_probe *__tp_probe; if (!tracepoint_dlopen.rcu_read_lock_sym_bp) return; tracepoint_dlopen.rcu_read_lock_sym_bp(); __tp_probe = ({ typeof(__tracepoint_qemu_tb_hash___flushall.probes) _________p1 = ((typeof(__tracepoint_qemu_tb_hash___flushall.probes)) (tracepoint_dlopen.rcu_dereference_sym_bp(((void *) (__tracepoint_qemu_tb_hash___flushall.probes))))); (_________p1); }); if (__builtin_expect(!!(!__tp_probe), 0)) goto end; do { void *__tp_cb = __tp_probe->func; void *__tp_data = __tp_probe->data; ((void (*)(void *__tp_data)) (__tp_cb)) (__tp_data); } while ((++__tp_probe)->func); end: tracepoint_dlopen.rcu_read_unlock_sym_bp(); } static __attribute__ (( always_inline )) __inline__ void I believe the problem comes from -Werror=old-style-definition not liking that empty "()", i.e. tracepoint_cb_qemu_tb_hash___flushall() { ... }. The following patch seems to work for me; is there a reason it isn't already written this way? diff --git a/include/lttng/tracepoint.h b/include/lttng/tracepoint.h index 4b773bb..60d8c73 100644 --- a/include/lttng/tracepoint.h +++ b/include/lttng/tracepoint.h @@ -88,7 +88,7 @@ extern "C" { #define _TP_EXDATA_VAR20(a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t) __tp_data,b,d,f,h,j,l,n,p,r,t /* _TP_EXPROTO* extract tuples of type, var */ -#define _TP_EXPROTO0() +#define _TP_EXPROTO0() void #define _TP_EXPROTO2(a,b) a b #define _TP_EXPROTO4(a,b,c,d) a b,c d #define _TP_EXPROTO6(a,b,c,d,e,f) a b,c d,e f If this is OK, I'm happy to resend as a proper patch. -- Hollis Blanchard Mentor Graphics, Embedded Systems Division From mburton at ciena.com Fri Jun 8 16:29:38 2012 From: mburton at ciena.com (Burton, Michael) Date: Fri, 8 Jun 2012 16:29:38 -0400 Subject: [lttng-dev] UST Hanging at Tracepoint In-Reply-To: <3F56B905CF2083408E8482044F20428AEF604D22@ONWVEXCHMB01.ciena.com> References: <3F56B905CF2083408E8482044F20428AEF604D22@ONWVEXCHMB01.ciena.com> Message-ID: <3F56B905CF2083408E8482044F20428AEF6A2DBB@ONWVEXCHMB01.ciena.com> For further insight, I've attached the strace from the hanging program. I also upgraded LTTng-UST to 2.0.3 and userspace-rcu to 0.7.3. Michael From: Burton, Michael [mailto:mburton at ciena.com] Sent: Thursday, June 07, 2012 2:16 PM To: lttng-dev at lists.lttng.org Subject: [lttng-dev] UST Hanging at Tracepoint I am working on getting LTTng 2.0 working in our 2.6.35 kernel. I have kernel tracing working but I'm having problems with UST. I have a program with a dynamic UST tracepoint. When I run the program with all UST events enabled (lttng enable-event -a -u) the program hangs on the tracepoint. The last line from the UST debug is: libust[6184/6185]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) Any insight into why this is happening? I've attached the UST and lttng-sessiond debug generated by the program. I am running the following: lttng-modules-2.6.32 (found through this mailing list) lttng-tools-2.0.1 lttng-ust-2.0.2 userspace-rcu-0.7.0 Thanks, Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: strace_debug.txt URL: From mathieu.desnoyers at efficios.com Fri Jun 8 21:38:55 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Fri, 8 Jun 2012 21:38:55 -0400 Subject: [lttng-dev] UST Hanging at Tracepoint In-Reply-To: <3F56B905CF2083408E8482044F20428AEF6A2DBB@ONWVEXCHMB01.ciena.com> References: <3F56B905CF2083408E8482044F20428AEF604D22@ONWVEXCHMB01.ciena.com> <3F56B905CF2083408E8482044F20428AEF6A2DBB@ONWVEXCHMB01.ciena.com> Message-ID: <20120609013854.GB10978@Krystal> * Burton, Michael (mburton at ciena.com) wrote: > For further insight, I've attached the strace from the hanging > program. I also upgraded LTTng-UST to 2.0.3 and userspace-rcu to > 0.7.3. Haven't had time to look at it yet, but could you also provide the output of: LTTNG_UST_DEBUG=1 youprogname (starting your UST instrumented program with LTTNG_UST_DEBUG=1 env. var. set) Thanks! Mathieu > > Michael > > From: Burton, Michael [mailto:mburton at ciena.com] > Sent: Thursday, June 07, 2012 2:16 PM > To: lttng-dev at lists.lttng.org > Subject: [lttng-dev] UST Hanging at Tracepoint > > I am working on getting LTTng 2.0 working in our 2.6.35 kernel. I have kernel tracing working but I'm having problems with UST. > > I have a program with a dynamic UST tracepoint. When I run the program with all UST events enabled (lttng enable-event -a -u) the program hangs on the tracepoint. The last line from the UST debug is: > > libust[6184/6185]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) > > Any insight into why this is happening? I've attached the UST and lttng-sessiond debug generated by the program. > > I am running the following: > lttng-modules-2.6.32 (found through this mailing list) > lttng-tools-2.0.1 > lttng-ust-2.0.2 > userspace-rcu-0.7.0 > > Thanks, > Michael Content-Description: strace_debug.txt > execve("/ciena/bin/idp", ["idp", "help"], [/* 11 vars */]) = 0 > brk(0) = 0x804e000 > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb783c000 > access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) > open("/etc/ld.so.cache", O_RDONLY) = 3 > fstat64(3, {st_mode=S_IFREG|0644, st_size=17420, ...}) = 0 > mmap2(NULL, 17420, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7837000 > close(3) = 0 > open("/usr/lib/liblttng-ust.so.0", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220A\0\0004\0\0\0$"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=209876, ...}) = 0 > mmap2(NULL, 212940, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7803000 > mmap2(0xb7832000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2e) = 0xb7832000 > close(3) = 0 > open("/usr/lib/liblttng-ust-ctl.so.0", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\200-\0\0004\0\0\0\274"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=140020, ...}) = 0 > mmap2(NULL, 142828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77e0000 > mmap2(0xb7802000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x21) = 0xb7802000 > close(3) = 0 > open("/usr/lib/liblttng-ust-fork.so.0", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\0\5\0\0004\0\0\0X"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=3864, ...}) = 0 > mmap2(NULL, 6820, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77de000 > mmap2(0xb77df000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77df000 > close(3) = 0 > open("/usr/lib/liblttng-ust-tracepoint.so.0", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220\v\0\0004\0\0\0008"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=16632, ...}) = 0 > mmap2(NULL, 19944, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d9000 > mmap2(0xb77dd000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3) = 0xb77dd000 > close(3) = 0 > open("/usr/lib/liblttng-ust-libc-wrapper.so.0", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\t\0\0004\0\0\0\364"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=9300, ...}) = 0 > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb77d8000 > mmap2(NULL, 12104, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d5000 > mmap2(0xb77d7000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77d7000 > close(3) = 0 > open("/usr/lib/liburcu.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d0000 > mmap2(0xb77d4000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77d4000 > close(3) = 0 > open("/usr/lib/liburcu-common.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340\5\0\0004\0\0\0\34"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=5084, ...}) = 0 > mmap2(NULL, 8032, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77ce000 > mmap2(0xb77cf000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77cf000 > close(3) = 0 > open("/usr/lib/liburcu-qsbr.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\22\0\0004\0\0\0\374"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77c9000 > mmap2(0xb77cd000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77cd000 > close(3) = 0 > open("/usr/lib/liburcu-mb.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77c4000 > mmap2(0xb77c8000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c8000 > close(3) = 0 > open("/usr/lib/liburcu-signal.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\23\0\0004\0\0\0\10"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=18712, ...}) = 0 > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb77c3000 > mmap2(NULL, 21704, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77bd000 > mmap2(0xb77c2000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c2000 > close(3) = 0 > open("/usr/lib/liburcu-cds.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\300\f\0\0004\0\0\0t"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=26204, ...}) = 0 > mmap2(NULL, 25004, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77b6000 > mmap2(0xb77bc000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb77bc000 > close(3) = 0 > open("/usr/lib/liburcu-bp.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320\24\0\0004\0\0\0\360"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=19968, ...}) = 0 > mmap2(NULL, 23128, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77b0000 > mmap2(0xb77b5000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77b5000 > close(3) = 0 > open("/lib/libdl.so.2", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0 \n\0\0004\0\0\0<"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=9668, ...}) = 0 > mmap2(NULL, 12408, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77ac000 > mmap2(0xb77ae000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77ae000 > close(3) = 0 > open("/lib/libuuid.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0@\r\0\0004\0\0\0\350"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=12872, ...}) = 0 > mmap2(NULL, 15632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77a8000 > mmap2(0xb77ab000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2) = 0xb77ab000 > close(3) = 0 > open("/lib/libc.so.6", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220n\1\0004\0\0\0L"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=1347780, ...}) = 0 > mmap2(NULL, 1358120, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb765c000 > mprotect(0xb77a1000, 4096, PROT_NONE) = 0 > mmap2(0xb77a2000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x145) = 0xb77a2000 > mmap2(0xb77a5000, 10536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb77a5000 > close(3) = 0 > open("/lib/librt.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0p\30\0\0004\0\0\0\230"..., 512) = 512 > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb765b000 > fstat64(3, {st_mode=S_IFREG|0755, st_size=30616, ...}) = 0 > mmap2(NULL, 33364, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7652000 > mmap2(0xb7659000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb7659000 > close(3) = 0 > open("/lib/libpthread.so.0", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360J\0\0004\0\0\0\354"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=88420, ...}) = 0 > mmap2(NULL, 98828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7639000 > mmap2(0xb764e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14) = 0xb764e000 > mmap2(0xb7650000, 4620, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7650000 > close(3) = 0 > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7638000 > mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7636000 > set_thread_area({entry_number:-1 -> 6, base_addr:0xb7636d80, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 > mprotect(0xb764e000, 4096, PROT_READ) = 0 > mprotect(0xb7659000, 4096, PROT_READ) = 0 > mprotect(0xb77a2000, 8192, PROT_READ) = 0 > mprotect(0xb77ae000, 4096, PROT_READ) = 0 > mprotect(0xb785a000, 4096, PROT_READ) = 0 > munmap(0xb7837000, 17420) = 0 > set_tid_address(0xb7636de8) = 3050 > set_robust_list(0xb7636df0, 0xc) = 0 > futex(0xbfaaf560, FUTEX_WAKE_PRIVATE, 1) = 0 > futex(0xbfaaf560, 0x189 /* FUTEX_??? */, 1, NULL, bfaaf570) = -1 EAGAIN (Resource temporarily unavailable) > rt_sigaction(SIGRTMIN, {0xb763d4f0, [], SA_SIGINFO}, NULL, 8) = 0 > rt_sigaction(SIGRT_1, {0xb763d9c0, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0 > rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0 > getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0 > uname({sys="Linux", node="5410", ...}) = 0 > rt_sigaction(SIGUSR1, {0xb77bec0a, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0 > futex(0xb77af06c, FUTEX_WAKE_PRIVATE, 2147483647) = 0 > brk(0) = 0x804e000 > brk(0x806f000) = 0x806f000 > gettimeofday({1339187216, 880428}, NULL) = 0 > open("/dev/urandom", O_RDONLY|O_LARGEFILE) = 3 > fcntl64(3, F_GETFD) = 0 > fcntl64(3, F_SETFD, FD_CLOEXEC) = 0 > getuid32() = 0 > gettimeofday({1339187216, 889603}, NULL) = 0 > gettimeofday({1339187216, 894471}, NULL) = 0 > read(3, "\275\0051F\215\373\371\202\377kfb/\362\303N"..., 16) = 16 > clock_gettime(CLOCK_REALTIME, {1339187216, 902329405}) = 0 > getuid32() = 0 > geteuid32() = 0 > rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0 > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6e36000 > mprotect(0xb6e36000, 4096, PROT_NONE) = 0 > clone(child_stack=0xb7634d64, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0xb7635bd8, {entry_number:6, base_addr:0xb7635b70, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xb7635bd8) = 3051 > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6636000 > mprotect(0xb6636000, 4096, PROT_NONE) = 0 > clone(child_stack=0xb6e34d64, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0xb6e35bd8, {entry_number:6, base_addr:0xb6e35b70, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xb6e35bd8) = 3052 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > gettimeofday({1339187216, 974869}, NULL) = 0 > futex(0xb7836e30, FUTEX_WAIT_PRIVATE, 0, {2, 927460405}) = -1 ETIMEDOUT (Connection timed out) > gettid() = 3050 > write(2, "libust[3050/3050]: Error: Timed o"..., 107libust[3050/3050]: Error: Timed out waiting for ltt-sessiond (in lttng_ust_init() at lttng-ust-comm.c:912) > ) = 107 > futex(0xb7836e80, FUTEX_WAIT_PRIVATE, 2, NULL > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From pavan.anumula at sasken.com Sat Jun 9 05:31:02 2012 From: pavan.anumula at sasken.com (Pavan Anumula) Date: Sat, 9 Jun 2012 15:01:02 +0530 Subject: [lttng-dev] Lttng start failed In-Reply-To: <20120608174226.GA23339@Krystal> References: <47D610AD9C485E458630BA38C324D3B60113FB00F9EA@EXGMBX01.sasken.com> <20120608174226.GA23339@Krystal> Message-ID: <47D610AD9C485E458630BA38C324D3B60113FB00FA00@EXGMBX01.sasken.com> Hi Mathue, Thanks for the quick reply, After inserting lttng modules , I had given the command "arm-none-linux-gnueabi-lttng-sessiond -vvv" as you said, Below is the output where there are error messages. Please help me in resolving the issue, SO that I can catch kernel and user traces. root at arago:/usr/lttng/modules# arm-none-linux-gnueabi-lttng-sessiond -d root at arago:/usr/lttng/modules# arm-none-linux-gnueabi-lttng-sessiond -vvv DEBUG3: Creating LTTng run directory: /var/run/lttng [in create_lttng_rundir() at main.c:4315] DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in main() at main.c:4543] DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command [in main() at main.c:4545] DEBUG1: Client socket path /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond [in main() at main.c:4593] DEBUG1: LTTng run directory path: /var/run/lttng [in main() at main.c:4594] DEBUG2: UST consumer 32 bits err path: /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] DEBUG2: UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] DEBUG2: UST consumer 64 bits err path: /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] DEBUG2: UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] Error: Already running daemon. Thanks in advance, Pavan -----Original Message----- From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] Sent: Friday, June 08, 2012 11:12 PM To: Pavan Anumula Cc: lttng-dev at lists.lttng.org Subject: Re: [lttng-dev] Lttng start failed * Pavan Anumula (pavan.anumula at sasken.com) wrote: > Hi , > > I am new to LTTng usage, I am trying to use LTTng 2.0.1 and LTTng-modules-2.0.2 for ARM(omapL138), with Linux kernel 2.6.33 wit RT-29 patch on it. > > After enabling all the kernel events I am facing the below errors. I loaded all the modules manually by insmod. > > > > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng create > newsessiom Session newsessiom created. > Traces will be written in > /home/root/lttng-traces/newsessiom-20110325-154922 > > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng enable-event > -a --kernel All kernel events are enabled in channel channel0 > > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng start > LTTng: Failure to write metadata to buffers (timeout) > Error: Starting kernel trace failed > > Please kindly help me on this. I think you should look into the lttng-sessiond --help : --consumerd32-path PATH Specify path for the 32-bit UST consumer daemon binary --consumerd32-libdir PATH Specify path for the 32-bit UST consumer daemon libraries --consumerd64-path PATH Specify path for the 64-bit UST consumer daemon binary --consumerd64-libdir PATH Specify path for the 64-bit UST consumer daemon libraries options. My guess is that lttng-sessiond is not able to find the consumerd binary files, maybe due to a rename or because they have been moved after install. One more thing that might help is to launch the lttng-sessiond with "-vvv" : it will provide verbose output and let us know where things fall apart. Thanks, Mathieu > > > Thanks in advance, > Pavan > > ________________________________ > SASKEN BUSINESS DISCLAIMER: This message may contain confidential, proprietary or legally privileged information. In case you are not the original intended Recipient of the message, you must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message and you are requested to delete it and inform the sender. Any views expressed in this message are those of the individual sender unless otherwise stated. Nothing contained in this message shall be construed as an offer or acceptance of any offer by Sasken Communication Technologies Limited ("Sasken") unless sent with that express intent and with due authority of Sasken. Sasken has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. > Read Disclaimer at http://www.sasken.com/extras/mail_disclaimer.html > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Sat Jun 9 08:32:52 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Sat, 9 Jun 2012 08:32:52 -0400 Subject: [lttng-dev] Lttng start failed In-Reply-To: <47D610AD9C485E458630BA38C324D3B60113FB00FA00@EXGMBX01.sasken.com> References: <47D610AD9C485E458630BA38C324D3B60113FB00F9EA@EXGMBX01.sasken.com> <20120608174226.GA23339@Krystal> <47D610AD9C485E458630BA38C324D3B60113FB00FA00@EXGMBX01.sasken.com> Message-ID: <20120609123251.GA14280@Krystal> * Pavan Anumula (pavan.anumula at sasken.com) wrote: > Hi Mathue, > > Thanks for the quick reply, > > After inserting lttng modules , I had given the command "arm-none-linux-gnueabi-lttng-sessiond -vvv" as you said, Below is the output where there are error messages. Please help me in resolving the issue, SO that I can catch kernel and user traces. > > > root at arago:/usr/lttng/modules# arm-none-linux-gnueabi-lttng-sessiond -d > root at arago:/usr/lttng/modules# arm-none-linux-gnueabi-lttng-sessiond -vvv > DEBUG3: Creating LTTng run directory: /var/run/lttng [in create_lttng_rundir() at main.c:4315] > DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in main() at main.c:4543] > DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command [in main() at main.c:4545] > DEBUG1: Client socket path /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] > DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond [in main() at main.c:4593] > DEBUG1: LTTng run directory path: /var/run/lttng [in main() at main.c:4594] > DEBUG2: UST consumer 32 bits err path: /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] > DEBUG2: UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] > DEBUG2: UST consumer 64 bits err path: /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] > DEBUG2: UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] > Error: Already running daemon. Please kill the lttng-sessiond that is already started (the one with -d). Instead of that one, run the sessiond with: lttng-sessiond -vvv (don't run lttng-sessiond -d before) Thanks, Mathieu > > > > > Thanks in advance, > Pavan > > -----Original Message----- > From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] > Sent: Friday, June 08, 2012 11:12 PM > To: Pavan Anumula > Cc: lttng-dev at lists.lttng.org > Subject: Re: [lttng-dev] Lttng start failed > > * Pavan Anumula (pavan.anumula at sasken.com) wrote: > > Hi , > > > > I am new to LTTng usage, I am trying to use LTTng 2.0.1 and LTTng-modules-2.0.2 for ARM(omapL138), with Linux kernel 2.6.33 wit RT-29 patch on it. > > > > After enabling all the kernel events I am facing the below errors. I loaded all the modules manually by insmod. > > > > > > > > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng create > > newsessiom Session newsessiom created. > > Traces will be written in > > /home/root/lttng-traces/newsessiom-20110325-154922 > > > > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng enable-event > > -a --kernel All kernel events are enabled in channel channel0 > > > > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng start > > LTTng: Failure to write metadata to buffers (timeout) > > Error: Starting kernel trace failed > > > > Please kindly help me on this. > > I think you should look into the lttng-sessiond --help : > > --consumerd32-path PATH Specify path for the 32-bit UST consumer daemon binary > --consumerd32-libdir PATH Specify path for the 32-bit UST consumer daemon libraries > --consumerd64-path PATH Specify path for the 64-bit UST consumer daemon binary > --consumerd64-libdir PATH Specify path for the 64-bit UST consumer daemon libraries > > options. My guess is that lttng-sessiond is not able to find the consumerd binary files, maybe due to a rename or because they have been moved after install. > > One more thing that might help is to launch the lttng-sessiond with "-vvv" : it will provide verbose output and let us know where things fall apart. > > Thanks, > > Mathieu > > > > > > > > Thanks in advance, > > Pavan > > > > ________________________________ > > SASKEN BUSINESS DISCLAIMER: This message may contain confidential, proprietary or legally privileged information. In case you are not the original intended Recipient of the message, you must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message and you are requested to delete it and inform the sender. Any views expressed in this message are those of the individual sender unless otherwise stated. Nothing contained in this message shall be construed as an offer or acceptance of any offer by Sasken Communication Technologies Limited ("Sasken") unless sent with that express intent and with due authority of Sasken. Sasken has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. > > Read Disclaimer at http://www.sasken.com/extras/mail_disclaimer.html > > > _______________________________________________ > > lttng-dev mailing list > > lttng-dev at lists.lttng.org > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > > -- > Mathieu Desnoyers > Operating System Efficiency R&D Consultant EfficiOS Inc. > http://www.efficios.com -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From toojays at toojays.net Sat Jun 9 19:37:01 2012 From: toojays at toojays.net (John Steele Scott) Date: Sun, 10 Jun 2012 09:07:01 +0930 Subject: [lttng-dev] lttng-ust with --std=c99 -pedantic Message-ID: I want to add lttng-ust tracepoints to a program which builds with "--std=c99 -pedantic". Right now this does not work. Using the demo program as an example, if you enable --std=c99, the first issue looks like: jscott at saaz:~/src/lttng-ust/tests/demo$ ccache gcc -std=c99 -DHAVE_CONFIG_H -I. -I../.. -I../../include/lttng -I../../include -Wall -g -O2 -MT demo.o -MD -MP -MF .deps/demo.Tpo -c -o demo.o demo.c In file included from demo.c:34:0: ust_tests_demo.h: In function ?__tracepoint_cb_ust_tests_demo___starting?: ust_tests_demo.h:27:23: warning: implicit declaration of function ?typeof? [-Wimplicit-function-declaration] ust_tests_demo.h:27:218: error: expected ?;? before ?_________p1? ust_tests_demo.h:27:395: error: ?_________p1? undeclared (first use in this function) ust_tests_demo.h:27:395: note: each undeclared identifier is reported only once for each function it appears in In file included from demo.c:34:0: ust_tests_demo.h: In function ?__tracepoint_cb_ust_tests_demo___done?: ust_tests_demo.h:35:214: error: expected ?;? before ?_________p1? ust_tests_demo.h:35:383: error: ?_________p1? undeclared (first use in this function) In file included from demo.c:35:0: ust_tests_demo2.h: In function ?__tracepoint_cb_ust_tests_demo2___loop?: ust_tests_demo2.h:27:299: error: expected ?;? before ?_________p1? ust_tests_demo2.h:27:470: error: ?_________p1? undeclared (first use in this function) In file included from demo.c:36:0: ust_tests_demo3.h: In function ?__tracepoint_cb_ust_tests_demo3___done?: ust_tests_demo3.h:27:215: error: expected ?;? before ?_________p1? ust_tests_demo3.h:27:386: error: ?_________p1? undeclared (first use in this function) This can be easily resolved by using __typeof__() instead of typeof(). Then I can build with --std=c99. But adding -pedantic still fails: jscott at saaz:~/src/lttng-ust/tests/demo$ ccache gcc -std=c99 -Dtypeof=__typeof__ -pedantic -DHAVE_CONFIG_H -I. -I../.. -I../../include/lttng -I../../include -Wall -g -O2 -MT demo.o -MD -MP -MF .deps/demo.Tpo -c -o demo.o demo.c In file included from ust_tests_demo.h:25:0, from demo.c:34: ../../include/lttng/tracepoint.h: In function ?__tracepoints__init?: ../../include/lttng/tracepoint.h:251:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] ../../include/lttng/tracepoint.h:255:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] ../../include/lttng/tracepoint.h:260:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] ../../include/lttng/tracepoint.h:264:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] ../../include/lttng/tracepoint.h:268:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] In file included from demo.c:34:0: ust_tests_demo.h: In function ?__tracepoint_cb_ust_tests_demo___starting?: ust_tests_demo.h:27:161: warning: ISO C forbids braced-groups within expressions [-pedantic] ust_tests_demo.h:27:549: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] In file included from demo.c:34:0: ust_tests_demo.h: In function ?__tracepoint_cb_ust_tests_demo___done?: ust_tests_demo.h:35:161: warning: ISO C forbids braced-groups within expressions [-pedantic] ust_tests_demo.h:35:537: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] In file included from demo.c:35:0: ust_tests_demo2.h: In function ?__tracepoint_cb_ust_tests_demo2___loop?: ust_tests_demo2.h:27:245: warning: ISO C forbids braced-groups within expressions [-pedantic] ust_tests_demo2.h:27:624: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] In file included from demo.c:36:0: ust_tests_demo3.h: In function ?__tracepoint_cb_ust_tests_demo3___done?: ust_tests_demo3.h:27:161: warning: ISO C forbids braced-groups within expressions [-pedantic] ust_tests_demo3.h:27:540: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] This is with 4.6.1-9ubuntu3 on Ubuntu 11.10, lttng-ust master 5a821c. Would it be particularly difficult to make the lttng-ust tracepoints compatible with programs built with -pedantic? cheers, John From pavan.anumula at sasken.com Mon Jun 11 01:42:28 2012 From: pavan.anumula at sasken.com (Pavan Anumula) Date: Mon, 11 Jun 2012 11:12:28 +0530 Subject: [lttng-dev] Lttng start failed In-Reply-To: <20120609123251.GA14280@Krystal> References: <47D610AD9C485E458630BA38C324D3B60113FB00F9EA@EXGMBX01.sasken.com> <20120608174226.GA23339@Krystal> <47D610AD9C485E458630BA38C324D3B60113FB00FA00@EXGMBX01.sasken.com> <20120609123251.GA14280@Krystal> Message-ID: <47D610AD9C485E458630BA38C324D3B60113FB00FA1E@EXGMBX01.sasken.com> Hi Mathieu, Please find the info below as per your comments ########## Output of command "arm-none-linux-gnueabi-lttng-sessiond -vvv " , After loading the modules ####### DEBUG3: Creating LTTng run directory: /var/run/lttng [in create_lttng_rundir() at main.c:4315] DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in main() at main.c:4543] DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command [in main() at main.c:4545] DEBUG1: Client socket path /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond [in main() at main.c:4593] DEBUG1: LTTng run directory path: /var/run/lttng [in main() at main.c:4594] DEBUG2: UST consumer 32 bits err path: /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] DEBUG2: UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] DEBUG2: UST consumer 64 bits err path: /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] DEBUG2: UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] DEBUG3: Created hashtable size 4 at 0x4a080 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x4a168 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Creating consumer directory: /var/run/lttng/kconsumerd [in set_consumer_sockets() at main.c:4357] FATAL: Module lttng_tracer not found. Error: Unable to load module lttng-tracer DEBUG2: Kernel tracer version validated (major version 2) [in kernel_validate_version() at kernel.c:675] DEBUG1: Modprobe successfully lttng-ftrace [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-kprobes [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-kretprobes [in modprobe_lttng_data() at modprobe.c:199] FATAL: Module lttng_lib_ring_buffer not found. Error: Unable to load module lttng-lib-ring-buffer FATAL: Module lttng_ring_buffer_client_discard not found. Error: Unable to load module lttng-ring-buffer-client-discard FATAL: Module lttng_ring_buffer_client_overwrite not found. Error: Unable to load module lttng-ring-buffer-client-overwrite FATAL: Module lttng_ring_buffer_metadata_client not found. Error: Unable to load module lttng-ring-buffer-metadata-client FATAL: Module lttng_ring_buffer_client_mmap_discard not found. Error: Unable to load module lttng-ring-buffer-client-mmap-discard FATAL: Module lttng_ring_buffer_client_mmap_overwrite not found. Error: Unable to load module lttng-ring-buffer-client-mmap-overwrite FATAL: Module lttng_ring_buffer_metadata_mmap_client not found. Error: Unable to load module lttng-ring-buffer-metadata-mmap-client FATAL: Module lttng_probe_lttng not found. Error: Unable to load module lttng-probe-lttng DEBUG1: Modprobe successfully lttng-types [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-block [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-irq [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-kvm [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-sched [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-signal [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-statedump [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-timer [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Kernel tracer fd 6 [in init_kernel_tracer() at main.c:1887] DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd64 [in set_consumer_sockets() at main.c:4357] DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() at main.c:4357] DEBUG1: Signal handler set for SIGTERM, SIGPIPE and SIGINT [in set_signal_handler() at main.c:4449] DEBUG1: All permissions are set [in set_permissions() at main.c:4250] DEBUG1: poll set max size set to 65535 [in compat_poll_set_max_size() at compat-poll.c:196] DEBUG1: [thread] Manage client started [in thread_manage_clients() at main.c:3794] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 9 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3826] DEBUG1: [thread] Dispatch UST command started [in thread_dispatch_ust_registration() at main.c:1324] DEBUG1: Futex n to 1 prepare done [in futex_nto1_prepare() at futex.c:73] DEBUG1: Woken up but nothing in the UST command queue [in thread_dispatch_ust_registration() at main.c:1334] DEBUG1: Thread manage kernel started [in thread_manage_kernel() at main.c:876] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 11 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: Updating kernel poll set [in update_kernel_poll() at main.c:748] DEBUG1: Thread kernel polling on 2 fds [in thread_manage_kernel() at main.c:905] DEBUG1: [thread] Manage application started [in thread_manage_apps() at main.c:1179] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 13 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: Apps thread polling on 2 fds [in thread_manage_apps() at main.c:1200] DEBUG1: [thread] Manage application registration started [in thread_registration_apps() at main.c:1392] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 10 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: Notifying applications of session daemon state: 1 [in notify_ust_apps() at main.c:687] DEBUG1: Got the wait shm fd 15 [in get_wait_shm() at shm.c:117] DEBUG1: Futex wait update active 1 [in futex_wait_update() at futex.c:62] DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1423] #### Output of command "arm-none-linux-gnueabi-lttng-sessiond -vvv " before loading the modules ############# DEBUG3: Creating LTTng run directory: /var/run/lttng [in create_lttng_rundir() at main.c:4315] DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in main() at main.c:4543] DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command [in main() at main.c:4545] DEBUG1: Client socket path /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond [in main() at main.c:4593] DEBUG1: LTTng run directory path: /var/run/lttng [in main() at main.c:4594] DEBUG2: UST consumer 32 bits err path: /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] DEBUG2: UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] DEBUG2: UST consumer 64 bits err path: /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] DEBUG2: UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] DEBUG3: Created hashtable size 4 at 0x4a080 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x4a168 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Creating consumer directory: /var/run/lttng/kconsumerd [in set_consumer_sockets() at main.c:4357] FATAL: Module lttng_tracer not found. Error: Unable to load module lttng-tracer DEBUG1: Failed to open /proc/lttng [in init_kernel_tracer() at main.c:1871] Error: Unable to remove module lttng-tracer Warning: No kernel tracer available DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd64 [in set_consumer_sockets() at main.c:4357] DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() at main.c:4357] DEBUG1: Signal handler set for SIGTERM, SIGPIPE and SIGINT [in set_signal_handler() at main.c:4449] DEBUG1: All permissions are set [in set_permissions() at main.c:4250] DEBUG1: poll set max size set to 65535 [in compat_poll_set_max_size() at compat-poll.c:196] DEBUG1: [thread] Manage application started [in thread_manage_apps() at main.c:1179] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 12 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: Apps thread polling on 2 fds [in thread_manage_apps() at main.c:1200] DEBUG1: Thread manage kernel started [in thread_manage_kernel() at main.c:876] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 10 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: Updating kernel poll set [in update_kernel_poll() at main.c:748] DEBUG1: Thread kernel polling on 2 fds [in thread_manage_kernel() at main.c:905] DEBUG1: [thread] Manage application registration started [in thread_registration_apps() at main.c:1392] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 9 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: Notifying applications of session daemon state: 1 [in notify_ust_apps() at main.c:687] DEBUG1: [thread] Dispatch UST command started [in thread_dispatch_ust_registration() at main.c:1324] DEBUG1: Futex n to 1 prepare done [in futex_nto1_prepare() at futex.c:73] DEBUG1: Woken up but nothing in the UST command queue [in thread_dispatch_ust_registration() at main.c:1334] DEBUG1: [thread] Manage client started [in thread_manage_clients() at main.c:3794] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 8 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3826] DEBUG1: Got the wait shm fd 14 [in get_wait_shm() at shm.c:117] DEBUG1: Futex wait update active 1 [in futex_wait_update() at futex.c:62] DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1423] Regards, Pavan -----Original Message----- From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] Sent: Saturday, June 09, 2012 6:03 PM To: Pavan Anumula Cc: lttng-dev at lists.lttng.org Subject: Re: [lttng-dev] Lttng start failed * Pavan Anumula (pavan.anumula at sasken.com) wrote: > Hi Mathue, > > Thanks for the quick reply, > > After inserting lttng modules , I had given the command "arm-none-linux-gnueabi-lttng-sessiond -vvv" as you said, Below is the output where there are error messages. Please help me in resolving the issue, SO that I can catch kernel and user traces. > > > root at arago:/usr/lttng/modules# arm-none-linux-gnueabi-lttng-sessiond > -d root at arago:/usr/lttng/modules# > arm-none-linux-gnueabi-lttng-sessiond -vvv > DEBUG3: Creating LTTng run directory: /var/run/lttng [in > create_lttng_rundir() at main.c:4315] > DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in > main() at main.c:4543] > DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command > [in main() at main.c:4545] > DEBUG1: Client socket path /var/run/lttng/client-lttng-sessiond [in > main() at main.c:4592] > DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond [in > main() at main.c:4593] > DEBUG1: LTTng run directory path: /var/run/lttng [in main() at > main.c:4594] > DEBUG2: UST consumer 32 bits err path: > /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] > DEBUG2: UST consumer 32 bits cmd path: > /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] > DEBUG2: UST consumer 64 bits err path: > /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] > DEBUG2: UST consumer 64 bits cmd path: > /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] > Error: Already running daemon. Please kill the lttng-sessiond that is already started (the one with -d). Instead of that one, run the sessiond with: lttng-sessiond -vvv (don't run lttng-sessiond -d before) Thanks, Mathieu > > > > > Thanks in advance, > Pavan > > -----Original Message----- > From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] > Sent: Friday, June 08, 2012 11:12 PM > To: Pavan Anumula > Cc: lttng-dev at lists.lttng.org > Subject: Re: [lttng-dev] Lttng start failed > > * Pavan Anumula (pavan.anumula at sasken.com) wrote: > > Hi , > > > > I am new to LTTng usage, I am trying to use LTTng 2.0.1 and LTTng-modules-2.0.2 for ARM(omapL138), with Linux kernel 2.6.33 wit RT-29 patch on it. > > > > After enabling all the kernel events I am facing the below errors. I loaded all the modules manually by insmod. > > > > > > > > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng create > > newsessiom Session newsessiom created. > > Traces will be written in > > /home/root/lttng-traces/newsessiom-20110325-154922 > > > > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng enable-event > > -a --kernel All kernel events are enabled in channel channel0 > > > > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng start > > LTTng: Failure to write metadata to buffers (timeout) > > Error: Starting kernel trace failed > > > > Please kindly help me on this. > > I think you should look into the lttng-sessiond --help : > > --consumerd32-path PATH Specify path for the 32-bit UST consumer daemon binary > --consumerd32-libdir PATH Specify path for the 32-bit UST consumer daemon libraries > --consumerd64-path PATH Specify path for the 64-bit UST consumer daemon binary > --consumerd64-libdir PATH Specify path for the 64-bit UST consumer daemon libraries > > options. My guess is that lttng-sessiond is not able to find the consumerd binary files, maybe due to a rename or because they have been moved after install. > > One more thing that might help is to launch the lttng-sessiond with "-vvv" : it will provide verbose output and let us know where things fall apart. > > Thanks, > > Mathieu > > > > > > > > Thanks in advance, > > Pavan > > > > ________________________________ > > SASKEN BUSINESS DISCLAIMER: This message may contain confidential, proprietary or legally privileged information. In case you are not the original intended Recipient of the message, you must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message and you are requested to delete it and inform the sender. Any views expressed in this message are those of the individual sender unless otherwise stated. Nothing contained in this message shall be construed as an offer or acceptance of any offer by Sasken Communication Technologies Limited ("Sasken") unless sent with that express intent and with due authority of Sasken. Sasken has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. > > Read Disclaimer at http://www.sasken.com/extras/mail_disclaimer.html > > > _______________________________________________ > > lttng-dev mailing list > > lttng-dev at lists.lttng.org > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > > -- > Mathieu Desnoyers > Operating System Efficiency R&D Consultant EfficiOS Inc. > http://www.efficios.com -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Mon Jun 11 09:52:29 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Mon, 11 Jun 2012 09:52:29 -0400 Subject: [lttng-dev] [RFC] [PATCH] -Werror=old-style-definition and UST tracepoints In-Reply-To: <4FD243CF.60104@mentor.com> References: <4FD243CF.60104@mentor.com> Message-ID: <20120611135229.GA18709@Krystal> * Hollis Blanchard (hollis_blanchard at mentor.com) wrote: [...] > I believe the problem comes from -Werror=old-style-definition not liking > that empty "()", i.e. tracepoint_cb_qemu_tb_hash___flushall() { ... }. > The following patch seems to work for me; is there a reason it isn't > already written this way? Good catch ! Fixed as master commit: commit 86637aa6452f66dee5f769461668c2ea059b3a30 Author: Mathieu Desnoyers Date: Mon Jun 11 09:53:07 2012 -0400 Fix: tracepoint.h should not generate old-style definitions and merged into stable-2.0. Thanks! Mathieu > > diff --git a/include/lttng/tracepoint.h b/include/lttng/tracepoint.h > > index 4b773bb..60d8c73 100644 > > --- a/include/lttng/tracepoint.h > > +++ b/include/lttng/tracepoint.h > > @@ -88,7 +88,7 @@ extern "C" { > > #define _TP_EXDATA_VAR20(a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t) __tp_data,b,d,f,h,j,l,n,p,r,t > > > > /* _TP_EXPROTO* extract tuples of type, var */ > > -#define _TP_EXPROTO0() > > +#define _TP_EXPROTO0() void > > #define _TP_EXPROTO2(a,b) a b > > #define _TP_EXPROTO4(a,b,c,d) a b,c d > > #define _TP_EXPROTO6(a,b,c,d,e,f) a b,c d,e f > > > If this is OK, I'm happy to resend as a proper patch. > > -- > Hollis Blanchard > Mentor Graphics, Embedded Systems Division > > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Mon Jun 11 09:54:22 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Mon, 11 Jun 2012 09:54:22 -0400 Subject: [lttng-dev] Lttng start failed In-Reply-To: <47D610AD9C485E458630BA38C324D3B60113FB00FA1E@EXGMBX01.sasken.com> References: <47D610AD9C485E458630BA38C324D3B60113FB00F9EA@EXGMBX01.sasken.com> <20120608174226.GA23339@Krystal> <47D610AD9C485E458630BA38C324D3B60113FB00FA00@EXGMBX01.sasken.com> <20120609123251.GA14280@Krystal> <47D610AD9C485E458630BA38C324D3B60113FB00FA1E@EXGMBX01.sasken.com> Message-ID: <20120611135422.GA19020@Krystal> * Pavan Anumula (pavan.anumula at sasken.com) wrote: > Hi Mathieu, > > Please find the info below as per your comments > > > ########## Output of command "arm-none-linux-gnueabi-lttng-sessiond -vvv " , After loading the modules ####### > > DEBUG3: Creating LTTng run directory: /var/run/lttng [in create_lttng_rundir() at main.c:4315] > DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in main() at main.c:4543] > DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command [in main() at main.c:4545] > DEBUG1: Client socket path /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] > DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond [in main() at main.c:4593] > DEBUG1: LTTng run directory path: /var/run/lttng [in main() at main.c:4594] > DEBUG2: UST consumer 32 bits err path: /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] > DEBUG2: UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] > DEBUG2: UST consumer 64 bits err path: /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] > DEBUG2: UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] > DEBUG3: Created hashtable size 4 at 0x4a080 of type 1 [in lttng_ht_new() at hashtable.c:96] > DEBUG3: Created hashtable size 4 at 0x4a168 of type 1 [in lttng_ht_new() at hashtable.c:96] > DEBUG2: Creating consumer directory: /var/run/lttng/kconsumerd [in set_consumer_sockets() at main.c:4357] > FATAL: Module lttng_tracer not found. > Error: Unable to load module lttng-tracer Hrm. You want to do kernel tracing, but modprobe cannot find the lttng kernel tracer modules. You might want to run depmod -a or something like that on your target, and ensure that modprobe works properly. Thanks, Mathieu > DEBUG2: Kernel tracer version validated (major version 2) [in kernel_validate_version() at kernel.c:675] > DEBUG1: Modprobe successfully lttng-ftrace [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-kprobes [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-kretprobes [in modprobe_lttng_data() at modprobe.c:199] > FATAL: Module lttng_lib_ring_buffer not found. > Error: Unable to load module lttng-lib-ring-buffer > FATAL: Module lttng_ring_buffer_client_discard not found. > Error: Unable to load module lttng-ring-buffer-client-discard > FATAL: Module lttng_ring_buffer_client_overwrite not found. > Error: Unable to load module lttng-ring-buffer-client-overwrite > FATAL: Module lttng_ring_buffer_metadata_client not found. > Error: Unable to load module lttng-ring-buffer-metadata-client > FATAL: Module lttng_ring_buffer_client_mmap_discard not found. > Error: Unable to load module lttng-ring-buffer-client-mmap-discard > FATAL: Module lttng_ring_buffer_client_mmap_overwrite not found. > Error: Unable to load module lttng-ring-buffer-client-mmap-overwrite > FATAL: Module lttng_ring_buffer_metadata_mmap_client not found. > Error: Unable to load module lttng-ring-buffer-metadata-mmap-client > FATAL: Module lttng_probe_lttng not found. > Error: Unable to load module lttng-probe-lttng > DEBUG1: Modprobe successfully lttng-types [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-block [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-irq [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-kvm [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-sched [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-signal [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-statedump [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-timer [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Kernel tracer fd 6 [in init_kernel_tracer() at main.c:1887] > DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd64 [in set_consumer_sockets() at main.c:4357] > DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() at main.c:4357] > DEBUG1: Signal handler set for SIGTERM, SIGPIPE and SIGINT [in set_signal_handler() at main.c:4449] > DEBUG1: All permissions are set [in set_permissions() at main.c:4250] > DEBUG1: poll set max size set to 65535 [in compat_poll_set_max_size() at compat-poll.c:196] > DEBUG1: [thread] Manage client started [in thread_manage_clients() at main.c:3794] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: fd 9 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3826] > DEBUG1: [thread] Dispatch UST command started [in thread_dispatch_ust_registration() at main.c:1324] > DEBUG1: Futex n to 1 prepare done [in futex_nto1_prepare() at futex.c:73] > DEBUG1: Woken up but nothing in the UST command queue [in thread_dispatch_ust_registration() at main.c:1334] > DEBUG1: Thread manage kernel started [in thread_manage_kernel() at main.c:876] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: fd 11 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: Updating kernel poll set [in update_kernel_poll() at main.c:748] > DEBUG1: Thread kernel polling on 2 fds [in thread_manage_kernel() at main.c:905] > DEBUG1: [thread] Manage application started [in thread_manage_apps() at main.c:1179] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: fd 13 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: Apps thread polling on 2 fds [in thread_manage_apps() at main.c:1200] > DEBUG1: [thread] Manage application registration started [in thread_registration_apps() at main.c:1392] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: fd 10 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: Notifying applications of session daemon state: 1 [in notify_ust_apps() at main.c:687] > DEBUG1: Got the wait shm fd 15 [in get_wait_shm() at shm.c:117] > DEBUG1: Futex wait update active 1 [in futex_wait_update() at futex.c:62] > DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1423] > > > > #### Output of command "arm-none-linux-gnueabi-lttng-sessiond -vvv " before loading the modules ############# > > DEBUG3: Creating LTTng run directory: /var/run/lttng [in create_lttng_rundir() at main.c:4315] > DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in main() at main.c:4543] > DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command [in main() at main.c:4545] > DEBUG1: Client socket path /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] > DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond [in main() at main.c:4593] > DEBUG1: LTTng run directory path: /var/run/lttng [in main() at main.c:4594] > DEBUG2: UST consumer 32 bits err path: /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] > DEBUG2: UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] > DEBUG2: UST consumer 64 bits err path: /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] > DEBUG2: UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] > DEBUG3: Created hashtable size 4 at 0x4a080 of type 1 [in lttng_ht_new() at hashtable.c:96] > DEBUG3: Created hashtable size 4 at 0x4a168 of type 1 [in lttng_ht_new() at hashtable.c:96] > DEBUG2: Creating consumer directory: /var/run/lttng/kconsumerd [in set_consumer_sockets() at main.c:4357] > FATAL: Module lttng_tracer not found. > Error: Unable to load module lttng-tracer > DEBUG1: Failed to open /proc/lttng [in init_kernel_tracer() at main.c:1871] > Error: Unable to remove module lttng-tracer > Warning: No kernel tracer available > DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd64 [in set_consumer_sockets() at main.c:4357] > DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() at main.c:4357] > DEBUG1: Signal handler set for SIGTERM, SIGPIPE and SIGINT [in set_signal_handler() at main.c:4449] > DEBUG1: All permissions are set [in set_permissions() at main.c:4250] > DEBUG1: poll set max size set to 65535 [in compat_poll_set_max_size() at compat-poll.c:196] > DEBUG1: [thread] Manage application started [in thread_manage_apps() at main.c:1179] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: fd 12 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: Apps thread polling on 2 fds [in thread_manage_apps() at main.c:1200] > DEBUG1: Thread manage kernel started [in thread_manage_kernel() at main.c:876] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: fd 10 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: Updating kernel poll set [in update_kernel_poll() at main.c:748] > DEBUG1: Thread kernel polling on 2 fds [in thread_manage_kernel() at main.c:905] > DEBUG1: [thread] Manage application registration started [in thread_registration_apps() at main.c:1392] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: fd 9 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: Notifying applications of session daemon state: 1 [in notify_ust_apps() at main.c:687] > DEBUG1: [thread] Dispatch UST command started [in thread_dispatch_ust_registration() at main.c:1324] > DEBUG1: Futex n to 1 prepare done [in futex_nto1_prepare() at futex.c:73] > DEBUG1: Woken up but nothing in the UST command queue [in thread_dispatch_ust_registration() at main.c:1334] > DEBUG1: [thread] Manage client started [in thread_manage_clients() at main.c:3794] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: fd 8 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3826] > DEBUG1: Got the wait shm fd 14 [in get_wait_shm() at shm.c:117] > DEBUG1: Futex wait update active 1 [in futex_wait_update() at futex.c:62] > DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1423] > > > Regards, > Pavan > > -----Original Message----- > From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] > Sent: Saturday, June 09, 2012 6:03 PM > To: Pavan Anumula > Cc: lttng-dev at lists.lttng.org > Subject: Re: [lttng-dev] Lttng start failed > > * Pavan Anumula (pavan.anumula at sasken.com) wrote: > > Hi Mathue, > > > > Thanks for the quick reply, > > > > After inserting lttng modules , I had given the command "arm-none-linux-gnueabi-lttng-sessiond -vvv" as you said, Below is the output where there are error messages. Please help me in resolving the issue, SO that I can catch kernel and user traces. > > > > > > root at arago:/usr/lttng/modules# arm-none-linux-gnueabi-lttng-sessiond > > -d root at arago:/usr/lttng/modules# > > arm-none-linux-gnueabi-lttng-sessiond -vvv > > DEBUG3: Creating LTTng run directory: /var/run/lttng [in > > create_lttng_rundir() at main.c:4315] > > DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in > > main() at main.c:4543] > > DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command > > [in main() at main.c:4545] > > DEBUG1: Client socket path /var/run/lttng/client-lttng-sessiond [in > > main() at main.c:4592] > > DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond [in > > main() at main.c:4593] > > DEBUG1: LTTng run directory path: /var/run/lttng [in main() at > > main.c:4594] > > DEBUG2: UST consumer 32 bits err path: > > /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] > > DEBUG2: UST consumer 32 bits cmd path: > > /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] > > DEBUG2: UST consumer 64 bits err path: > > /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] > > DEBUG2: UST consumer 64 bits cmd path: > > /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] > > Error: Already running daemon. > > Please kill the lttng-sessiond that is already started (the one with -d). Instead of that one, run the sessiond with: > > lttng-sessiond -vvv > > (don't run lttng-sessiond -d before) > > Thanks, > > Mathieu > > > > > > > > > > > Thanks in advance, > > Pavan > > > > -----Original Message----- > > From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] > > Sent: Friday, June 08, 2012 11:12 PM > > To: Pavan Anumula > > Cc: lttng-dev at lists.lttng.org > > Subject: Re: [lttng-dev] Lttng start failed > > > > * Pavan Anumula (pavan.anumula at sasken.com) wrote: > > > Hi , > > > > > > I am new to LTTng usage, I am trying to use LTTng 2.0.1 and LTTng-modules-2.0.2 for ARM(omapL138), with Linux kernel 2.6.33 wit RT-29 patch on it. > > > > > > After enabling all the kernel events I am facing the below errors. I loaded all the modules manually by insmod. > > > > > > > > > > > > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng create > > > newsessiom Session newsessiom created. > > > Traces will be written in > > > /home/root/lttng-traces/newsessiom-20110325-154922 > > > > > > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng enable-event > > > -a --kernel All kernel events are enabled in channel channel0 > > > > > > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng start > > > LTTng: Failure to write metadata to buffers (timeout) > > > Error: Starting kernel trace failed > > > > > > Please kindly help me on this. > > > > I think you should look into the lttng-sessiond --help : > > > > --consumerd32-path PATH Specify path for the 32-bit UST consumer daemon binary > > --consumerd32-libdir PATH Specify path for the 32-bit UST consumer daemon libraries > > --consumerd64-path PATH Specify path for the 64-bit UST consumer daemon binary > > --consumerd64-libdir PATH Specify path for the 64-bit UST consumer daemon libraries > > > > options. My guess is that lttng-sessiond is not able to find the consumerd binary files, maybe due to a rename or because they have been moved after install. > > > > One more thing that might help is to launch the lttng-sessiond with "-vvv" : it will provide verbose output and let us know where things fall apart. > > > > Thanks, > > > > Mathieu > > > > > > > > > > > > > Thanks in advance, > > > Pavan > > > > > > ________________________________ > > > SASKEN BUSINESS DISCLAIMER: This message may contain confidential, proprietary or legally privileged information. In case you are not the original intended Recipient of the message, you must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message and you are requested to delete it and inform the sender. Any views expressed in this message are those of the individual sender unless otherwise stated. Nothing contained in this message shall be construed as an offer or acceptance of any offer by Sasken Communication Technologies Limited ("Sasken") unless sent with that express intent and with due authority of Sasken. Sasken has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. > > > Read Disclaimer at http://www.sasken.com/extras/mail_disclaimer.html > > > > > _______________________________________________ > > > lttng-dev mailing list > > > lttng-dev at lists.lttng.org > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > > > > > -- > > Mathieu Desnoyers > > Operating System Efficiency R&D Consultant EfficiOS Inc. > > http://www.efficios.com > > -- > Mathieu Desnoyers > Operating System Efficiency R&D Consultant EfficiOS Inc. > http://www.efficios.com -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mburton at ciena.com Mon Jun 11 11:08:15 2012 From: mburton at ciena.com (Burton, Michael) Date: Mon, 11 Jun 2012 11:08:15 -0400 Subject: [lttng-dev] UST Hanging at Tracepoint In-Reply-To: <20120609013854.GB10978@Krystal> References: <3F56B905CF2083408E8482044F20428AEF604D22@ONWVEXCHMB01.ciena.com> <3F56B905CF2083408E8482044F20428AEF6A2DBB@ONWVEXCHMB01.ciena.com> <20120609013854.GB10978@Krystal> Message-ID: <3F56B905CF2083408E8482044F20428AEF6A3108@ONWVEXCHMB01.ciena.com> Here is the output from UST debug: liblttng_ust_tracepoint[1022/1022]: just registered a tracepoints section from 0xb76f9e04 and having 1 tracepoints (in tracepoint_register_lib() at tracepoint.c:639) liblttng_ust_tracepoint[1022/1022]: registered tracepoint: lttng_ust:metadata (in tracepoint_register_lib() at tracepoint.c:644) libust[1022/1022]: LTT : ltt ring buffer client init (in ltt_ring_buffer_metadata_client_init() at ltt-ring-buffer-metadata-client.h:334) libust[1022/1022]: LTT : ltt ring buffer client init (in ltt_ring_buffer_client_overwrite_init() at ltt-ring-buffer-client.h:584) libust[1022/1022]: LTT : ltt ring buffer client init (in ltt_ring_buffer_client_discard_init() at ltt-ring-buffer-client.h:584) libust[1022/1024]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[1022/1024]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[1022/1024]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[1022/1024]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[1022/1024]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[1022/1024]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[1022/1024]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[1022/1024]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[1022/1024]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[1022/1024]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[1022/1023]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) -----Original Message----- From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] Sent: Friday, June 08, 2012 9:39 PM To: Burton, Michael Cc: lttng-dev at lists.lttng.org Subject: Re: [lttng-dev] UST Hanging at Tracepoint * Burton, Michael (mburton at ciena.com) wrote: > For further insight, I've attached the strace from the hanging > program. I also upgraded LTTng-UST to 2.0.3 and userspace-rcu to > 0.7.3. Haven't had time to look at it yet, but could you also provide the output of: LTTNG_UST_DEBUG=1 youprogname (starting your UST instrumented program with LTTNG_UST_DEBUG=1 env. var. set) Thanks! Mathieu > > Michael > > From: Burton, Michael [mailto:mburton at ciena.com] > Sent: Thursday, June 07, 2012 2:16 PM > To: lttng-dev at lists.lttng.org > Subject: [lttng-dev] UST Hanging at Tracepoint > > I am working on getting LTTng 2.0 working in our 2.6.35 kernel. I have kernel tracing working but I'm having problems with UST. > > I have a program with a dynamic UST tracepoint. When I run the program with all UST events enabled (lttng enable-event -a -u) the program hangs on the tracepoint. The last line from the UST debug is: > > libust[6184/6185]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) > > Any insight into why this is happening? I've attached the UST and lttng-sessiond debug generated by the program. > > I am running the following: > lttng-modules-2.6.32 (found through this mailing list) > lttng-tools-2.0.1 > lttng-ust-2.0.2 > userspace-rcu-0.7.0 > > Thanks, > Michael Content-Description: strace_debug.txt > execve("/ciena/bin/idp", ["idp", "help"], [/* 11 vars */]) = 0 > brk(0) = 0x804e000 > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb783c000 > access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) > open("/etc/ld.so.cache", O_RDONLY) = 3 > fstat64(3, {st_mode=S_IFREG|0644, st_size=17420, ...}) = 0 > mmap2(NULL, 17420, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7837000 > close(3) = 0 > open("/usr/lib/liblttng-ust.so.0", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220A\0\0004\0\0\0$"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=209876, ...}) = 0 > mmap2(NULL, 212940, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7803000 > mmap2(0xb7832000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2e) = 0xb7832000 > close(3) = 0 > open("/usr/lib/liblttng-ust-ctl.so.0", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\200-\0\0004\0\0\0\274"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=140020, ...}) = 0 > mmap2(NULL, 142828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77e0000 > mmap2(0xb7802000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x21) = 0xb7802000 > close(3) = 0 > open("/usr/lib/liblttng-ust-fork.so.0", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\0\5\0\0004\0\0\0X"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=3864, ...}) = 0 > mmap2(NULL, 6820, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77de000 > mmap2(0xb77df000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77df000 > close(3) = 0 > open("/usr/lib/liblttng-ust-tracepoint.so.0", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220\v\0\0004\0\0\0008"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=16632, ...}) = 0 > mmap2(NULL, 19944, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d9000 > mmap2(0xb77dd000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3) = 0xb77dd000 > close(3) = 0 > open("/usr/lib/liblttng-ust-libc-wrapper.so.0", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\t\0\0004\0\0\0\364"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=9300, ...}) = 0 > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb77d8000 > mmap2(NULL, 12104, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d5000 > mmap2(0xb77d7000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77d7000 > close(3) = 0 > open("/usr/lib/liburcu.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d0000 > mmap2(0xb77d4000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77d4000 > close(3) = 0 > open("/usr/lib/liburcu-common.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340\5\0\0004\0\0\0\34"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=5084, ...}) = 0 > mmap2(NULL, 8032, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77ce000 > mmap2(0xb77cf000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77cf000 > close(3) = 0 > open("/usr/lib/liburcu-qsbr.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\22\0\0004\0\0\0\374"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77c9000 > mmap2(0xb77cd000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77cd000 > close(3) = 0 > open("/usr/lib/liburcu-mb.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77c4000 > mmap2(0xb77c8000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c8000 > close(3) = 0 > open("/usr/lib/liburcu-signal.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\23\0\0004\0\0\0\10"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=18712, ...}) = 0 > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb77c3000 > mmap2(NULL, 21704, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77bd000 > mmap2(0xb77c2000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c2000 > close(3) = 0 > open("/usr/lib/liburcu-cds.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\300\f\0\0004\0\0\0t"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=26204, ...}) = 0 > mmap2(NULL, 25004, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77b6000 > mmap2(0xb77bc000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb77bc000 > close(3) = 0 > open("/usr/lib/liburcu-bp.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320\24\0\0004\0\0\0\360"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=19968, ...}) = 0 > mmap2(NULL, 23128, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77b0000 > mmap2(0xb77b5000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77b5000 > close(3) = 0 > open("/lib/libdl.so.2", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0 \n\0\0004\0\0\0<"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=9668, ...}) = 0 > mmap2(NULL, 12408, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77ac000 > mmap2(0xb77ae000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77ae000 > close(3) = 0 > open("/lib/libuuid.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0@\r\0\0004\0\0\0\350"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=12872, ...}) = 0 > mmap2(NULL, 15632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77a8000 > mmap2(0xb77ab000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2) = 0xb77ab000 > close(3) = 0 > open("/lib/libc.so.6", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220n\1\0004\0\0\0L"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=1347780, ...}) = 0 > mmap2(NULL, 1358120, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb765c000 > mprotect(0xb77a1000, 4096, PROT_NONE) = 0 > mmap2(0xb77a2000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x145) = 0xb77a2000 > mmap2(0xb77a5000, 10536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb77a5000 > close(3) = 0 > open("/lib/librt.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0p\30\0\0004\0\0\0\230"..., 512) = 512 > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb765b000 > fstat64(3, {st_mode=S_IFREG|0755, st_size=30616, ...}) = 0 > mmap2(NULL, 33364, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7652000 > mmap2(0xb7659000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb7659000 > close(3) = 0 > open("/lib/libpthread.so.0", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360J\0\0004\0\0\0\354"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=88420, ...}) = 0 > mmap2(NULL, 98828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7639000 > mmap2(0xb764e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14) = 0xb764e000 > mmap2(0xb7650000, 4620, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7650000 > close(3) = 0 > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7638000 > mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7636000 > set_thread_area({entry_number:-1 -> 6, base_addr:0xb7636d80, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 > mprotect(0xb764e000, 4096, PROT_READ) = 0 > mprotect(0xb7659000, 4096, PROT_READ) = 0 > mprotect(0xb77a2000, 8192, PROT_READ) = 0 > mprotect(0xb77ae000, 4096, PROT_READ) = 0 > mprotect(0xb785a000, 4096, PROT_READ) = 0 > munmap(0xb7837000, 17420) = 0 > set_tid_address(0xb7636de8) = 3050 > set_robust_list(0xb7636df0, 0xc) = 0 > futex(0xbfaaf560, FUTEX_WAKE_PRIVATE, 1) = 0 > futex(0xbfaaf560, 0x189 /* FUTEX_??? */, 1, NULL, bfaaf570) = -1 EAGAIN (Resource temporarily unavailable) > rt_sigaction(SIGRTMIN, {0xb763d4f0, [], SA_SIGINFO}, NULL, 8) = 0 > rt_sigaction(SIGRT_1, {0xb763d9c0, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0 > rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0 > getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0 > uname({sys="Linux", node="5410", ...}) = 0 > rt_sigaction(SIGUSR1, {0xb77bec0a, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0 > futex(0xb77af06c, FUTEX_WAKE_PRIVATE, 2147483647) = 0 > brk(0) = 0x804e000 > brk(0x806f000) = 0x806f000 > gettimeofday({1339187216, 880428}, NULL) = 0 > open("/dev/urandom", O_RDONLY|O_LARGEFILE) = 3 > fcntl64(3, F_GETFD) = 0 > fcntl64(3, F_SETFD, FD_CLOEXEC) = 0 > getuid32() = 0 > gettimeofday({1339187216, 889603}, NULL) = 0 > gettimeofday({1339187216, 894471}, NULL) = 0 > read(3, "\275\0051F\215\373\371\202\377kfb/\362\303N"..., 16) = 16 > clock_gettime(CLOCK_REALTIME, {1339187216, 902329405}) = 0 > getuid32() = 0 > geteuid32() = 0 > rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0 > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6e36000 > mprotect(0xb6e36000, 4096, PROT_NONE) = 0 > clone(child_stack=0xb7634d64, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0xb7635bd8, {entry_number:6, base_addr:0xb7635b70, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xb7635bd8) = 3051 > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6636000 > mprotect(0xb6636000, 4096, PROT_NONE) = 0 > clone(child_stack=0xb6e34d64, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0xb6e35bd8, {entry_number:6, base_addr:0xb6e35b70, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xb6e35bd8) = 3052 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > gettimeofday({1339187216, 974869}, NULL) = 0 > futex(0xb7836e30, FUTEX_WAIT_PRIVATE, 0, {2, 927460405}) = -1 ETIMEDOUT (Connection timed out) > gettid() = 3050 > write(2, "libust[3050/3050]: Error: Timed o"..., 107libust[3050/3050]: Error: Timed out waiting for ltt-sessiond (in lttng_ust_init() at lttng-ust-comm.c:912) > ) = 107 > futex(0xb7836e80, FUTEX_WAIT_PRIVATE, 2, NULL > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From dgoulet at efficios.com Mon Jun 11 14:28:03 2012 From: dgoulet at efficios.com (David Goulet) Date: Mon, 11 Jun 2012 14:28:03 -0400 Subject: [lttng-dev] [Fix:lttng-tools-PATCH] Fix:Change the type of enabled in lttng_event to a signed int In-Reply-To: <1339165148-6653-1-git-send-email-danny.serres@efficios.com> References: <1339165148-6653-1-git-send-email-danny.serres@efficios.com> Message-ID: <4FD638B3.6040200@efficios.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hey, Merged upstream! I'm still considering if I "can" put it in the stable-2.0 branch since this basically change the lttng.h ABI ... Thanks! David On 08/06/12 10:19 AM, Danny Serres wrote: > Signed-off-by: Danny Serres --- > include/lttng/lttng.h | 2 +- 1 file changed, 1 insertion(+), 1 > deletion(-) > > diff --git a/include/lttng/lttng.h b/include/lttng/lttng.h index > 3b1be61..c80e282 100644 --- a/include/lttng/lttng.h +++ > b/include/lttng/lttng.h @@ -214,7 +214,7 @@ struct lttng_event { enum > lttng_loglevel_type loglevel_type; int loglevel; > > - uint32_t enabled; + int32_t enabled; /* Does not apply: -1 */ pid_t pid; > > char padding[LTTNG_EVENT_PADDING1]; -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBAgAGBQJP1jiwAAoJEELoaioR9I02DmgIAI5JS3wQCwoBOIxKJz4GkA2+ QC7AEtCRKfG6AGPi3MN7EGav+hCTHAxfGK3CWr3CDi/Vo7uE3ofZ6wI3HgTdF1aH rEZPho96hhvsdu5p33HGg1BHuKhteYKa/TukzURBRPlu0+ruDol0rFsJn/SXblTe WczjFTRku+EAmF55V+gXOwhZ0wazZ47lcKIIg5EZMuSQo5PsRh/OD+w5j+15+2QP nHCFzEWEevnKmfmlhSfi0voyvNlxLzmc9ppv0AWUYX5mbppL6aCjHVu+RUphAAL4 R2D+C6ApRQOWnn+pFj3SALxas3sWuDHDCDCXSesjCWYT5wRoM6wY/ykgETC3Q+U= =sPTt -----END PGP SIGNATURE----- From mathieu.desnoyers at efficios.com Mon Jun 11 14:40:27 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Mon, 11 Jun 2012 14:40:27 -0400 Subject: [lttng-dev] [Fix:lttng-tools-PATCH] Fix:Change the type of enabled in lttng_event to a signed int In-Reply-To: <4FD638B3.6040200@efficios.com> References: <1339165148-6653-1-git-send-email-danny.serres@efficios.com> <4FD638B3.6040200@efficios.com> Message-ID: <20120611184027.GB22987@Krystal> * David Goulet (dgoulet at efficios.com) wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hey, > > Merged upstream! > > I'm still considering if I "can" put it in the stable-2.0 branch since this > basically change the lttng.h ABI ... The "enabled" field is almost never used, and having "-1" in a uint32_t is not really nice, and it is shown to the API user in the list tracepoints API. So I think it needs fixing in stable-2.0, and it should not cause any problem, just maybe fix some odd pretty-printing issue of "-1" some might have. Thanks, Mathieu > > Thanks! > David > > On 08/06/12 10:19 AM, Danny Serres wrote: > > Signed-off-by: Danny Serres --- > > include/lttng/lttng.h | 2 +- 1 file changed, 1 insertion(+), 1 > > deletion(-) > > > > diff --git a/include/lttng/lttng.h b/include/lttng/lttng.h index > > 3b1be61..c80e282 100644 --- a/include/lttng/lttng.h +++ > > b/include/lttng/lttng.h @@ -214,7 +214,7 @@ struct lttng_event { enum > > lttng_loglevel_type loglevel_type; int loglevel; > > > > - uint32_t enabled; + int32_t enabled; /* Does not apply: -1 */ pid_t pid; > > > > char padding[LTTNG_EVENT_PADDING1]; > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.12 (GNU/Linux) > > iQEcBAEBAgAGBQJP1jiwAAoJEELoaioR9I02DmgIAI5JS3wQCwoBOIxKJz4GkA2+ > QC7AEtCRKfG6AGPi3MN7EGav+hCTHAxfGK3CWr3CDi/Vo7uE3ofZ6wI3HgTdF1aH > rEZPho96hhvsdu5p33HGg1BHuKhteYKa/TukzURBRPlu0+ruDol0rFsJn/SXblTe > WczjFTRku+EAmF55V+gXOwhZ0wazZ47lcKIIg5EZMuSQo5PsRh/OD+w5j+15+2QP > nHCFzEWEevnKmfmlhSfi0voyvNlxLzmc9ppv0AWUYX5mbppL6aCjHVu+RUphAAL4 > R2D+C6ApRQOWnn+pFj3SALxas3sWuDHDCDCXSesjCWYT5wRoM6wY/ykgETC3Q+U= > =sPTt > -----END PGP SIGNATURE----- > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mburton at ciena.com Mon Jun 11 22:48:38 2012 From: mburton at ciena.com (Burton, Michael) Date: Mon, 11 Jun 2012 22:48:38 -0400 Subject: [lttng-dev] UST Hanging at Tracepoint In-Reply-To: <20120609013854.GB10978@Krystal> References: <3F56B905CF2083408E8482044F20428AEF604D22@ONWVEXCHMB01.ciena.com> <3F56B905CF2083408E8482044F20428AEF6A2DBB@ONWVEXCHMB01.ciena.com>, <20120609013854.GB10978@Krystal> Message-ID: <3F56B905CF2083408E8482044F20428AEF78BC79@ONWVEXCHMB01.ciena.com> Mathieu, I figured out the problem. When we start sessiond on our system as root, the /.lttng/ folder is not created, along with its contents. UST timed out since there was no sock_path. I was able to get UST working once I got sessiond running. Thanks. Michael ________________________________________ From: Mathieu Desnoyers [mathieu.desnoyers at efficios.com] Sent: 08 June 2012 21:38 To: Burton, Michael Cc: lttng-dev at lists.lttng.org Subject: Re: [lttng-dev] UST Hanging at Tracepoint * Burton, Michael (mburton at ciena.com) wrote: > For further insight, I've attached the strace from the hanging > program. I also upgraded LTTng-UST to 2.0.3 and userspace-rcu to > 0.7.3. Haven't had time to look at it yet, but could you also provide the output of: LTTNG_UST_DEBUG=1 youprogname (starting your UST instrumented program with LTTNG_UST_DEBUG=1 env. var. set) Thanks! Mathieu > > Michael > > From: Burton, Michael [mailto:mburton at ciena.com] > Sent: Thursday, June 07, 2012 2:16 PM > To: lttng-dev at lists.lttng.org > Subject: [lttng-dev] UST Hanging at Tracepoint > > I am working on getting LTTng 2.0 working in our 2.6.35 kernel. I have kernel tracing working but I'm having problems with UST. > > I have a program with a dynamic UST tracepoint. When I run the program with all UST events enabled (lttng enable-event -a -u) the program hangs on the tracepoint. The last line from the UST debug is: > > libust[6184/6185]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) > > Any insight into why this is happening? I've attached the UST and lttng-sessiond debug generated by the program. > > I am running the following: > lttng-modules-2.6.32 (found through this mailing list) > lttng-tools-2.0.1 > lttng-ust-2.0.2 > userspace-rcu-0.7.0 > > Thanks, > Michael Content-Description: strace_debug.txt > execve("/ciena/bin/idp", ["idp", "help"], [/* 11 vars */]) = 0 > brk(0) = 0x804e000 > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb783c000 > access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) > open("/etc/ld.so.cache", O_RDONLY) = 3 > fstat64(3, {st_mode=S_IFREG|0644, st_size=17420, ...}) = 0 > mmap2(NULL, 17420, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7837000 > close(3) = 0 > open("/usr/lib/liblttng-ust.so.0", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220A\0\0004\0\0\0$"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=209876, ...}) = 0 > mmap2(NULL, 212940, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7803000 > mmap2(0xb7832000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2e) = 0xb7832000 > close(3) = 0 > open("/usr/lib/liblttng-ust-ctl.so.0", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\200-\0\0004\0\0\0\274"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=140020, ...}) = 0 > mmap2(NULL, 142828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77e0000 > mmap2(0xb7802000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x21) = 0xb7802000 > close(3) = 0 > open("/usr/lib/liblttng-ust-fork.so.0", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\0\5\0\0004\0\0\0X"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=3864, ...}) = 0 > mmap2(NULL, 6820, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77de000 > mmap2(0xb77df000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77df000 > close(3) = 0 > open("/usr/lib/liblttng-ust-tracepoint.so.0", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220\v\0\0004\0\0\0008"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=16632, ...}) = 0 > mmap2(NULL, 19944, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d9000 > mmap2(0xb77dd000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3) = 0xb77dd000 > close(3) = 0 > open("/usr/lib/liblttng-ust-libc-wrapper.so.0", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\t\0\0004\0\0\0\364"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=9300, ...}) = 0 > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb77d8000 > mmap2(NULL, 12104, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d5000 > mmap2(0xb77d7000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77d7000 > close(3) = 0 > open("/usr/lib/liburcu.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d0000 > mmap2(0xb77d4000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77d4000 > close(3) = 0 > open("/usr/lib/liburcu-common.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340\5\0\0004\0\0\0\34"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=5084, ...}) = 0 > mmap2(NULL, 8032, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77ce000 > mmap2(0xb77cf000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77cf000 > close(3) = 0 > open("/usr/lib/liburcu-qsbr.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\22\0\0004\0\0\0\374"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77c9000 > mmap2(0xb77cd000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77cd000 > close(3) = 0 > open("/usr/lib/liburcu-mb.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77c4000 > mmap2(0xb77c8000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c8000 > close(3) = 0 > open("/usr/lib/liburcu-signal.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\23\0\0004\0\0\0\10"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=18712, ...}) = 0 > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb77c3000 > mmap2(NULL, 21704, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77bd000 > mmap2(0xb77c2000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c2000 > close(3) = 0 > open("/usr/lib/liburcu-cds.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\300\f\0\0004\0\0\0t"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=26204, ...}) = 0 > mmap2(NULL, 25004, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77b6000 > mmap2(0xb77bc000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb77bc000 > close(3) = 0 > open("/usr/lib/liburcu-bp.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320\24\0\0004\0\0\0\360"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=19968, ...}) = 0 > mmap2(NULL, 23128, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77b0000 > mmap2(0xb77b5000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77b5000 > close(3) = 0 > open("/lib/libdl.so.2", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0 \n\0\0004\0\0\0<"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=9668, ...}) = 0 > mmap2(NULL, 12408, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77ac000 > mmap2(0xb77ae000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77ae000 > close(3) = 0 > open("/lib/libuuid.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0@\r\0\0004\0\0\0\350"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=12872, ...}) = 0 > mmap2(NULL, 15632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77a8000 > mmap2(0xb77ab000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2) = 0xb77ab000 > close(3) = 0 > open("/lib/libc.so.6", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220n\1\0004\0\0\0L"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=1347780, ...}) = 0 > mmap2(NULL, 1358120, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb765c000 > mprotect(0xb77a1000, 4096, PROT_NONE) = 0 > mmap2(0xb77a2000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x145) = 0xb77a2000 > mmap2(0xb77a5000, 10536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb77a5000 > close(3) = 0 > open("/lib/librt.so.1", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0p\30\0\0004\0\0\0\230"..., 512) = 512 > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb765b000 > fstat64(3, {st_mode=S_IFREG|0755, st_size=30616, ...}) = 0 > mmap2(NULL, 33364, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7652000 > mmap2(0xb7659000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb7659000 > close(3) = 0 > open("/lib/libpthread.so.0", O_RDONLY) = 3 > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360J\0\0004\0\0\0\354"..., 512) = 512 > fstat64(3, {st_mode=S_IFREG|0755, st_size=88420, ...}) = 0 > mmap2(NULL, 98828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7639000 > mmap2(0xb764e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14) = 0xb764e000 > mmap2(0xb7650000, 4620, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7650000 > close(3) = 0 > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7638000 > mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7636000 > set_thread_area({entry_number:-1 -> 6, base_addr:0xb7636d80, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 > mprotect(0xb764e000, 4096, PROT_READ) = 0 > mprotect(0xb7659000, 4096, PROT_READ) = 0 > mprotect(0xb77a2000, 8192, PROT_READ) = 0 > mprotect(0xb77ae000, 4096, PROT_READ) = 0 > mprotect(0xb785a000, 4096, PROT_READ) = 0 > munmap(0xb7837000, 17420) = 0 > set_tid_address(0xb7636de8) = 3050 > set_robust_list(0xb7636df0, 0xc) = 0 > futex(0xbfaaf560, FUTEX_WAKE_PRIVATE, 1) = 0 > futex(0xbfaaf560, 0x189 /* FUTEX_??? */, 1, NULL, bfaaf570) = -1 EAGAIN (Resource temporarily unavailable) > rt_sigaction(SIGRTMIN, {0xb763d4f0, [], SA_SIGINFO}, NULL, 8) = 0 > rt_sigaction(SIGRT_1, {0xb763d9c0, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0 > rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0 > getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0 > uname({sys="Linux", node="5410", ...}) = 0 > rt_sigaction(SIGUSR1, {0xb77bec0a, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0 > futex(0xb77af06c, FUTEX_WAKE_PRIVATE, 2147483647) = 0 > brk(0) = 0x804e000 > brk(0x806f000) = 0x806f000 > gettimeofday({1339187216, 880428}, NULL) = 0 > open("/dev/urandom", O_RDONLY|O_LARGEFILE) = 3 > fcntl64(3, F_GETFD) = 0 > fcntl64(3, F_SETFD, FD_CLOEXEC) = 0 > getuid32() = 0 > gettimeofday({1339187216, 889603}, NULL) = 0 > gettimeofday({1339187216, 894471}, NULL) = 0 > read(3, "\275\0051F\215\373\371\202\377kfb/\362\303N"..., 16) = 16 > clock_gettime(CLOCK_REALTIME, {1339187216, 902329405}) = 0 > getuid32() = 0 > geteuid32() = 0 > rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0 > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6e36000 > mprotect(0xb6e36000, 4096, PROT_NONE) = 0 > clone(child_stack=0xb7634d64, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0xb7635bd8, {entry_number:6, base_addr:0xb7635b70, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xb7635bd8) = 3051 > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6636000 > mprotect(0xb6636000, 4096, PROT_NONE) = 0 > clone(child_stack=0xb6e34d64, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0xb6e35bd8, {entry_number:6, base_addr:0xb6e35b70, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xb6e35bd8) = 3052 > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > gettimeofday({1339187216, 974869}, NULL) = 0 > futex(0xb7836e30, FUTEX_WAIT_PRIVATE, 0, {2, 927460405}) = -1 ETIMEDOUT (Connection timed out) > gettid() = 3050 > write(2, "libust[3050/3050]: Error: Timed o"..., 107libust[3050/3050]: Error: Timed out waiting for ltt-sessiond (in lttng_ust_init() at lttng-ust-comm.c:912) > ) = 107 > futex(0xb7836e80, FUTEX_WAIT_PRIVATE, 2, NULL > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From yanglei.fage at gmail.com Tue Jun 12 02:16:05 2012 From: yanglei.fage at gmail.com (lei yang) Date: Tue, 12 Jun 2012 14:16:05 +0800 Subject: [lttng-dev] git tree dead? Message-ID: Hi list, lyang0 at lyang0-OptiPlex-755:~/git$ git clone git://git.lttng.org/linux-2.6-lttng.git ???? 'linux-2.6-lttng'... fatal: The remote end hung up unexpectedly Lei From pavan.anumula at sasken.com Tue Jun 12 07:24:52 2012 From: pavan.anumula at sasken.com (Pavan Anumula) Date: Tue, 12 Jun 2012 16:54:52 +0530 Subject: [lttng-dev] Lttng start failed In-Reply-To: <20120611135422.GA19020@Krystal> References: <47D610AD9C485E458630BA38C324D3B60113FB00F9EA@EXGMBX01.sasken.com> <20120608174226.GA23339@Krystal> <47D610AD9C485E458630BA38C324D3B60113FB00FA00@EXGMBX01.sasken.com> <20120609123251.GA14280@Krystal> <47D610AD9C485E458630BA38C324D3B60113FB00FA1E@EXGMBX01.sasken.com>, <20120611135422.GA19020@Krystal> Message-ID: <47D610AD9C485E458630BA38C324D3B60113FB0E22D1@EXGMBX01.sasken.com> Hi Mathieu, Still I am unable to start tracing, And I am facing the same start errors. Please kindly help me on this. This time I loaded the modules by modprobe( not with insmod as I did before), Then I run the command " arm-none-linux-gnueabi-lttng-sessiond -vvv " (before arm-none-linux-gnueabi-lttng-sessiond -d), The below is the Output, And It got hanged overthere. Later when I started lttng start I am acing the same erros below: root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng start LTTng: Failure to write metadata to buffers (timeout) Error: Starting kernel trace failed ************************************************************************************************ DEBUG3: Creating LTTng run directory: /var/run/lttng [in create_lttng_rundir() at main.c:4315] DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in main() at main.c:4543] DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command [in main() at main.c:4545] DEBUG1: Client socket path /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond [in main() at main.c:4593] DEBUG1: LTTng run directory path: /var/run/lttng [in main() at main.c:4594] DEBUG2: UST consumer 32 bits err path: /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] DEBUG2: UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] DEBUG2: UST consumer 64 bits err path: /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] DEBUG2: UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] DEBUG3: Created hashtable size 4 at 0x4a080 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x4a168 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Creating consumer directory: /var/run/lttng/kconsumerd [in set_consumer_sockets() at main.c:4357] DEBUG1: Modprobe successfully lttng-tracer [in modprobe_lttng_control() at modprobe.c:163] DEBUG2: Kernel tracer version validated (major version 2) [in kernel_validate_version() at kernel.c:675] DEBUG1: Modprobe successfully lttng-ftrace [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-kprobes [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-kretprobes [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-lib-ring-buffer [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-ring-buffer-client-discard [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-ring-buffer-client-overwrite [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-ring-buffer-metadata-client [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-ring-buffer-client-mmap-discard [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-ring-buffer-client-mmap-overwrite [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-ring-buffer-metadata-mmap-client [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-lttng [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-types [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-block [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-irq [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-kvm [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-sched [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-signal [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-statedump [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-timer [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Kernel tracer fd 6 [in init_kernel_tracer() at main.c:1887] DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd64 [in set_consumer_sockets() at main.c:4357] DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() at main.c:4357] DEBUG1: Signal handler set for SIGTERM, SIGPIPE and SIGINT [in set_signal_handler() at main.c:4449] DEBUG1: All permissions are set [in set_permissions() at main.c:4250] DEBUG1: poll set max size set to 65535 [in compat_poll_set_max_size() at compat-poll.c:196] DEBUG1: [thread] Manage application started [in thread_manage_apps() at main.c:1179] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 13 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: Apps thread polling on 2 fds [in thread_manage_apps() at main.c:1200] DEBUG1: Thread manage kernel started [in thread_manage_kernel() at main.c:876] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 11 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: Updating kernel poll set [in update_kernel_poll() at main.c:748] DEBUG1: Thread kernel polling on 2 fds [in thread_manage_kernel() at main.c:905] DEBUG1: [thread] Manage application registration started [in thread_registration_apps() at main.c:1392] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 10 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: Notifying applications of session daemon state: 1 [in notify_ust_apps() at main.c:687] DEBUG1: [thread] Dispatch UST command started [in thread_dispatch_ust_registration() at main.c:1324] DEBUG1: Futex n to 1 prepare done [in futex_nto1_prepare() at futex.c:73] DEBUG1: Woken up but nothing in the UST command queue [in thread_dispatch_ust_registration() at main.c:1334] DEBUG1: [thread] Manage client started [in thread_manage_clients() at main.c:3794] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 9 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3826] DEBUG1: Got the wait shm fd 15 [in get_wait_shm() at shm.c:117] DEBUG1: Futex wait update active 1 [in futex_wait_update() at futex.c:62] DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1423] ******************************************************************************************************** Am I missing anything?? Thanks in Advance, Pavan ________________________________________ From: Mathieu Desnoyers [mathieu.desnoyers at efficios.com] Sent: Monday, June 11, 2012 7:24 PM To: Pavan Anumula Cc: lttng-dev at lists.lttng.org Subject: Re: [lttng-dev] Lttng start failed * Pavan Anumula (pavan.anumula at sasken.com) wrote: > Hi Mathieu, > > Please find the info below as per your comments > > > ########## Output of command "arm-none-linux-gnueabi-lttng-sessiond -vvv " , After loading the modules ####### > > DEBUG3: Creating LTTng run directory: /var/run/lttng [in create_lttng_rundir() at main.c:4315] > DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in main() at main.c:4543] > DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command [in main() at main.c:4545] > DEBUG1: Client socket path /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] > DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond [in main() at main.c:4593] > DEBUG1: LTTng run directory path: /var/run/lttng [in main() at main.c:4594] > DEBUG2: UST consumer 32 bits err path: /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] > DEBUG2: UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] > DEBUG2: UST consumer 64 bits err path: /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] > DEBUG2: UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] > DEBUG3: Created hashtable size 4 at 0x4a080 of type 1 [in lttng_ht_new() at hashtable.c:96] > DEBUG3: Created hashtable size 4 at 0x4a168 of type 1 [in lttng_ht_new() at hashtable.c:96] > DEBUG2: Creating consumer directory: /var/run/lttng/kconsumerd [in set_consumer_sockets() at main.c:4357] > FATAL: Module lttng_tracer not found. > Error: Unable to load module lttng-tracer Hrm. You want to do kernel tracing, but modprobe cannot find the lttng kernel tracer modules. You might want to run depmod -a or something like that on your target, and ensure that modprobe works properly. Thanks, Mathieu > DEBUG2: Kernel tracer version validated (major version 2) [in kernel_validate_version() at kernel.c:675] > DEBUG1: Modprobe successfully lttng-ftrace [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-kprobes [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-kretprobes [in modprobe_lttng_data() at modprobe.c:199] > FATAL: Module lttng_lib_ring_buffer not found. > Error: Unable to load module lttng-lib-ring-buffer > FATAL: Module lttng_ring_buffer_client_discard not found. > Error: Unable to load module lttng-ring-buffer-client-discard > FATAL: Module lttng_ring_buffer_client_overwrite not found. > Error: Unable to load module lttng-ring-buffer-client-overwrite > FATAL: Module lttng_ring_buffer_metadata_client not found. > Error: Unable to load module lttng-ring-buffer-metadata-client > FATAL: Module lttng_ring_buffer_client_mmap_discard not found. > Error: Unable to load module lttng-ring-buffer-client-mmap-discard > FATAL: Module lttng_ring_buffer_client_mmap_overwrite not found. > Error: Unable to load module lttng-ring-buffer-client-mmap-overwrite > FATAL: Module lttng_ring_buffer_metadata_mmap_client not found. > Error: Unable to load module lttng-ring-buffer-metadata-mmap-client > FATAL: Module lttng_probe_lttng not found. > Error: Unable to load module lttng-probe-lttng > DEBUG1: Modprobe successfully lttng-types [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-block [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-irq [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-kvm [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-sched [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-signal [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-statedump [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-timer [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Kernel tracer fd 6 [in init_kernel_tracer() at main.c:1887] > DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd64 [in set_consumer_sockets() at main.c:4357] > DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() at main.c:4357] > DEBUG1: Signal handler set for SIGTERM, SIGPIPE and SIGINT [in set_signal_handler() at main.c:4449] > DEBUG1: All permissions are set [in set_permissions() at main.c:4250] > DEBUG1: poll set max size set to 65535 [in compat_poll_set_max_size() at compat-poll.c:196] > DEBUG1: [thread] Manage client started [in thread_manage_clients() at main.c:3794] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: fd 9 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3826] > DEBUG1: [thread] Dispatch UST command started [in thread_dispatch_ust_registration() at main.c:1324] > DEBUG1: Futex n to 1 prepare done [in futex_nto1_prepare() at futex.c:73] > DEBUG1: Woken up but nothing in the UST command queue [in thread_dispatch_ust_registration() at main.c:1334] > DEBUG1: Thread manage kernel started [in thread_manage_kernel() at main.c:876] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: fd 11 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: Updating kernel poll set [in update_kernel_poll() at main.c:748] > DEBUG1: Thread kernel polling on 2 fds [in thread_manage_kernel() at main.c:905] > DEBUG1: [thread] Manage application started [in thread_manage_apps() at main.c:1179] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: fd 13 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: Apps thread polling on 2 fds [in thread_manage_apps() at main.c:1200] > DEBUG1: [thread] Manage application registration started [in thread_registration_apps() at main.c:1392] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: fd 10 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: Notifying applications of session daemon state: 1 [in notify_ust_apps() at main.c:687] > DEBUG1: Got the wait shm fd 15 [in get_wait_shm() at shm.c:117] > DEBUG1: Futex wait update active 1 [in futex_wait_update() at futex.c:62] > DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1423] > > > > #### Output of command "arm-none-linux-gnueabi-lttng-sessiond -vvv " before loading the modules ############# > > DEBUG3: Creating LTTng run directory: /var/run/lttng [in create_lttng_rundir() at main.c:4315] > DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in main() at main.c:4543] > DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command [in main() at main.c:4545] > DEBUG1: Client socket path /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] > DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond [in main() at main.c:4593] > DEBUG1: LTTng run directory path: /var/run/lttng [in main() at main.c:4594] > DEBUG2: UST consumer 32 bits err path: /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] > DEBUG2: UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] > DEBUG2: UST consumer 64 bits err path: /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] > DEBUG2: UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] > DEBUG3: Created hashtable size 4 at 0x4a080 of type 1 [in lttng_ht_new() at hashtable.c:96] > DEBUG3: Created hashtable size 4 at 0x4a168 of type 1 [in lttng_ht_new() at hashtable.c:96] > DEBUG2: Creating consumer directory: /var/run/lttng/kconsumerd [in set_consumer_sockets() at main.c:4357] > FATAL: Module lttng_tracer not found. > Error: Unable to load module lttng-tracer > DEBUG1: Failed to open /proc/lttng [in init_kernel_tracer() at main.c:1871] > Error: Unable to remove module lttng-tracer > Warning: No kernel tracer available > DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd64 [in set_consumer_sockets() at main.c:4357] > DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() at main.c:4357] > DEBUG1: Signal handler set for SIGTERM, SIGPIPE and SIGINT [in set_signal_handler() at main.c:4449] > DEBUG1: All permissions are set [in set_permissions() at main.c:4250] > DEBUG1: poll set max size set to 65535 [in compat_poll_set_max_size() at compat-poll.c:196] > DEBUG1: [thread] Manage application started [in thread_manage_apps() at main.c:1179] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: fd 12 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: Apps thread polling on 2 fds [in thread_manage_apps() at main.c:1200] > DEBUG1: Thread manage kernel started [in thread_manage_kernel() at main.c:876] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: fd 10 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: Updating kernel poll set [in update_kernel_poll() at main.c:748] > DEBUG1: Thread kernel polling on 2 fds [in thread_manage_kernel() at main.c:905] > DEBUG1: [thread] Manage application registration started [in thread_registration_apps() at main.c:1392] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: fd 9 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: Notifying applications of session daemon state: 1 [in notify_ust_apps() at main.c:687] > DEBUG1: [thread] Dispatch UST command started [in thread_dispatch_ust_registration() at main.c:1324] > DEBUG1: Futex n to 1 prepare done [in futex_nto1_prepare() at futex.c:73] > DEBUG1: Woken up but nothing in the UST command queue [in thread_dispatch_ust_registration() at main.c:1334] > DEBUG1: [thread] Manage client started [in thread_manage_clients() at main.c:3794] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: fd 8 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3826] > DEBUG1: Got the wait shm fd 14 [in get_wait_shm() at shm.c:117] > DEBUG1: Futex wait update active 1 [in futex_wait_update() at futex.c:62] > DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1423] > > > Regards, > Pavan > > -----Original Message----- > From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] > Sent: Saturday, June 09, 2012 6:03 PM > To: Pavan Anumula > Cc: lttng-dev at lists.lttng.org > Subject: Re: [lttng-dev] Lttng start failed > > * Pavan Anumula (pavan.anumula at sasken.com) wrote: > > Hi Mathue, > > > > Thanks for the quick reply, > > > > After inserting lttng modules , I had given the command "arm-none-linux-gnueabi-lttng-sessiond -vvv" as you said, Below is the output where there are error messages. Please help me in resolving the issue, SO that I can catch kernel and user traces. > > > > > > root at arago:/usr/lttng/modules# arm-none-linux-gnueabi-lttng-sessiond > > -d root at arago:/usr/lttng/modules# > > arm-none-linux-gnueabi-lttng-sessiond -vvv > > DEBUG3: Creating LTTng run directory: /var/run/lttng [in > > create_lttng_rundir() at main.c:4315] > > DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in > > main() at main.c:4543] > > DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command > > [in main() at main.c:4545] > > DEBUG1: Client socket path /var/run/lttng/client-lttng-sessiond [in > > main() at main.c:4592] > > DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond [in > > main() at main.c:4593] > > DEBUG1: LTTng run directory path: /var/run/lttng [in main() at > > main.c:4594] > > DEBUG2: UST consumer 32 bits err path: > > /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] > > DEBUG2: UST consumer 32 bits cmd path: > > /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] > > DEBUG2: UST consumer 64 bits err path: > > /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] > > DEBUG2: UST consumer 64 bits cmd path: > > /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] > > Error: Already running daemon. > > Please kill the lttng-sessiond that is already started (the one with -d). Instead of that one, run the sessiond with: > > lttng-sessiond -vvv > > (don't run lttng-sessiond -d before) > > Thanks, > > Mathieu > > > > > > > > > > > Thanks in advance, > > Pavan > > > > -----Original Message----- > > From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] > > Sent: Friday, June 08, 2012 11:12 PM > > To: Pavan Anumula > > Cc: lttng-dev at lists.lttng.org > > Subject: Re: [lttng-dev] Lttng start failed > > > > * Pavan Anumula (pavan.anumula at sasken.com) wrote: > > > Hi , > > > > > > I am new to LTTng usage, I am trying to use LTTng 2.0.1 and LTTng-modules-2.0.2 for ARM(omapL138), with Linux kernel 2.6.33 wit RT-29 patch on it. > > > > > > After enabling all the kernel events I am facing the below errors. I loaded all the modules manually by insmod. > > > > > > > > > > > > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng create > > > newsessiom Session newsessiom created. > > > Traces will be written in > > > /home/root/lttng-traces/newsessiom-20110325-154922 > > > > > > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng enable-event > > > -a --kernel All kernel events are enabled in channel channel0 > > > > > > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng start > > > LTTng: Failure to write metadata to buffers (timeout) > > > Error: Starting kernel trace failed > > > > > > Please kindly help me on this. > > > > I think you should look into the lttng-sessiond --help : > > > > --consumerd32-path PATH Specify path for the 32-bit UST consumer daemon binary > > --consumerd32-libdir PATH Specify path for the 32-bit UST consumer daemon libraries > > --consumerd64-path PATH Specify path for the 64-bit UST consumer daemon binary > > --consumerd64-libdir PATH Specify path for the 64-bit UST consumer daemon libraries > > > > options. My guess is that lttng-sessiond is not able to find the consumerd binary files, maybe due to a rename or because they have been moved after install. > > > > One more thing that might help is to launch the lttng-sessiond with "-vvv" : it will provide verbose output and let us know where things fall apart. > > > > Thanks, > > > > Mathieu > > > > > > > > > > > > > Thanks in advance, > > > Pavan > > > > > > ________________________________ > > > SASKEN BUSINESS DISCLAIMER: This message may contain confidential, proprietary or legally privileged information. In case you are not the original intended Recipient of the message, you must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message and you are requested to delete it and inform the sender. Any views expressed in this message are those of the individual sender unless otherwise stated. Nothing contained in this message shall be construed as an offer or acceptance of any offer by Sasken Communication Technologies Limited ("Sasken") unless sent with that express intent and with due authority of Sasken. Sasken has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. > > > Read Disclaimer at http://www.sasken.com/extras/mail_disclaimer.html > > > > > _______________________________________________ > > > lttng-dev mailing list > > > lttng-dev at lists.lttng.org > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > > > > > -- > > Mathieu Desnoyers > > Operating System Efficiency R&D Consultant EfficiOS Inc. > > http://www.efficios.com > > -- > Mathieu Desnoyers > Operating System Efficiency R&D Consultant EfficiOS Inc. > http://www.efficios.com -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From pavan.anumula at sasken.com Tue Jun 12 07:25:32 2012 From: pavan.anumula at sasken.com (Pavan Anumula) Date: Tue, 12 Jun 2012 16:55:32 +0530 Subject: [lttng-dev] Lttng start failed In-Reply-To: <20120611135422.GA19020@Krystal> References: <47D610AD9C485E458630BA38C324D3B60113FB00F9EA@EXGMBX01.sasken.com> <20120608174226.GA23339@Krystal> <47D610AD9C485E458630BA38C324D3B60113FB00FA00@EXGMBX01.sasken.com> <20120609123251.GA14280@Krystal> <47D610AD9C485E458630BA38C324D3B60113FB00FA1E@EXGMBX01.sasken.com> <20120611135422.GA19020@Krystal> Message-ID: <47D610AD9C485E458630BA38C324D3B60113FB00FA8F@EXGMBX01.sasken.com> Hi Mathieu, Still I am unable to start tracing, And I am facing the same start errors. Please kindly help me on this. This time I loaded the modules by modprobe( not with insmod as I did before), Then I run the command " arm-none-linux-gnueabi-lttng-sessiond -vvv " (before arm-none-linux-gnueabi-lttng-sessiond -d), The below is the Output, And It got hanged overthere. Later when I started lttng start I am acing the same erros below: root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng start LTTng: Failure to write metadata to buffers (timeout) Error: Starting kernel trace failed ************************************************************************************************ DEBUG3: Creating LTTng run directory: /var/run/lttng [in create_lttng_rundir() at main.c:4315] DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in main() at main.c:4543] DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command [in main() at main.c:4545] DEBUG1: Client socket path /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond [in main() at main.c:4593] DEBUG1: LTTng run directory path: /var/run/lttng [in main() at main.c:4594] DEBUG2: UST consumer 32 bits err path: /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] DEBUG2: UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] DEBUG2: UST consumer 64 bits err path: /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] DEBUG2: UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] DEBUG3: Created hashtable size 4 at 0x4a080 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x4a168 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Creating consumer directory: /var/run/lttng/kconsumerd [in set_consumer_sockets() at main.c:4357] DEBUG1: Modprobe successfully lttng-tracer [in modprobe_lttng_control() at modprobe.c:163] DEBUG2: Kernel tracer version validated (major version 2) [in kernel_validate_version() at kernel.c:675] DEBUG1: Modprobe successfully lttng-ftrace [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-kprobes [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-kretprobes [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-lib-ring-buffer [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-ring-buffer-client-discard [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-ring-buffer-client-overwrite [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-ring-buffer-metadata-client [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-ring-buffer-client-mmap-discard [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-ring-buffer-client-mmap-overwrite [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-ring-buffer-metadata-mmap-client [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-lttng [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-types [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-block [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-irq [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-kvm [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-sched [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-signal [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-statedump [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-timer [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Kernel tracer fd 6 [in init_kernel_tracer() at main.c:1887] DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd64 [in set_consumer_sockets() at main.c:4357] DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() at main.c:4357] DEBUG1: Signal handler set for SIGTERM, SIGPIPE and SIGINT [in set_signal_handler() at main.c:4449] DEBUG1: All permissions are set [in set_permissions() at main.c:4250] DEBUG1: poll set max size set to 65535 [in compat_poll_set_max_size() at compat-poll.c:196] DEBUG1: [thread] Manage application started [in thread_manage_apps() at main.c:1179] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 13 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: Apps thread polling on 2 fds [in thread_manage_apps() at main.c:1200] DEBUG1: Thread manage kernel started [in thread_manage_kernel() at main.c:876] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 11 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: Updating kernel poll set [in update_kernel_poll() at main.c:748] DEBUG1: Thread kernel polling on 2 fds [in thread_manage_kernel() at main.c:905] DEBUG1: [thread] Manage application registration started [in thread_registration_apps() at main.c:1392] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 10 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: Notifying applications of session daemon state: 1 [in notify_ust_apps() at main.c:687] DEBUG1: [thread] Dispatch UST command started [in thread_dispatch_ust_registration() at main.c:1324] DEBUG1: Futex n to 1 prepare done [in futex_nto1_prepare() at futex.c:73] DEBUG1: Woken up but nothing in the UST command queue [in thread_dispatch_ust_registration() at main.c:1334] DEBUG1: [thread] Manage client started [in thread_manage_clients() at main.c:3794] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 9 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3826] DEBUG1: Got the wait shm fd 15 [in get_wait_shm() at shm.c:117] DEBUG1: Futex wait update active 1 [in futex_wait_update() at futex.c:62] DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1423] ******************************************************************************************************** Am I missing anything?? Thanks in Advance, Pavan -----Original Message----- From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] Sent: Monday, June 11, 2012 7:24 PM To: Pavan Anumula Cc: lttng-dev at lists.lttng.org Subject: Re: [lttng-dev] Lttng start failed * Pavan Anumula (pavan.anumula at sasken.com) wrote: > Hi Mathieu, > > Please find the info below as per your comments > > > ########## Output of command "arm-none-linux-gnueabi-lttng-sessiond > -vvv " , After loading the modules ####### > > DEBUG3: Creating LTTng run directory: /var/run/lttng [in > create_lttng_rundir() at main.c:4315] > DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in > main() at main.c:4543] > DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command > [in main() at main.c:4545] > DEBUG1: Client socket path /var/run/lttng/client-lttng-sessiond [in > main() at main.c:4592] > DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond [in > main() at main.c:4593] > DEBUG1: LTTng run directory path: /var/run/lttng [in main() at > main.c:4594] > DEBUG2: UST consumer 32 bits err path: > /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] > DEBUG2: UST consumer 32 bits cmd path: > /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] > DEBUG2: UST consumer 64 bits err path: > /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] > DEBUG2: UST consumer 64 bits cmd path: > /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] > DEBUG3: Created hashtable size 4 at 0x4a080 of type 1 [in > lttng_ht_new() at hashtable.c:96] > DEBUG3: Created hashtable size 4 at 0x4a168 of type 1 [in > lttng_ht_new() at hashtable.c:96] > DEBUG2: Creating consumer directory: /var/run/lttng/kconsumerd [in > set_consumer_sockets() at main.c:4357] > FATAL: Module lttng_tracer not found. > Error: Unable to load module lttng-tracer Hrm. You want to do kernel tracing, but modprobe cannot find the lttng kernel tracer modules. You might want to run depmod -a or something like that on your target, and ensure that modprobe works properly. Thanks, Mathieu > DEBUG2: Kernel tracer version validated (major version 2) [in > kernel_validate_version() at kernel.c:675] > DEBUG1: Modprobe successfully lttng-ftrace [in modprobe_lttng_data() > at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-kprobes [in modprobe_lttng_data() > at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-kretprobes [in > modprobe_lttng_data() at modprobe.c:199] > FATAL: Module lttng_lib_ring_buffer not found. > Error: Unable to load module lttng-lib-ring-buffer > FATAL: Module lttng_ring_buffer_client_discard not found. > Error: Unable to load module lttng-ring-buffer-client-discard > FATAL: Module lttng_ring_buffer_client_overwrite not found. > Error: Unable to load module lttng-ring-buffer-client-overwrite > FATAL: Module lttng_ring_buffer_metadata_client not found. > Error: Unable to load module lttng-ring-buffer-metadata-client > FATAL: Module lttng_ring_buffer_client_mmap_discard not found. > Error: Unable to load module lttng-ring-buffer-client-mmap-discard > FATAL: Module lttng_ring_buffer_client_mmap_overwrite not found. > Error: Unable to load module lttng-ring-buffer-client-mmap-overwrite > FATAL: Module lttng_ring_buffer_metadata_mmap_client not found. > Error: Unable to load module lttng-ring-buffer-metadata-mmap-client > FATAL: Module lttng_probe_lttng not found. > Error: Unable to load module lttng-probe-lttng > DEBUG1: Modprobe successfully lttng-types [in modprobe_lttng_data() at > modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-block [in > modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-irq [in > modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-kvm [in > modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-sched [in > modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-signal [in > modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-statedump [in > modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-timer [in > modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Kernel tracer fd 6 [in init_kernel_tracer() at main.c:1887] > DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd64 [in > set_consumer_sockets() at main.c:4357] > DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd32 [in > set_consumer_sockets() at main.c:4357] > DEBUG1: Signal handler set for SIGTERM, SIGPIPE and SIGINT [in > set_signal_handler() at main.c:4449] > DEBUG1: All permissions are set [in set_permissions() at main.c:4250] > DEBUG1: poll set max size set to 65535 [in compat_poll_set_max_size() > at compat-poll.c:196] > DEBUG1: [thread] Manage client started [in thread_manage_clients() at > main.c:3794] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at > compat-poll.c:91] > DEBUG1: fd 9 of 2 added to pollfd [in compat_poll_add() at > compat-poll.c:91] > DEBUG1: Accepting client command ... [in thread_manage_clients() at > main.c:3826] > DEBUG1: [thread] Dispatch UST command started [in > thread_dispatch_ust_registration() at main.c:1324] > DEBUG1: Futex n to 1 prepare done [in futex_nto1_prepare() at > futex.c:73] > DEBUG1: Woken up but nothing in the UST command queue [in > thread_dispatch_ust_registration() at main.c:1334] > DEBUG1: Thread manage kernel started [in thread_manage_kernel() at > main.c:876] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at > compat-poll.c:91] > DEBUG1: fd 11 of 2 added to pollfd [in compat_poll_add() at > compat-poll.c:91] > DEBUG1: Updating kernel poll set [in update_kernel_poll() at > main.c:748] > DEBUG1: Thread kernel polling on 2 fds [in thread_manage_kernel() at > main.c:905] > DEBUG1: [thread] Manage application started [in thread_manage_apps() > at main.c:1179] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at > compat-poll.c:91] > DEBUG1: fd 13 of 2 added to pollfd [in compat_poll_add() at > compat-poll.c:91] > DEBUG1: Apps thread polling on 2 fds [in thread_manage_apps() at > main.c:1200] > DEBUG1: [thread] Manage application registration started [in > thread_registration_apps() at main.c:1392] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at > compat-poll.c:91] > DEBUG1: fd 10 of 2 added to pollfd [in compat_poll_add() at > compat-poll.c:91] > DEBUG1: Notifying applications of session daemon state: 1 [in > notify_ust_apps() at main.c:687] > DEBUG1: Got the wait shm fd 15 [in get_wait_shm() at shm.c:117] > DEBUG1: Futex wait update active 1 [in futex_wait_update() at > futex.c:62] > DEBUG1: Accepting application registration [in > thread_registration_apps() at main.c:1423] > > > > #### Output of command "arm-none-linux-gnueabi-lttng-sessiond -vvv " > before loading the modules ############# > > DEBUG3: Creating LTTng run directory: /var/run/lttng [in > create_lttng_rundir() at main.c:4315] > DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in > main() at main.c:4543] > DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command > [in main() at main.c:4545] > DEBUG1: Client socket path /var/run/lttng/client-lttng-sessiond [in > main() at main.c:4592] > DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond [in > main() at main.c:4593] > DEBUG1: LTTng run directory path: /var/run/lttng [in main() at > main.c:4594] > DEBUG2: UST consumer 32 bits err path: > /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] > DEBUG2: UST consumer 32 bits cmd path: > /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] > DEBUG2: UST consumer 64 bits err path: > /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] > DEBUG2: UST consumer 64 bits cmd path: > /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] > DEBUG3: Created hashtable size 4 at 0x4a080 of type 1 [in > lttng_ht_new() at hashtable.c:96] > DEBUG3: Created hashtable size 4 at 0x4a168 of type 1 [in > lttng_ht_new() at hashtable.c:96] > DEBUG2: Creating consumer directory: /var/run/lttng/kconsumerd [in > set_consumer_sockets() at main.c:4357] > FATAL: Module lttng_tracer not found. > Error: Unable to load module lttng-tracer > DEBUG1: Failed to open /proc/lttng [in init_kernel_tracer() at > main.c:1871] > Error: Unable to remove module lttng-tracer > Warning: No kernel tracer available > DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd64 [in > set_consumer_sockets() at main.c:4357] > DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd32 [in > set_consumer_sockets() at main.c:4357] > DEBUG1: Signal handler set for SIGTERM, SIGPIPE and SIGINT [in > set_signal_handler() at main.c:4449] > DEBUG1: All permissions are set [in set_permissions() at main.c:4250] > DEBUG1: poll set max size set to 65535 [in compat_poll_set_max_size() > at compat-poll.c:196] > DEBUG1: [thread] Manage application started [in thread_manage_apps() > at main.c:1179] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at > compat-poll.c:91] > DEBUG1: fd 12 of 2 added to pollfd [in compat_poll_add() at > compat-poll.c:91] > DEBUG1: Apps thread polling on 2 fds [in thread_manage_apps() at > main.c:1200] > DEBUG1: Thread manage kernel started [in thread_manage_kernel() at > main.c:876] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at > compat-poll.c:91] > DEBUG1: fd 10 of 2 added to pollfd [in compat_poll_add() at > compat-poll.c:91] > DEBUG1: Updating kernel poll set [in update_kernel_poll() at > main.c:748] > DEBUG1: Thread kernel polling on 2 fds [in thread_manage_kernel() at > main.c:905] > DEBUG1: [thread] Manage application registration started [in > thread_registration_apps() at main.c:1392] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at > compat-poll.c:91] > DEBUG1: fd 9 of 2 added to pollfd [in compat_poll_add() at > compat-poll.c:91] > DEBUG1: Notifying applications of session daemon state: 1 [in > notify_ust_apps() at main.c:687] > DEBUG1: [thread] Dispatch UST command started [in > thread_dispatch_ust_registration() at main.c:1324] > DEBUG1: Futex n to 1 prepare done [in futex_nto1_prepare() at > futex.c:73] > DEBUG1: Woken up but nothing in the UST command queue [in > thread_dispatch_ust_registration() at main.c:1334] > DEBUG1: [thread] Manage client started [in thread_manage_clients() at > main.c:3794] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at > compat-poll.c:91] > DEBUG1: fd 8 of 2 added to pollfd [in compat_poll_add() at > compat-poll.c:91] > DEBUG1: Accepting client command ... [in thread_manage_clients() at > main.c:3826] > DEBUG1: Got the wait shm fd 14 [in get_wait_shm() at shm.c:117] > DEBUG1: Futex wait update active 1 [in futex_wait_update() at > futex.c:62] > DEBUG1: Accepting application registration [in > thread_registration_apps() at main.c:1423] > > > Regards, > Pavan > > -----Original Message----- > From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] > Sent: Saturday, June 09, 2012 6:03 PM > To: Pavan Anumula > Cc: lttng-dev at lists.lttng.org > Subject: Re: [lttng-dev] Lttng start failed > > * Pavan Anumula (pavan.anumula at sasken.com) wrote: > > Hi Mathue, > > > > Thanks for the quick reply, > > > > After inserting lttng modules , I had given the command "arm-none-linux-gnueabi-lttng-sessiond -vvv" as you said, Below is the output where there are error messages. Please help me in resolving the issue, SO that I can catch kernel and user traces. > > > > > > root at arago:/usr/lttng/modules# arm-none-linux-gnueabi-lttng-sessiond > > -d root at arago:/usr/lttng/modules# > > arm-none-linux-gnueabi-lttng-sessiond -vvv > > DEBUG3: Creating LTTng run directory: /var/run/lttng [in > > create_lttng_rundir() at main.c:4315] > > DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error > > [in > > main() at main.c:4543] > > DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command > > [in main() at main.c:4545] > > DEBUG1: Client socket path /var/run/lttng/client-lttng-sessiond [in > > main() at main.c:4592] > > DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond > > [in > > main() at main.c:4593] > > DEBUG1: LTTng run directory path: /var/run/lttng [in main() at > > main.c:4594] > > DEBUG2: UST consumer 32 bits err path: > > /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] > > DEBUG2: UST consumer 32 bits cmd path: > > /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] > > DEBUG2: UST consumer 64 bits err path: > > /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] > > DEBUG2: UST consumer 64 bits cmd path: > > /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] > > Error: Already running daemon. > > Please kill the lttng-sessiond that is already started (the one with -d). Instead of that one, run the sessiond with: > > lttng-sessiond -vvv > > (don't run lttng-sessiond -d before) > > Thanks, > > Mathieu > > > > > > > > > > > Thanks in advance, > > Pavan > > > > -----Original Message----- > > From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] > > Sent: Friday, June 08, 2012 11:12 PM > > To: Pavan Anumula > > Cc: lttng-dev at lists.lttng.org > > Subject: Re: [lttng-dev] Lttng start failed > > > > * Pavan Anumula (pavan.anumula at sasken.com) wrote: > > > Hi , > > > > > > I am new to LTTng usage, I am trying to use LTTng 2.0.1 and LTTng-modules-2.0.2 for ARM(omapL138), with Linux kernel 2.6.33 wit RT-29 patch on it. > > > > > > After enabling all the kernel events I am facing the below errors. I loaded all the modules manually by insmod. > > > > > > > > > > > > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng create > > > newsessiom Session newsessiom created. > > > Traces will be written in > > > /home/root/lttng-traces/newsessiom-20110325-154922 > > > > > > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng > > > enable-event -a --kernel All kernel events are enabled in channel > > > channel0 > > > > > > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng start > > > LTTng: Failure to write metadata to buffers (timeout) > > > Error: Starting kernel trace failed > > > > > > Please kindly help me on this. > > > > I think you should look into the lttng-sessiond --help : > > > > --consumerd32-path PATH Specify path for the 32-bit UST consumer daemon binary > > --consumerd32-libdir PATH Specify path for the 32-bit UST consumer daemon libraries > > --consumerd64-path PATH Specify path for the 64-bit UST consumer daemon binary > > --consumerd64-libdir PATH Specify path for the 64-bit UST consumer daemon libraries > > > > options. My guess is that lttng-sessiond is not able to find the consumerd binary files, maybe due to a rename or because they have been moved after install. > > > > One more thing that might help is to launch the lttng-sessiond with "-vvv" : it will provide verbose output and let us know where things fall apart. > > > > Thanks, > > > > Mathieu > > > > > > > > > > > > > Thanks in advance, > > > Pavan > > > > > > ________________________________ > > > SASKEN BUSINESS DISCLAIMER: This message may contain confidential, proprietary or legally privileged information. In case you are not the original intended Recipient of the message, you must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message and you are requested to delete it and inform the sender. Any views expressed in this message are those of the individual sender unless otherwise stated. Nothing contained in this message shall be construed as an offer or acceptance of any offer by Sasken Communication Technologies Limited ("Sasken") unless sent with that express intent and with due authority of Sasken. Sasken has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. > > > Read Disclaimer at > > > http://www.sasken.com/extras/mail_disclaimer.html > > > > > _______________________________________________ > > > lttng-dev mailing list > > > lttng-dev at lists.lttng.org > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > > > > > -- > > Mathieu Desnoyers > > Operating System Efficiency R&D Consultant EfficiOS Inc. > > http://www.efficios.com > > -- > Mathieu Desnoyers > Operating System Efficiency R&D Consultant EfficiOS Inc. > http://www.efficios.com -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Tue Jun 12 09:29:26 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 12 Jun 2012 09:29:26 -0400 Subject: [lttng-dev] git tree dead? In-Reply-To: References: Message-ID: <20120612132926.GB13240@Krystal> * lei yang (yanglei.fage at gmail.com) wrote: > Hi list, > > lyang0 at lyang0-OptiPlex-755:~/git$ git clone > git://git.lttng.org/linux-2.6-lttng.git > ???? 'linux-2.6-lttng'... > fatal: The remote end hung up unexpectedly Good catch. Yannick, could you do: cd /var/cache/git; ln -s /home/git/linux-2.6-lttng.git . on the git.lttng.org machine ? This symlink seems to be missing. Thanks, Mathieu > > Lei > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From yannick.brosseau at gmail.com Tue Jun 12 09:31:34 2012 From: yannick.brosseau at gmail.com (Yannick Brosseau) Date: Tue, 12 Jun 2012 09:31:34 -0400 Subject: [lttng-dev] git tree dead? In-Reply-To: <20120612132926.GB13240@Krystal> References: <20120612132926.GB13240@Krystal> Message-ID: <4FD744B6.5000809@gmail.com> On 2012-06-12 09:29, Mathieu Desnoyers wrote: > * lei yang (yanglei.fage at gmail.com) wrote: >> Hi list, >> >> lyang0 at lyang0-OptiPlex-755:~/git$ git clone >> git://git.lttng.org/linux-2.6-lttng.git >> ???? 'linux-2.6-lttng'... >> fatal: The remote end hung up unexpectedly > Good catch. Yannick, could you do: > > cd /var/cache/git; ln -s /home/git/linux-2.6-lttng.git . > > on the git.lttng.org machine ? This symlink seems to be missing. > Fixed From mathieu.desnoyers at efficios.com Tue Jun 12 10:05:48 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 12 Jun 2012 10:05:48 -0400 Subject: [lttng-dev] Lttng start failed In-Reply-To: <47D610AD9C485E458630BA38C324D3B60113FB0E22D1@EXGMBX01.sasken.com> References: <47D610AD9C485E458630BA38C324D3B60113FB00F9EA@EXGMBX01.sasken.com> <20120608174226.GA23339@Krystal> <47D610AD9C485E458630BA38C324D3B60113FB00FA00@EXGMBX01.sasken.com> <20120609123251.GA14280@Krystal> <47D610AD9C485E458630BA38C324D3B60113FB0E22D1@EXGMBX01.sasken.com> Message-ID: <20120612140548.GA13917@Krystal> * Pavan Anumula (pavan.anumula at sasken.com) wrote: > Hi Mathieu, > Still I am unable to start tracing, And I am facing the same start errors. Please kindly help me on this. > This time I loaded the modules by modprobe( not with insmod as I did before), Then I run the command " arm-none-linux-gnueabi-lttng-sessiond -vvv " (before arm-none-linux-gnueabi-lttng-sessiond -d), When using "arm-none-linux-gnueabi-lttng-sessiond -vvv", you should _not_ run "arm-none-linux-gnueabi-lttng-sessiond -d". > The below is the Output, And It got hanged overthere. This is normal: the sessiond is waiting for applications to connect. It does not hang, it just waits. David, can you follow up on this issue ? It seems to be sessiond-related. Thanks, Mathieu > Later when I started lttng start I am acing the same erros below: > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng start > LTTng: Failure to write metadata to buffers (timeout) > Error: Starting kernel trace failed > > ************************************************************************************************ > DEBUG3: Creating LTTng run directory: /var/run/lttng [in create_lttng_rundir() at main.c:4315] > DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in main() at main.c:4543] > DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command [in main() at main.c:4545] > DEBUG1: Client socket path /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] > DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond [in main() at main.c:4593] > DEBUG1: LTTng run directory path: /var/run/lttng [in main() at main.c:4594] > DEBUG2: UST consumer 32 bits err path: /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] > DEBUG2: UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] > DEBUG2: UST consumer 64 bits err path: /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] > DEBUG2: UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] > DEBUG3: Created hashtable size 4 at 0x4a080 of type 1 [in lttng_ht_new() at hashtable.c:96] > DEBUG3: Created hashtable size 4 at 0x4a168 of type 1 [in lttng_ht_new() at hashtable.c:96] > DEBUG2: Creating consumer directory: /var/run/lttng/kconsumerd [in set_consumer_sockets() at main.c:4357] > DEBUG1: Modprobe successfully lttng-tracer [in modprobe_lttng_control() at modprobe.c:163] > DEBUG2: Kernel tracer version validated (major version 2) [in kernel_validate_version() at kernel.c:675] > DEBUG1: Modprobe successfully lttng-ftrace [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-kprobes [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-kretprobes [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-lib-ring-buffer [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-ring-buffer-client-discard [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-ring-buffer-client-overwrite [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-ring-buffer-metadata-client [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-ring-buffer-client-mmap-discard [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-ring-buffer-client-mmap-overwrite [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-ring-buffer-metadata-mmap-client [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-lttng [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-types [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-block [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-irq [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-kvm [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-sched [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-signal [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-statedump [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Modprobe successfully lttng-probe-timer [in modprobe_lttng_data() at modprobe.c:199] > DEBUG1: Kernel tracer fd 6 [in init_kernel_tracer() at main.c:1887] > DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd64 [in set_consumer_sockets() at main.c:4357] > DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() at main.c:4357] > DEBUG1: Signal handler set for SIGTERM, SIGPIPE and SIGINT [in set_signal_handler() at main.c:4449] > DEBUG1: All permissions are set [in set_permissions() at main.c:4250] > DEBUG1: poll set max size set to 65535 [in compat_poll_set_max_size() at compat-poll.c:196] > DEBUG1: [thread] Manage application started [in thread_manage_apps() at main.c:1179] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: fd 13 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: Apps thread polling on 2 fds [in thread_manage_apps() at main.c:1200] > DEBUG1: Thread manage kernel started [in thread_manage_kernel() at main.c:876] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: fd 11 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: Updating kernel poll set [in update_kernel_poll() at main.c:748] > DEBUG1: Thread kernel polling on 2 fds [in thread_manage_kernel() at main.c:905] > DEBUG1: [thread] Manage application registration started [in thread_registration_apps() at main.c:1392] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: fd 10 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: Notifying applications of session daemon state: 1 [in notify_ust_apps() at main.c:687] > DEBUG1: [thread] Dispatch UST command started [in thread_dispatch_ust_registration() at main.c:1324] > DEBUG1: Futex n to 1 prepare done [in futex_nto1_prepare() at futex.c:73] > DEBUG1: Woken up but nothing in the UST command queue [in thread_dispatch_ust_registration() at main.c:1334] > DEBUG1: [thread] Manage client started [in thread_manage_clients() at main.c:3794] > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: fd 9 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3826] > DEBUG1: Got the wait shm fd 15 [in get_wait_shm() at shm.c:117] > DEBUG1: Futex wait update active 1 [in futex_wait_update() at futex.c:62] > DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1423] > ******************************************************************************************************** > Am I missing anything?? > > Thanks in Advance, > Pavan > > > ________________________________________ > From: Mathieu Desnoyers [mathieu.desnoyers at efficios.com] > Sent: Monday, June 11, 2012 7:24 PM > To: Pavan Anumula > Cc: lttng-dev at lists.lttng.org > Subject: Re: [lttng-dev] Lttng start failed > > * Pavan Anumula (pavan.anumula at sasken.com) wrote: > > Hi Mathieu, > > > > Please find the info below as per your comments > > > > > > ########## Output of command "arm-none-linux-gnueabi-lttng-sessiond -vvv " , After loading the modules ####### > > > > DEBUG3: Creating LTTng run directory: /var/run/lttng [in create_lttng_rundir() at main.c:4315] > > DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in main() at main.c:4543] > > DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command [in main() at main.c:4545] > > DEBUG1: Client socket path /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] > > DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond [in main() at main.c:4593] > > DEBUG1: LTTng run directory path: /var/run/lttng [in main() at main.c:4594] > > DEBUG2: UST consumer 32 bits err path: /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] > > DEBUG2: UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] > > DEBUG2: UST consumer 64 bits err path: /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] > > DEBUG2: UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] > > DEBUG3: Created hashtable size 4 at 0x4a080 of type 1 [in lttng_ht_new() at hashtable.c:96] > > DEBUG3: Created hashtable size 4 at 0x4a168 of type 1 [in lttng_ht_new() at hashtable.c:96] > > DEBUG2: Creating consumer directory: /var/run/lttng/kconsumerd [in set_consumer_sockets() at main.c:4357] > > FATAL: Module lttng_tracer not found. > > Error: Unable to load module lttng-tracer > > Hrm. You want to do kernel tracing, but modprobe cannot find the lttng > kernel tracer modules. You might want to run depmod -a or something like > that on your target, and ensure that modprobe works properly. > > Thanks, > > Mathieu > > > DEBUG2: Kernel tracer version validated (major version 2) [in kernel_validate_version() at kernel.c:675] > > DEBUG1: Modprobe successfully lttng-ftrace [in modprobe_lttng_data() at modprobe.c:199] > > DEBUG1: Modprobe successfully lttng-kprobes [in modprobe_lttng_data() at modprobe.c:199] > > DEBUG1: Modprobe successfully lttng-kretprobes [in modprobe_lttng_data() at modprobe.c:199] > > FATAL: Module lttng_lib_ring_buffer not found. > > Error: Unable to load module lttng-lib-ring-buffer > > FATAL: Module lttng_ring_buffer_client_discard not found. > > Error: Unable to load module lttng-ring-buffer-client-discard > > FATAL: Module lttng_ring_buffer_client_overwrite not found. > > Error: Unable to load module lttng-ring-buffer-client-overwrite > > FATAL: Module lttng_ring_buffer_metadata_client not found. > > Error: Unable to load module lttng-ring-buffer-metadata-client > > FATAL: Module lttng_ring_buffer_client_mmap_discard not found. > > Error: Unable to load module lttng-ring-buffer-client-mmap-discard > > FATAL: Module lttng_ring_buffer_client_mmap_overwrite not found. > > Error: Unable to load module lttng-ring-buffer-client-mmap-overwrite > > FATAL: Module lttng_ring_buffer_metadata_mmap_client not found. > > Error: Unable to load module lttng-ring-buffer-metadata-mmap-client > > FATAL: Module lttng_probe_lttng not found. > > Error: Unable to load module lttng-probe-lttng > > DEBUG1: Modprobe successfully lttng-types [in modprobe_lttng_data() at modprobe.c:199] > > DEBUG1: Modprobe successfully lttng-probe-block [in modprobe_lttng_data() at modprobe.c:199] > > DEBUG1: Modprobe successfully lttng-probe-irq [in modprobe_lttng_data() at modprobe.c:199] > > DEBUG1: Modprobe successfully lttng-probe-kvm [in modprobe_lttng_data() at modprobe.c:199] > > DEBUG1: Modprobe successfully lttng-probe-sched [in modprobe_lttng_data() at modprobe.c:199] > > DEBUG1: Modprobe successfully lttng-probe-signal [in modprobe_lttng_data() at modprobe.c:199] > > DEBUG1: Modprobe successfully lttng-probe-statedump [in modprobe_lttng_data() at modprobe.c:199] > > DEBUG1: Modprobe successfully lttng-probe-timer [in modprobe_lttng_data() at modprobe.c:199] > > DEBUG1: Kernel tracer fd 6 [in init_kernel_tracer() at main.c:1887] > > DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd64 [in set_consumer_sockets() at main.c:4357] > > DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() at main.c:4357] > > DEBUG1: Signal handler set for SIGTERM, SIGPIPE and SIGINT [in set_signal_handler() at main.c:4449] > > DEBUG1: All permissions are set [in set_permissions() at main.c:4250] > > DEBUG1: poll set max size set to 65535 [in compat_poll_set_max_size() at compat-poll.c:196] > > DEBUG1: [thread] Manage client started [in thread_manage_clients() at main.c:3794] > > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > > DEBUG1: fd 9 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > > DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3826] > > DEBUG1: [thread] Dispatch UST command started [in thread_dispatch_ust_registration() at main.c:1324] > > DEBUG1: Futex n to 1 prepare done [in futex_nto1_prepare() at futex.c:73] > > DEBUG1: Woken up but nothing in the UST command queue [in thread_dispatch_ust_registration() at main.c:1334] > > DEBUG1: Thread manage kernel started [in thread_manage_kernel() at main.c:876] > > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > > DEBUG1: fd 11 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > > DEBUG1: Updating kernel poll set [in update_kernel_poll() at main.c:748] > > DEBUG1: Thread kernel polling on 2 fds [in thread_manage_kernel() at main.c:905] > > DEBUG1: [thread] Manage application started [in thread_manage_apps() at main.c:1179] > > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > > DEBUG1: fd 13 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > > DEBUG1: Apps thread polling on 2 fds [in thread_manage_apps() at main.c:1200] > > DEBUG1: [thread] Manage application registration started [in thread_registration_apps() at main.c:1392] > > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > > DEBUG1: fd 10 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > > DEBUG1: Notifying applications of session daemon state: 1 [in notify_ust_apps() at main.c:687] > > DEBUG1: Got the wait shm fd 15 [in get_wait_shm() at shm.c:117] > > DEBUG1: Futex wait update active 1 [in futex_wait_update() at futex.c:62] > > DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1423] > > > > > > > > #### Output of command "arm-none-linux-gnueabi-lttng-sessiond -vvv " before loading the modules ############# > > > > DEBUG3: Creating LTTng run directory: /var/run/lttng [in create_lttng_rundir() at main.c:4315] > > DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in main() at main.c:4543] > > DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command [in main() at main.c:4545] > > DEBUG1: Client socket path /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] > > DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond [in main() at main.c:4593] > > DEBUG1: LTTng run directory path: /var/run/lttng [in main() at main.c:4594] > > DEBUG2: UST consumer 32 bits err path: /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] > > DEBUG2: UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] > > DEBUG2: UST consumer 64 bits err path: /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] > > DEBUG2: UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] > > DEBUG3: Created hashtable size 4 at 0x4a080 of type 1 [in lttng_ht_new() at hashtable.c:96] > > DEBUG3: Created hashtable size 4 at 0x4a168 of type 1 [in lttng_ht_new() at hashtable.c:96] > > DEBUG2: Creating consumer directory: /var/run/lttng/kconsumerd [in set_consumer_sockets() at main.c:4357] > > FATAL: Module lttng_tracer not found. > > Error: Unable to load module lttng-tracer > > DEBUG1: Failed to open /proc/lttng [in init_kernel_tracer() at main.c:1871] > > Error: Unable to remove module lttng-tracer > > Warning: No kernel tracer available > > DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd64 [in set_consumer_sockets() at main.c:4357] > > DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() at main.c:4357] > > DEBUG1: Signal handler set for SIGTERM, SIGPIPE and SIGINT [in set_signal_handler() at main.c:4449] > > DEBUG1: All permissions are set [in set_permissions() at main.c:4250] > > DEBUG1: poll set max size set to 65535 [in compat_poll_set_max_size() at compat-poll.c:196] > > DEBUG1: [thread] Manage application started [in thread_manage_apps() at main.c:1179] > > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > > DEBUG1: fd 12 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > > DEBUG1: Apps thread polling on 2 fds [in thread_manage_apps() at main.c:1200] > > DEBUG1: Thread manage kernel started [in thread_manage_kernel() at main.c:876] > > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > > DEBUG1: fd 10 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > > DEBUG1: Updating kernel poll set [in update_kernel_poll() at main.c:748] > > DEBUG1: Thread kernel polling on 2 fds [in thread_manage_kernel() at main.c:905] > > DEBUG1: [thread] Manage application registration started [in thread_registration_apps() at main.c:1392] > > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > > DEBUG1: fd 9 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > > DEBUG1: Notifying applications of session daemon state: 1 [in notify_ust_apps() at main.c:687] > > DEBUG1: [thread] Dispatch UST command started [in thread_dispatch_ust_registration() at main.c:1324] > > DEBUG1: Futex n to 1 prepare done [in futex_nto1_prepare() at futex.c:73] > > DEBUG1: Woken up but nothing in the UST command queue [in thread_dispatch_ust_registration() at main.c:1334] > > DEBUG1: [thread] Manage client started [in thread_manage_clients() at main.c:3794] > > DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > > DEBUG1: fd 8 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] > > DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3826] > > DEBUG1: Got the wait shm fd 14 [in get_wait_shm() at shm.c:117] > > DEBUG1: Futex wait update active 1 [in futex_wait_update() at futex.c:62] > > DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1423] > > > > > > Regards, > > Pavan > > > > -----Original Message----- > > From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] > > Sent: Saturday, June 09, 2012 6:03 PM > > To: Pavan Anumula > > Cc: lttng-dev at lists.lttng.org > > Subject: Re: [lttng-dev] Lttng start failed > > > > * Pavan Anumula (pavan.anumula at sasken.com) wrote: > > > Hi Mathue, > > > > > > Thanks for the quick reply, > > > > > > After inserting lttng modules , I had given the command "arm-none-linux-gnueabi-lttng-sessiond -vvv" as you said, Below is the output where there are error messages. Please help me in resolving the issue, SO that I can catch kernel and user traces. > > > > > > > > > root at arago:/usr/lttng/modules# arm-none-linux-gnueabi-lttng-sessiond > > > -d root at arago:/usr/lttng/modules# > > > arm-none-linux-gnueabi-lttng-sessiond -vvv > > > DEBUG3: Creating LTTng run directory: /var/run/lttng [in > > > create_lttng_rundir() at main.c:4315] > > > DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in > > > main() at main.c:4543] > > > DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command > > > [in main() at main.c:4545] > > > DEBUG1: Client socket path /var/run/lttng/client-lttng-sessiond [in > > > main() at main.c:4592] > > > DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond [in > > > main() at main.c:4593] > > > DEBUG1: LTTng run directory path: /var/run/lttng [in main() at > > > main.c:4594] > > > DEBUG2: UST consumer 32 bits err path: > > > /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] > > > DEBUG2: UST consumer 32 bits cmd path: > > > /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] > > > DEBUG2: UST consumer 64 bits err path: > > > /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] > > > DEBUG2: UST consumer 64 bits cmd path: > > > /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] > > > Error: Already running daemon. > > > > Please kill the lttng-sessiond that is already started (the one with -d). Instead of that one, run the sessiond with: > > > > lttng-sessiond -vvv > > > > (don't run lttng-sessiond -d before) > > > > Thanks, > > > > Mathieu > > > > > > > > > > > > > > > > > Thanks in advance, > > > Pavan > > > > > > -----Original Message----- > > > From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] > > > Sent: Friday, June 08, 2012 11:12 PM > > > To: Pavan Anumula > > > Cc: lttng-dev at lists.lttng.org > > > Subject: Re: [lttng-dev] Lttng start failed > > > > > > * Pavan Anumula (pavan.anumula at sasken.com) wrote: > > > > Hi , > > > > > > > > I am new to LTTng usage, I am trying to use LTTng 2.0.1 and LTTng-modules-2.0.2 for ARM(omapL138), with Linux kernel 2.6.33 wit RT-29 patch on it. > > > > > > > > After enabling all the kernel events I am facing the below errors. I loaded all the modules manually by insmod. > > > > > > > > > > > > > > > > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng create > > > > newsessiom Session newsessiom created. > > > > Traces will be written in > > > > /home/root/lttng-traces/newsessiom-20110325-154922 > > > > > > > > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng enable-event > > > > -a --kernel All kernel events are enabled in channel channel0 > > > > > > > > root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng start > > > > LTTng: Failure to write metadata to buffers (timeout) > > > > Error: Starting kernel trace failed > > > > > > > > Please kindly help me on this. > > > > > > I think you should look into the lttng-sessiond --help : > > > > > > --consumerd32-path PATH Specify path for the 32-bit UST consumer daemon binary > > > --consumerd32-libdir PATH Specify path for the 32-bit UST consumer daemon libraries > > > --consumerd64-path PATH Specify path for the 64-bit UST consumer daemon binary > > > --consumerd64-libdir PATH Specify path for the 64-bit UST consumer daemon libraries > > > > > > options. My guess is that lttng-sessiond is not able to find the consumerd binary files, maybe due to a rename or because they have been moved after install. > > > > > > One more thing that might help is to launch the lttng-sessiond with "-vvv" : it will provide verbose output and let us know where things fall apart. > > > > > > Thanks, > > > > > > Mathieu > > > > > > > > > > > > > > > > > > Thanks in advance, > > > > Pavan > > > > > > > > ________________________________ > > > > SASKEN BUSINESS DISCLAIMER: This message may contain confidential, proprietary or legally privileged information. In case you are not the original intended Recipient of the message, you must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message and you are requested to delete it and inform the sender. Any views expressed in this message are those of the individual sender unless otherwise stated. Nothing contained in this message shall be construed as an offer or acceptance of any offer by Sasken Communication Technologies Limited ("Sasken") unless sent with that express intent and with due authority of Sasken. Sasken has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. > > > > Read Disclaimer at http://www.sasken.com/extras/mail_disclaimer.html > > > > > > > _______________________________________________ > > > > lttng-dev mailing list > > > > lttng-dev at lists.lttng.org > > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > > > > > > > > -- > > > Mathieu Desnoyers > > > Operating System Efficiency R&D Consultant EfficiOS Inc. > > > http://www.efficios.com > > > > -- > > Mathieu Desnoyers > > Operating System Efficiency R&D Consultant EfficiOS Inc. > > http://www.efficios.com > > -- > Mathieu Desnoyers > Operating System Efficiency R&D Consultant > EfficiOS Inc. > http://www.efficios.com -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Tue Jun 12 10:09:26 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 12 Jun 2012 10:09:26 -0400 Subject: [lttng-dev] UST Hanging at Tracepoint In-Reply-To: <3F56B905CF2083408E8482044F20428AEF78BC79@ONWVEXCHMB01.ciena.com> References: <3F56B905CF2083408E8482044F20428AEF604D22@ONWVEXCHMB01.ciena.com> <3F56B905CF2083408E8482044F20428AEF78BC79@ONWVEXCHMB01.ciena.com> Message-ID: <20120612140925.GB13917@Krystal> Hi Michael, Please note that you should be able to run UST with either (or both) of: - root lttng-sessiond, for system-wide tracing - per-user lttng-sessiond, for per-user tracing. It is entirely normal that one of the two UST threads within the application can't find an active sessiond if, for instance, you have just a root (and no per-user) sessiond running. So although I'm glad that you got things working, I fail to understand your explanation: it should have worked fine with the root sessiond, even if no ~/.lttng exists. David, can you look into this ? Thanks, Mathieu * Burton, Michael (mburton at ciena.com) wrote: > Mathieu, > > I figured out the problem. When we start sessiond on our system as root, the /.lttng/ folder is not created, along with its contents. UST timed out since there was no sock_path. I was able to get UST working once I got sessiond running. > > Thanks. > Michael > > ________________________________________ > From: Mathieu Desnoyers [mathieu.desnoyers at efficios.com] > Sent: 08 June 2012 21:38 > To: Burton, Michael > Cc: lttng-dev at lists.lttng.org > Subject: Re: [lttng-dev] UST Hanging at Tracepoint > > * Burton, Michael (mburton at ciena.com) wrote: > > For further insight, I've attached the strace from the hanging > > program. I also upgraded LTTng-UST to 2.0.3 and userspace-rcu to > > 0.7.3. > > Haven't had time to look at it yet, but could you also provide the > output of: > > LTTNG_UST_DEBUG=1 youprogname > > (starting your UST instrumented program with LTTNG_UST_DEBUG=1 env. var. > set) > > Thanks! > > Mathieu > > > > > Michael > > > > From: Burton, Michael [mailto:mburton at ciena.com] > > Sent: Thursday, June 07, 2012 2:16 PM > > To: lttng-dev at lists.lttng.org > > Subject: [lttng-dev] UST Hanging at Tracepoint > > > > I am working on getting LTTng 2.0 working in our 2.6.35 kernel. I have kernel tracing working but I'm having problems with UST. > > > > I have a program with a dynamic UST tracepoint. When I run the program with all UST events enabled (lttng enable-event -a -u) the program hangs on the tracepoint. The last line from the UST debug is: > > > > libust[6184/6185]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) > > > > Any insight into why this is happening? I've attached the UST and lttng-sessiond debug generated by the program. > > > > I am running the following: > > lttng-modules-2.6.32 (found through this mailing list) > > lttng-tools-2.0.1 > > lttng-ust-2.0.2 > > userspace-rcu-0.7.0 > > > > Thanks, > > Michael > > Content-Description: strace_debug.txt > > execve("/ciena/bin/idp", ["idp", "help"], [/* 11 vars */]) = 0 > > brk(0) = 0x804e000 > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb783c000 > > access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) > > open("/etc/ld.so.cache", O_RDONLY) = 3 > > fstat64(3, {st_mode=S_IFREG|0644, st_size=17420, ...}) = 0 > > mmap2(NULL, 17420, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7837000 > > close(3) = 0 > > open("/usr/lib/liblttng-ust.so.0", O_RDONLY) = 3 > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220A\0\0004\0\0\0$"..., 512) = 512 > > fstat64(3, {st_mode=S_IFREG|0755, st_size=209876, ...}) = 0 > > mmap2(NULL, 212940, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7803000 > > mmap2(0xb7832000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2e) = 0xb7832000 > > close(3) = 0 > > open("/usr/lib/liblttng-ust-ctl.so.0", O_RDONLY) = 3 > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\200-\0\0004\0\0\0\274"..., 512) = 512 > > fstat64(3, {st_mode=S_IFREG|0755, st_size=140020, ...}) = 0 > > mmap2(NULL, 142828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77e0000 > > mmap2(0xb7802000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x21) = 0xb7802000 > > close(3) = 0 > > open("/usr/lib/liblttng-ust-fork.so.0", O_RDONLY) = 3 > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\0\5\0\0004\0\0\0X"..., 512) = 512 > > fstat64(3, {st_mode=S_IFREG|0755, st_size=3864, ...}) = 0 > > mmap2(NULL, 6820, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77de000 > > mmap2(0xb77df000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77df000 > > close(3) = 0 > > open("/usr/lib/liblttng-ust-tracepoint.so.0", O_RDONLY) = 3 > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220\v\0\0004\0\0\0008"..., 512) = 512 > > fstat64(3, {st_mode=S_IFREG|0755, st_size=16632, ...}) = 0 > > mmap2(NULL, 19944, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d9000 > > mmap2(0xb77dd000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3) = 0xb77dd000 > > close(3) = 0 > > open("/usr/lib/liblttng-ust-libc-wrapper.so.0", O_RDONLY) = 3 > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\t\0\0004\0\0\0\364"..., 512) = 512 > > fstat64(3, {st_mode=S_IFREG|0755, st_size=9300, ...}) = 0 > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb77d8000 > > mmap2(NULL, 12104, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d5000 > > mmap2(0xb77d7000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77d7000 > > close(3) = 0 > > open("/usr/lib/liburcu.so.1", O_RDONLY) = 3 > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"..., 512) = 512 > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d0000 > > mmap2(0xb77d4000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77d4000 > > close(3) = 0 > > open("/usr/lib/liburcu-common.so.1", O_RDONLY) = 3 > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340\5\0\0004\0\0\0\34"..., 512) = 512 > > fstat64(3, {st_mode=S_IFREG|0755, st_size=5084, ...}) = 0 > > mmap2(NULL, 8032, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77ce000 > > mmap2(0xb77cf000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77cf000 > > close(3) = 0 > > open("/usr/lib/liburcu-qsbr.so.1", O_RDONLY) = 3 > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\22\0\0004\0\0\0\374"..., 512) = 512 > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77c9000 > > mmap2(0xb77cd000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77cd000 > > close(3) = 0 > > open("/usr/lib/liburcu-mb.so.1", O_RDONLY) = 3 > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"..., 512) = 512 > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77c4000 > > mmap2(0xb77c8000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c8000 > > close(3) = 0 > > open("/usr/lib/liburcu-signal.so.1", O_RDONLY) = 3 > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\23\0\0004\0\0\0\10"..., 512) = 512 > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18712, ...}) = 0 > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb77c3000 > > mmap2(NULL, 21704, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77bd000 > > mmap2(0xb77c2000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c2000 > > close(3) = 0 > > open("/usr/lib/liburcu-cds.so.1", O_RDONLY) = 3 > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\300\f\0\0004\0\0\0t"..., 512) = 512 > > fstat64(3, {st_mode=S_IFREG|0755, st_size=26204, ...}) = 0 > > mmap2(NULL, 25004, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77b6000 > > mmap2(0xb77bc000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb77bc000 > > close(3) = 0 > > open("/usr/lib/liburcu-bp.so.1", O_RDONLY) = 3 > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320\24\0\0004\0\0\0\360"..., 512) = 512 > > fstat64(3, {st_mode=S_IFREG|0755, st_size=19968, ...}) = 0 > > mmap2(NULL, 23128, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77b0000 > > mmap2(0xb77b5000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77b5000 > > close(3) = 0 > > open("/lib/libdl.so.2", O_RDONLY) = 3 > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0 \n\0\0004\0\0\0<"..., 512) = 512 > > fstat64(3, {st_mode=S_IFREG|0755, st_size=9668, ...}) = 0 > > mmap2(NULL, 12408, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77ac000 > > mmap2(0xb77ae000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77ae000 > > close(3) = 0 > > open("/lib/libuuid.so.1", O_RDONLY) = 3 > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0@\r\0\0004\0\0\0\350"..., 512) = 512 > > fstat64(3, {st_mode=S_IFREG|0755, st_size=12872, ...}) = 0 > > mmap2(NULL, 15632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77a8000 > > mmap2(0xb77ab000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2) = 0xb77ab000 > > close(3) = 0 > > open("/lib/libc.so.6", O_RDONLY) = 3 > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220n\1\0004\0\0\0L"..., 512) = 512 > > fstat64(3, {st_mode=S_IFREG|0755, st_size=1347780, ...}) = 0 > > mmap2(NULL, 1358120, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb765c000 > > mprotect(0xb77a1000, 4096, PROT_NONE) = 0 > > mmap2(0xb77a2000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x145) = 0xb77a2000 > > mmap2(0xb77a5000, 10536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb77a5000 > > close(3) = 0 > > open("/lib/librt.so.1", O_RDONLY) = 3 > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0p\30\0\0004\0\0\0\230"..., 512) = 512 > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb765b000 > > fstat64(3, {st_mode=S_IFREG|0755, st_size=30616, ...}) = 0 > > mmap2(NULL, 33364, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7652000 > > mmap2(0xb7659000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb7659000 > > close(3) = 0 > > open("/lib/libpthread.so.0", O_RDONLY) = 3 > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360J\0\0004\0\0\0\354"..., 512) = 512 > > fstat64(3, {st_mode=S_IFREG|0755, st_size=88420, ...}) = 0 > > mmap2(NULL, 98828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7639000 > > mmap2(0xb764e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14) = 0xb764e000 > > mmap2(0xb7650000, 4620, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7650000 > > close(3) = 0 > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7638000 > > mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7636000 > > set_thread_area({entry_number:-1 -> 6, base_addr:0xb7636d80, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 > > mprotect(0xb764e000, 4096, PROT_READ) = 0 > > mprotect(0xb7659000, 4096, PROT_READ) = 0 > > mprotect(0xb77a2000, 8192, PROT_READ) = 0 > > mprotect(0xb77ae000, 4096, PROT_READ) = 0 > > mprotect(0xb785a000, 4096, PROT_READ) = 0 > > munmap(0xb7837000, 17420) = 0 > > set_tid_address(0xb7636de8) = 3050 > > set_robust_list(0xb7636df0, 0xc) = 0 > > futex(0xbfaaf560, FUTEX_WAKE_PRIVATE, 1) = 0 > > futex(0xbfaaf560, 0x189 /* FUTEX_??? */, 1, NULL, bfaaf570) = -1 EAGAIN (Resource temporarily unavailable) > > rt_sigaction(SIGRTMIN, {0xb763d4f0, [], SA_SIGINFO}, NULL, 8) = 0 > > rt_sigaction(SIGRT_1, {0xb763d9c0, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0 > > rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0 > > getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0 > > uname({sys="Linux", node="5410", ...}) = 0 > > rt_sigaction(SIGUSR1, {0xb77bec0a, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0 > > futex(0xb77af06c, FUTEX_WAKE_PRIVATE, 2147483647) = 0 > > brk(0) = 0x804e000 > > brk(0x806f000) = 0x806f000 > > gettimeofday({1339187216, 880428}, NULL) = 0 > > open("/dev/urandom", O_RDONLY|O_LARGEFILE) = 3 > > fcntl64(3, F_GETFD) = 0 > > fcntl64(3, F_SETFD, FD_CLOEXEC) = 0 > > getuid32() = 0 > > gettimeofday({1339187216, 889603}, NULL) = 0 > > gettimeofday({1339187216, 894471}, NULL) = 0 > > read(3, "\275\0051F\215\373\371\202\377kfb/\362\303N"..., 16) = 16 > > clock_gettime(CLOCK_REALTIME, {1339187216, 902329405}) = 0 > > getuid32() = 0 > > geteuid32() = 0 > > rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0 > > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6e36000 > > mprotect(0xb6e36000, 4096, PROT_NONE) = 0 > > clone(child_stack=0xb7634d64, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0xb7635bd8, {entry_number:6, base_addr:0xb7635b70, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xb7635bd8) = 3051 > > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6636000 > > mprotect(0xb6636000, 4096, PROT_NONE) = 0 > > clone(child_stack=0xb6e34d64, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0xb6e35bd8, {entry_number:6, base_addr:0xb6e35b70, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xb6e35bd8) = 3052 > > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > > gettimeofday({1339187216, 974869}, NULL) = 0 > > futex(0xb7836e30, FUTEX_WAIT_PRIVATE, 0, {2, 927460405}) = -1 ETIMEDOUT (Connection timed out) > > gettid() = 3050 > > write(2, "libust[3050/3050]: Error: Timed o"..., 107libust[3050/3050]: Error: Timed out waiting for ltt-sessiond (in lttng_ust_init() at lttng-ust-comm.c:912) > > ) = 107 > > futex(0xb7836e80, FUTEX_WAIT_PRIVATE, 2, NULL > > _______________________________________________ > > lttng-dev mailing list > > lttng-dev at lists.lttng.org > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > > -- > Mathieu Desnoyers > Operating System Efficiency R&D Consultant > EfficiOS Inc. > http://www.efficios.com -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Tue Jun 12 10:13:44 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 12 Jun 2012 10:13:44 -0400 Subject: [lttng-dev] UST Hanging at Tracepoint In-Reply-To: <20120612140925.GB13917@Krystal> References: <3F56B905CF2083408E8482044F20428AEF604D22@ONWVEXCHMB01.ciena.com> <3F56B905CF2083408E8482044F20428AEF78BC79@ONWVEXCHMB01.ciena.com> <20120612140925.GB13917@Krystal> Message-ID: <20120612141344.GC13917@Krystal> * Mathieu Desnoyers (mathieu.desnoyers at efficios.com) wrote: > Hi Michael, > > Please note that you should be able to run UST with either (or both) of: > > - root lttng-sessiond, for system-wide tracing > - per-user lttng-sessiond, for per-user tracing. > > It is entirely normal that one of the two UST threads within the > application can't find an active sessiond if, for instance, you have > just a root (and no per-user) sessiond running. > > So although I'm glad that you got things working, I fail to understand > your explanation: it should have worked fine with the root sessiond, > even if no ~/.lttng exists. > > David, can you look into this ? hrm, further insight: is your program _really_ hang ? Try to make sure you put a fprintf(stderr, ".....") at the beginning of your main(). Looking once more at your detailed UST output indicates that the per-user UST thread tells you that it cannot see any running sessiond. So yes, this thread blocks, but the thread talking to the root sessiond can very well be working fine, and your application might also be running fine. This should be double-checked. Thanks, Mathieu > > Thanks, > > Mathieu > > * Burton, Michael (mburton at ciena.com) wrote: > > Mathieu, > > > > I figured out the problem. When we start sessiond on our system as root, the /.lttng/ folder is not created, along with its contents. UST timed out since there was no sock_path. I was able to get UST working once I got sessiond running. > > > > Thanks. > > Michael > > > > ________________________________________ > > From: Mathieu Desnoyers [mathieu.desnoyers at efficios.com] > > Sent: 08 June 2012 21:38 > > To: Burton, Michael > > Cc: lttng-dev at lists.lttng.org > > Subject: Re: [lttng-dev] UST Hanging at Tracepoint > > > > * Burton, Michael (mburton at ciena.com) wrote: > > > For further insight, I've attached the strace from the hanging > > > program. I also upgraded LTTng-UST to 2.0.3 and userspace-rcu to > > > 0.7.3. > > > > Haven't had time to look at it yet, but could you also provide the > > output of: > > > > LTTNG_UST_DEBUG=1 youprogname > > > > (starting your UST instrumented program with LTTNG_UST_DEBUG=1 env. var. > > set) > > > > Thanks! > > > > Mathieu > > > > > > > > Michael > > > > > > From: Burton, Michael [mailto:mburton at ciena.com] > > > Sent: Thursday, June 07, 2012 2:16 PM > > > To: lttng-dev at lists.lttng.org > > > Subject: [lttng-dev] UST Hanging at Tracepoint > > > > > > I am working on getting LTTng 2.0 working in our 2.6.35 kernel. I have kernel tracing working but I'm having problems with UST. > > > > > > I have a program with a dynamic UST tracepoint. When I run the program with all UST events enabled (lttng enable-event -a -u) the program hangs on the tracepoint. The last line from the UST debug is: > > > > > > libust[6184/6185]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) > > > > > > Any insight into why this is happening? I've attached the UST and lttng-sessiond debug generated by the program. > > > > > > I am running the following: > > > lttng-modules-2.6.32 (found through this mailing list) > > > lttng-tools-2.0.1 > > > lttng-ust-2.0.2 > > > userspace-rcu-0.7.0 > > > > > > Thanks, > > > Michael > > > > Content-Description: strace_debug.txt > > > execve("/ciena/bin/idp", ["idp", "help"], [/* 11 vars */]) = 0 > > > brk(0) = 0x804e000 > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb783c000 > > > access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) > > > open("/etc/ld.so.cache", O_RDONLY) = 3 > > > fstat64(3, {st_mode=S_IFREG|0644, st_size=17420, ...}) = 0 > > > mmap2(NULL, 17420, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7837000 > > > close(3) = 0 > > > open("/usr/lib/liblttng-ust.so.0", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220A\0\0004\0\0\0$"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=209876, ...}) = 0 > > > mmap2(NULL, 212940, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7803000 > > > mmap2(0xb7832000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2e) = 0xb7832000 > > > close(3) = 0 > > > open("/usr/lib/liblttng-ust-ctl.so.0", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\200-\0\0004\0\0\0\274"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=140020, ...}) = 0 > > > mmap2(NULL, 142828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77e0000 > > > mmap2(0xb7802000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x21) = 0xb7802000 > > > close(3) = 0 > > > open("/usr/lib/liblttng-ust-fork.so.0", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\0\5\0\0004\0\0\0X"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=3864, ...}) = 0 > > > mmap2(NULL, 6820, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77de000 > > > mmap2(0xb77df000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77df000 > > > close(3) = 0 > > > open("/usr/lib/liblttng-ust-tracepoint.so.0", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220\v\0\0004\0\0\0008"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=16632, ...}) = 0 > > > mmap2(NULL, 19944, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d9000 > > > mmap2(0xb77dd000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3) = 0xb77dd000 > > > close(3) = 0 > > > open("/usr/lib/liblttng-ust-libc-wrapper.so.0", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\t\0\0004\0\0\0\364"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=9300, ...}) = 0 > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb77d8000 > > > mmap2(NULL, 12104, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d5000 > > > mmap2(0xb77d7000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77d7000 > > > close(3) = 0 > > > open("/usr/lib/liburcu.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d0000 > > > mmap2(0xb77d4000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77d4000 > > > close(3) = 0 > > > open("/usr/lib/liburcu-common.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340\5\0\0004\0\0\0\34"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=5084, ...}) = 0 > > > mmap2(NULL, 8032, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77ce000 > > > mmap2(0xb77cf000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77cf000 > > > close(3) = 0 > > > open("/usr/lib/liburcu-qsbr.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\22\0\0004\0\0\0\374"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77c9000 > > > mmap2(0xb77cd000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77cd000 > > > close(3) = 0 > > > open("/usr/lib/liburcu-mb.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77c4000 > > > mmap2(0xb77c8000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c8000 > > > close(3) = 0 > > > open("/usr/lib/liburcu-signal.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\23\0\0004\0\0\0\10"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18712, ...}) = 0 > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb77c3000 > > > mmap2(NULL, 21704, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77bd000 > > > mmap2(0xb77c2000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c2000 > > > close(3) = 0 > > > open("/usr/lib/liburcu-cds.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\300\f\0\0004\0\0\0t"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=26204, ...}) = 0 > > > mmap2(NULL, 25004, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77b6000 > > > mmap2(0xb77bc000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb77bc000 > > > close(3) = 0 > > > open("/usr/lib/liburcu-bp.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320\24\0\0004\0\0\0\360"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=19968, ...}) = 0 > > > mmap2(NULL, 23128, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77b0000 > > > mmap2(0xb77b5000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77b5000 > > > close(3) = 0 > > > open("/lib/libdl.so.2", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0 \n\0\0004\0\0\0<"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=9668, ...}) = 0 > > > mmap2(NULL, 12408, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77ac000 > > > mmap2(0xb77ae000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77ae000 > > > close(3) = 0 > > > open("/lib/libuuid.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0@\r\0\0004\0\0\0\350"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=12872, ...}) = 0 > > > mmap2(NULL, 15632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77a8000 > > > mmap2(0xb77ab000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2) = 0xb77ab000 > > > close(3) = 0 > > > open("/lib/libc.so.6", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220n\1\0004\0\0\0L"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=1347780, ...}) = 0 > > > mmap2(NULL, 1358120, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb765c000 > > > mprotect(0xb77a1000, 4096, PROT_NONE) = 0 > > > mmap2(0xb77a2000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x145) = 0xb77a2000 > > > mmap2(0xb77a5000, 10536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb77a5000 > > > close(3) = 0 > > > open("/lib/librt.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0p\30\0\0004\0\0\0\230"..., 512) = 512 > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb765b000 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=30616, ...}) = 0 > > > mmap2(NULL, 33364, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7652000 > > > mmap2(0xb7659000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb7659000 > > > close(3) = 0 > > > open("/lib/libpthread.so.0", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360J\0\0004\0\0\0\354"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=88420, ...}) = 0 > > > mmap2(NULL, 98828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7639000 > > > mmap2(0xb764e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14) = 0xb764e000 > > > mmap2(0xb7650000, 4620, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7650000 > > > close(3) = 0 > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7638000 > > > mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7636000 > > > set_thread_area({entry_number:-1 -> 6, base_addr:0xb7636d80, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 > > > mprotect(0xb764e000, 4096, PROT_READ) = 0 > > > mprotect(0xb7659000, 4096, PROT_READ) = 0 > > > mprotect(0xb77a2000, 8192, PROT_READ) = 0 > > > mprotect(0xb77ae000, 4096, PROT_READ) = 0 > > > mprotect(0xb785a000, 4096, PROT_READ) = 0 > > > munmap(0xb7837000, 17420) = 0 > > > set_tid_address(0xb7636de8) = 3050 > > > set_robust_list(0xb7636df0, 0xc) = 0 > > > futex(0xbfaaf560, FUTEX_WAKE_PRIVATE, 1) = 0 > > > futex(0xbfaaf560, 0x189 /* FUTEX_??? */, 1, NULL, bfaaf570) = -1 EAGAIN (Resource temporarily unavailable) > > > rt_sigaction(SIGRTMIN, {0xb763d4f0, [], SA_SIGINFO}, NULL, 8) = 0 > > > rt_sigaction(SIGRT_1, {0xb763d9c0, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0 > > > rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0 > > > getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0 > > > uname({sys="Linux", node="5410", ...}) = 0 > > > rt_sigaction(SIGUSR1, {0xb77bec0a, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0 > > > futex(0xb77af06c, FUTEX_WAKE_PRIVATE, 2147483647) = 0 > > > brk(0) = 0x804e000 > > > brk(0x806f000) = 0x806f000 > > > gettimeofday({1339187216, 880428}, NULL) = 0 > > > open("/dev/urandom", O_RDONLY|O_LARGEFILE) = 3 > > > fcntl64(3, F_GETFD) = 0 > > > fcntl64(3, F_SETFD, FD_CLOEXEC) = 0 > > > getuid32() = 0 > > > gettimeofday({1339187216, 889603}, NULL) = 0 > > > gettimeofday({1339187216, 894471}, NULL) = 0 > > > read(3, "\275\0051F\215\373\371\202\377kfb/\362\303N"..., 16) = 16 > > > clock_gettime(CLOCK_REALTIME, {1339187216, 902329405}) = 0 > > > getuid32() = 0 > > > geteuid32() = 0 > > > rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0 > > > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6e36000 > > > mprotect(0xb6e36000, 4096, PROT_NONE) = 0 > > > clone(child_stack=0xb7634d64, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0xb7635bd8, {entry_number:6, base_addr:0xb7635b70, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xb7635bd8) = 3051 > > > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6636000 > > > mprotect(0xb6636000, 4096, PROT_NONE) = 0 > > > clone(child_stack=0xb6e34d64, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0xb6e35bd8, {entry_number:6, base_addr:0xb6e35b70, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xb6e35bd8) = 3052 > > > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > > > gettimeofday({1339187216, 974869}, NULL) = 0 > > > futex(0xb7836e30, FUTEX_WAIT_PRIVATE, 0, {2, 927460405}) = -1 ETIMEDOUT (Connection timed out) > > > gettid() = 3050 > > > write(2, "libust[3050/3050]: Error: Timed o"..., 107libust[3050/3050]: Error: Timed out waiting for ltt-sessiond (in lttng_ust_init() at lttng-ust-comm.c:912) > > > ) = 107 > > > futex(0xb7836e80, FUTEX_WAIT_PRIVATE, 2, NULL > > > _______________________________________________ > > > lttng-dev mailing list > > > lttng-dev at lists.lttng.org > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > > > > > -- > > Mathieu Desnoyers > > Operating System Efficiency R&D Consultant > > EfficiOS Inc. > > http://www.efficios.com > > -- > Mathieu Desnoyers > Operating System Efficiency R&D Consultant > EfficiOS Inc. > http://www.efficios.com > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Tue Jun 12 11:32:57 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 12 Jun 2012 11:32:57 -0400 Subject: [lttng-dev] lttng-ust with --std=c99 -pedantic In-Reply-To: References: Message-ID: <20120612153257.GB14189@Krystal> Hi John, * John Steele Scott (toojays at toojays.net) wrote: > I want to add lttng-ust tracepoints to a program which builds with "--std=c99 -pedantic". Right now this does not work. > > Using the demo program as an example, if you enable --std=c99, the first issue looks like: > > jscott at saaz:~/src/lttng-ust/tests/demo$ ccache gcc -std=c99 -DHAVE_CONFIG_H -I. -I../.. -I../../include/lttng -I../../include -Wall -g -O2 -MT demo.o -MD -MP -MF .deps/demo.Tpo -c -o demo.o demo.c > In file included from demo.c:34:0: > ust_tests_demo.h: In function ?__tracepoint_cb_ust_tests_demo___starting?: > ust_tests_demo.h:27:23: warning: implicit declaration of function ?typeof? [-Wimplicit-function-declaration] > ust_tests_demo.h:27:218: error: expected ?;? before ?_________p1? > ust_tests_demo.h:27:395: error: ?_________p1? undeclared (first use in this function) > ust_tests_demo.h:27:395: note: each undeclared identifier is reported only once for each function it appears in > In file included from demo.c:34:0: > ust_tests_demo.h: In function ?__tracepoint_cb_ust_tests_demo___done?: > ust_tests_demo.h:35:214: error: expected ?;? before ?_________p1? > ust_tests_demo.h:35:383: error: ?_________p1? undeclared (first use in this function) > In file included from demo.c:35:0: > ust_tests_demo2.h: In function ?__tracepoint_cb_ust_tests_demo2___loop?: > ust_tests_demo2.h:27:299: error: expected ?;? before ?_________p1? > ust_tests_demo2.h:27:470: error: ?_________p1? undeclared (first use in this function) > In file included from demo.c:36:0: > ust_tests_demo3.h: In function ?__tracepoint_cb_ust_tests_demo3___done?: > ust_tests_demo3.h:27:215: error: expected ?;? before ?_________p1? > ust_tests_demo3.h:27:386: error: ?_________p1? undeclared (first use in this function) > > This can be easily resolved by using __typeof__() instead of typeof(). Then I can build with --std=c99. But adding -pedantic still fails: I pushed a fix for this: commit 6423c3134bf07d4a7db56f69f2c79b540a79c4f1 Author: Mathieu Desnoyers Date: Mon Jun 11 10:15:25 2012 -0400 Fix c99 compatibility: use __typeof__ instead of typeof in public headers Reported-by: John Steele Scott Signed-off-by: Mathieu Desnoyers I also had issues with (void) arg parameters with tracepoints, so pushed: commit 4495dd39c05739d0fb2bc463b7c093d2459ce2b6 Author: Mathieu Desnoyers Date: Tue Jun 12 11:22:46 2012 -0400 Fix: support -std=c99 in tracepoint macros Signed-off-by: Mathieu Desnoyers I had also to fix userspace RCU library, with: commit e51500edbd9919cee53bc85cbb4b22cd4786fc42 Author: Mathieu Desnoyers Date: Tue Jun 12 11:24:31 2012 -0400 Fix c99 compatibility: use __asm__ and __volatile__ in public headers Signed-off-by: Mathieu Desnoyers commit bdffa73aa208ad5f1e5b3a3cb6cbf86ac6996559 Author: Mathieu Desnoyers Date: Mon Jun 11 10:16:35 2012 -0400 Fix c99 compatibility: use __typeof__ instead of typeof in public headers Reported-by: John Steele Scott Signed-off-by: Mathieu Desnoyers > > jscott at saaz:~/src/lttng-ust/tests/demo$ ccache gcc -std=c99 -Dtypeof=__typeof__ -pedantic -DHAVE_CONFIG_H -I. -I../.. -I../../include/lttng -I../../include -Wall -g -O2 -MT demo.o -MD -MP -MF .deps/demo.Tpo -c -o demo.o demo.c > In file included from ust_tests_demo.h:25:0, > from demo.c:34: > ../../include/lttng/tracepoint.h: In function ?__tracepoints__init?: > ../../include/lttng/tracepoint.h:251:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > ../../include/lttng/tracepoint.h:255:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > ../../include/lttng/tracepoint.h:260:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > ../../include/lttng/tracepoint.h:264:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > ../../include/lttng/tracepoint.h:268:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > In file included from demo.c:34:0: > ust_tests_demo.h: In function ?__tracepoint_cb_ust_tests_demo___starting?: > ust_tests_demo.h:27:161: warning: ISO C forbids braced-groups within expressions [-pedantic] > ust_tests_demo.h:27:549: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > In file included from demo.c:34:0: > ust_tests_demo.h: In function ?__tracepoint_cb_ust_tests_demo___done?: > ust_tests_demo.h:35:161: warning: ISO C forbids braced-groups within expressions [-pedantic] > ust_tests_demo.h:35:537: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > In file included from demo.c:35:0: > ust_tests_demo2.h: In function ?__tracepoint_cb_ust_tests_demo2___loop?: > ust_tests_demo2.h:27:245: warning: ISO C forbids braced-groups within expressions [-pedantic] > ust_tests_demo2.h:27:624: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > In file included from demo.c:36:0: > ust_tests_demo3.h: In function ?__tracepoint_cb_ust_tests_demo3___done?: > ust_tests_demo3.h:27:161: warning: ISO C forbids braced-groups within expressions [-pedantic] > ust_tests_demo3.h:27:540: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] I see these warnings, the only thing that currently fails is due to BYTE_ORDER and BIG_ENDIAN not being defined. By adding: -DBYTE_ORDER=__BYTE_ORDER -DBIG_ENDIAN=__BIG_ENDIAN my tests/hello program, with a Makefile.am modified to do: hello_CFLAGS = -Werror=old-style-definition --std=c99 -pedantic prints many pedantic warnings, and fails with: ././ust_tests_hello.h:55:1: error: zero or negative size array ?__event_fields___ust_tests_hello___tptest_sighandler? which seems to be caused by my event with 0 fields, which try to create an array of length 0. I'll look into this one. Thanks, Mathieu > > This is with 4.6.1-9ubuntu3 on Ubuntu 11.10, lttng-ust master 5a821c. > > Would it be particularly difficult to make the lttng-ust tracepoints compatible with programs built with -pedantic? > > cheers, > > John > > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From dgoulet at efficios.com Tue Jun 12 11:34:39 2012 From: dgoulet at efficios.com (David Goulet) Date: Tue, 12 Jun 2012 11:34:39 -0400 Subject: [lttng-dev] Lttng start failed In-Reply-To: <20120612140548.GA13917@Krystal> References: <47D610AD9C485E458630BA38C324D3B60113FB00F9EA@EXGMBX01.sasken.com> <20120608174226.GA23339@Krystal> <47D610AD9C485E458630BA38C324D3B60113FB00FA00@EXGMBX01.sasken.com> <20120609123251.GA14280@Krystal> <47D610AD9C485E458630BA38C324D3B60113FB0E22D1@EXGMBX01.sasken.com> <20120612140548.GA13917@Krystal> Message-ID: <4FD7618F.8090407@efficios.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Pavan, The output below is normal and should be working fine. However, the problem is that you have _NO_ debug message after the "lttng start". That command should at least produce 10+ lines of messages after this last line: "DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1423]" Can you enlight me and tell me if by doing "lttng create newsessiom" and "lttng enable-event -a -k", you have output messages coming out of the session daemon in -vvv mode ? If NOT, you are talking to another daemon! A quick "ps faux | grep lttng-sessiond" could help clear that question. If only one daemon is running and stuck at the above debug message, the next step is to tell me on what the daemon is waiting on using either "strace -p" or "gdb" also. There is 6 threads so having the strace -p output for all of them would help me greatly. Thanks! David On 12/06/12 10:05 AM, Mathieu Desnoyers wrote: > * Pavan Anumula (pavan.anumula at sasken.com) wrote: >> Hi Mathieu, Still I am unable to start tracing, And I am facing the same >> start errors. Please kindly help me on this. This time I loaded the >> modules by modprobe( not with insmod as I did before), Then I run the >> command " arm-none-linux-gnueabi-lttng-sessiond -vvv " (before >> arm-none-linux-gnueabi-lttng-sessiond -d), > > When using "arm-none-linux-gnueabi-lttng-sessiond -vvv", you should _not_ > run "arm-none-linux-gnueabi-lttng-sessiond -d". > > >> The below is the Output, And It got hanged overthere. > > This is normal: the sessiond is waiting for applications to connect. It > does not hang, it just waits. > > David, can you follow up on this issue ? It seems to be sessiond-related. > > Thanks, > > Mathieu > > >> Later when I started lttng start I am acing the same erros below: >> root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng start LTTng: >> Failure to write metadata to buffers (timeout) Error: Starting kernel >> trace failed >> >> ************************************************************************************************ >> >> DEBUG3: Creating LTTng run directory: /var/run/lttng [in create_lttng_rundir() at main.c:4315] >> DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in >> main() at main.c:4543] DEBUG2: Kernel consumer cmd path: >> /var/run/lttng/kconsumerd/command [in main() at main.c:4545] DEBUG1: >> Client socket path /var/run/lttng/client-lttng-sessiond [in main() at >> main.c:4592] DEBUG1: Application socket path >> /var/run/lttng/apps-lttng-sessiond [in main() at main.c:4593] DEBUG1: >> LTTng run directory path: /var/run/lttng [in main() at main.c:4594] >> DEBUG2: UST consumer 32 bits err path: >> /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] DEBUG2: >> UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command [in >> main() at main.c:4605] DEBUG2: UST consumer 64 bits err path: >> /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] DEBUG2: >> UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command [in >> main() at main.c:4616] DEBUG3: Created hashtable size 4 at 0x4a080 of >> type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable >> size 4 at 0x4a168 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: >> Creating consumer directory: /var/run/lttng/kconsumerd [in >> set_consumer_sockets() at main.c:4357] DEBUG1: Modprobe successfully >> lttng-tracer [in modprobe_lttng_control() at modprobe.c:163] DEBUG2: >> Kernel tracer version validated (major version 2) [in >> kernel_validate_version() at kernel.c:675] DEBUG1: Modprobe successfully >> lttng-ftrace [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: >> Modprobe successfully lttng-kprobes [in modprobe_lttng_data() at >> modprobe.c:199] DEBUG1: Modprobe successfully lttng-kretprobes [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully >> lttng-lib-ring-buffer [in modprobe_lttng_data() at modprobe.c:199] >> DEBUG1: Modprobe successfully lttng-ring-buffer-client-discard [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully >> lttng-ring-buffer-client-overwrite [in modprobe_lttng_data() at >> modprobe.c:199] DEBUG1: Modprobe successfully >> lttng-ring-buffer-metadata-client [in modprobe_lttng_data() at >> modprobe.c:199] DEBUG1: Modprobe successfully >> lttng-ring-buffer-client-mmap-discard [in modprobe_lttng_data() at >> modprobe.c:199] DEBUG1: Modprobe successfully >> lttng-ring-buffer-client-mmap-overwrite [in modprobe_lttng_data() at >> modprobe.c:199] DEBUG1: Modprobe successfully >> lttng-ring-buffer-metadata-mmap-client [in modprobe_lttng_data() at >> modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-lttng [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully >> lttng-types [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >> successfully lttng-probe-block [in modprobe_lttng_data() at >> modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-irq [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully >> lttng-probe-kvm [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: >> Modprobe successfully lttng-probe-sched [in modprobe_lttng_data() at >> modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-signal [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully >> lttng-probe-statedump [in modprobe_lttng_data() at modprobe.c:199] >> DEBUG1: Modprobe successfully lttng-probe-timer [in modprobe_lttng_data() >> at modprobe.c:199] DEBUG1: Kernel tracer fd 6 [in init_kernel_tracer() at >> main.c:1887] DEBUG2: Creating consumer directory: >> /var/run/lttng/ustconsumerd64 [in set_consumer_sockets() at main.c:4357] >> DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd32 [in >> set_consumer_sockets() at main.c:4357] DEBUG1: Signal handler set for >> SIGTERM, SIGPIPE and SIGINT [in set_signal_handler() at main.c:4449] >> DEBUG1: All permissions are set [in set_permissions() at main.c:4250] >> DEBUG1: poll set max size set to 65535 [in compat_poll_set_max_size() at >> compat-poll.c:196] DEBUG1: [thread] Manage application started [in >> thread_manage_apps() at main.c:1179] DEBUG1: fd 3 of 1 added to pollfd >> [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 13 of 2 added to >> pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: Apps thread >> polling on 2 fds [in thread_manage_apps() at main.c:1200] DEBUG1: Thread >> manage kernel started [in thread_manage_kernel() at main.c:876] DEBUG1: >> fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] >> DEBUG1: fd 11 of 2 added to pollfd [in compat_poll_add() at >> compat-poll.c:91] DEBUG1: Updating kernel poll set [in >> update_kernel_poll() at main.c:748] DEBUG1: Thread kernel polling on 2 >> fds [in thread_manage_kernel() at main.c:905] DEBUG1: [thread] Manage >> application registration started [in thread_registration_apps() at >> main.c:1392] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at >> compat-poll.c:91] DEBUG1: fd 10 of 2 added to pollfd [in >> compat_poll_add() at compat-poll.c:91] DEBUG1: Notifying applications of >> session daemon state: 1 [in notify_ust_apps() at main.c:687] DEBUG1: >> [thread] Dispatch UST command started [in >> thread_dispatch_ust_registration() at main.c:1324] DEBUG1: Futex n to 1 >> prepare done [in futex_nto1_prepare() at futex.c:73] DEBUG1: Woken up but >> nothing in the UST command queue [in thread_dispatch_ust_registration() >> at main.c:1334] DEBUG1: [thread] Manage client started [in >> thread_manage_clients() at main.c:3794] DEBUG1: fd 3 of 1 added to pollfd >> [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 9 of 2 added to >> pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: Accepting >> client command ... [in thread_manage_clients() at main.c:3826] DEBUG1: >> Got the wait shm fd 15 [in get_wait_shm() at shm.c:117] DEBUG1: Futex >> wait update active 1 [in futex_wait_update() at futex.c:62] DEBUG1: >> Accepting application registration [in thread_registration_apps() at >> main.c:1423] >> ******************************************************************************************************** >> >> Am I missing anything?? >> >> Thanks in Advance, Pavan >> >> >> ________________________________________ From: Mathieu Desnoyers >> [mathieu.desnoyers at efficios.com] Sent: Monday, June 11, 2012 7:24 PM To: >> Pavan Anumula Cc: lttng-dev at lists.lttng.org Subject: Re: [lttng-dev] >> Lttng start failed >> >> * Pavan Anumula (pavan.anumula at sasken.com) wrote: >>> Hi Mathieu, >>> >>> Please find the info below as per your comments >>> >>> >>> ########## Output of command "arm-none-linux-gnueabi-lttng-sessiond >>> -vvv " , After loading the modules ####### >>> >>> DEBUG3: Creating LTTng run directory: /var/run/lttng [in >>> create_lttng_rundir() at main.c:4315] DEBUG2: Kernel consumer err path: >>> /var/run/lttng/kconsumerd/error [in main() at main.c:4543] DEBUG2: >>> Kernel consumer cmd path: /var/run/lttng/kconsumerd/command [in main() >>> at main.c:4545] DEBUG1: Client socket path >>> /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] DEBUG1: >>> Application socket path /var/run/lttng/apps-lttng-sessiond [in main() >>> at main.c:4593] DEBUG1: LTTng run directory path: /var/run/lttng [in >>> main() at main.c:4594] DEBUG2: UST consumer 32 bits err path: >>> /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] DEBUG2: >>> UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command >>> [in main() at main.c:4605] DEBUG2: UST consumer 64 bits err path: >>> /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] DEBUG2: >>> UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command >>> [in main() at main.c:4616] DEBUG3: Created hashtable size 4 at 0x4a080 >>> of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created >>> hashtable size 4 at 0x4a168 of type 1 [in lttng_ht_new() at >>> hashtable.c:96] DEBUG2: Creating consumer directory: >>> /var/run/lttng/kconsumerd [in set_consumer_sockets() at main.c:4357] >>> FATAL: Module lttng_tracer not found. Error: Unable to load module >>> lttng-tracer >> >> Hrm. You want to do kernel tracing, but modprobe cannot find the lttng >> kernel tracer modules. You might want to run depmod -a or something like >> that on your target, and ensure that modprobe works properly. >> >> Thanks, >> >> Mathieu >> >>> DEBUG2: Kernel tracer version validated (major version 2) [in >>> kernel_validate_version() at kernel.c:675] DEBUG1: Modprobe >>> successfully lttng-ftrace [in modprobe_lttng_data() at modprobe.c:199] >>> DEBUG1: Modprobe successfully lttng-kprobes [in modprobe_lttng_data() >>> at modprobe.c:199] DEBUG1: Modprobe successfully lttng-kretprobes [in >>> modprobe_lttng_data() at modprobe.c:199] FATAL: Module >>> lttng_lib_ring_buffer not found. Error: Unable to load module >>> lttng-lib-ring-buffer FATAL: Module lttng_ring_buffer_client_discard >>> not found. Error: Unable to load module >>> lttng-ring-buffer-client-discard FATAL: Module >>> lttng_ring_buffer_client_overwrite not found. Error: Unable to load >>> module lttng-ring-buffer-client-overwrite FATAL: Module >>> lttng_ring_buffer_metadata_client not found. Error: Unable to load >>> module lttng-ring-buffer-metadata-client FATAL: Module >>> lttng_ring_buffer_client_mmap_discard not found. Error: Unable to load >>> module lttng-ring-buffer-client-mmap-discard FATAL: Module >>> lttng_ring_buffer_client_mmap_overwrite not found. Error: Unable to >>> load module lttng-ring-buffer-client-mmap-overwrite FATAL: Module >>> lttng_ring_buffer_metadata_mmap_client not found. Error: Unable to load >>> module lttng-ring-buffer-metadata-mmap-client FATAL: Module >>> lttng_probe_lttng not found. Error: Unable to load module >>> lttng-probe-lttng DEBUG1: Modprobe successfully lttng-types [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully >>> lttng-probe-block [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: >>> Modprobe successfully lttng-probe-irq [in modprobe_lttng_data() at >>> modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-kvm [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully >>> lttng-probe-sched [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: >>> Modprobe successfully lttng-probe-signal [in modprobe_lttng_data() at >>> modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-statedump [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully >>> lttng-probe-timer [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: >>> Kernel tracer fd 6 [in init_kernel_tracer() at main.c:1887] DEBUG2: >>> Creating consumer directory: /var/run/lttng/ustconsumerd64 [in >>> set_consumer_sockets() at main.c:4357] DEBUG2: Creating consumer >>> directory: /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() at >>> main.c:4357] DEBUG1: Signal handler set for SIGTERM, SIGPIPE and SIGINT >>> [in set_signal_handler() at main.c:4449] DEBUG1: All permissions are >>> set [in set_permissions() at main.c:4250] DEBUG1: poll set max size set >>> to 65535 [in compat_poll_set_max_size() at compat-poll.c:196] DEBUG1: >>> [thread] Manage client started [in thread_manage_clients() at >>> main.c:3794] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at >>> compat-poll.c:91] DEBUG1: fd 9 of 2 added to pollfd [in >>> compat_poll_add() at compat-poll.c:91] DEBUG1: Accepting client command >>> ... [in thread_manage_clients() at main.c:3826] DEBUG1: [thread] >>> Dispatch UST command started [in thread_dispatch_ust_registration() at >>> main.c:1324] DEBUG1: Futex n to 1 prepare done [in futex_nto1_prepare() >>> at futex.c:73] DEBUG1: Woken up but nothing in the UST command queue >>> [in thread_dispatch_ust_registration() at main.c:1334] DEBUG1: Thread >>> manage kernel started [in thread_manage_kernel() at main.c:876] DEBUG1: >>> fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] >>> DEBUG1: fd 11 of 2 added to pollfd [in compat_poll_add() at >>> compat-poll.c:91] DEBUG1: Updating kernel poll set [in >>> update_kernel_poll() at main.c:748] DEBUG1: Thread kernel polling on 2 >>> fds [in thread_manage_kernel() at main.c:905] DEBUG1: [thread] Manage >>> application started [in thread_manage_apps() at main.c:1179] DEBUG1: fd >>> 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] >>> DEBUG1: fd 13 of 2 added to pollfd [in compat_poll_add() at >>> compat-poll.c:91] DEBUG1: Apps thread polling on 2 fds [in >>> thread_manage_apps() at main.c:1200] DEBUG1: [thread] Manage >>> application registration started [in thread_registration_apps() at >>> main.c:1392] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at >>> compat-poll.c:91] DEBUG1: fd 10 of 2 added to pollfd [in >>> compat_poll_add() at compat-poll.c:91] DEBUG1: Notifying applications >>> of session daemon state: 1 [in notify_ust_apps() at main.c:687] DEBUG1: >>> Got the wait shm fd 15 [in get_wait_shm() at shm.c:117] DEBUG1: Futex >>> wait update active 1 [in futex_wait_update() at futex.c:62] DEBUG1: >>> Accepting application registration [in thread_registration_apps() at >>> main.c:1423] >>> >>> >>> >>> #### Output of command "arm-none-linux-gnueabi-lttng-sessiond -vvv " >>> before loading the modules ############# >>> >>> DEBUG3: Creating LTTng run directory: /var/run/lttng [in >>> create_lttng_rundir() at main.c:4315] DEBUG2: Kernel consumer err path: >>> /var/run/lttng/kconsumerd/error [in main() at main.c:4543] DEBUG2: >>> Kernel consumer cmd path: /var/run/lttng/kconsumerd/command [in main() >>> at main.c:4545] DEBUG1: Client socket path >>> /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] DEBUG1: >>> Application socket path /var/run/lttng/apps-lttng-sessiond [in main() >>> at main.c:4593] DEBUG1: LTTng run directory path: /var/run/lttng [in >>> main() at main.c:4594] DEBUG2: UST consumer 32 bits err path: >>> /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] DEBUG2: >>> UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command >>> [in main() at main.c:4605] DEBUG2: UST consumer 64 bits err path: >>> /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] DEBUG2: >>> UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command >>> [in main() at main.c:4616] DEBUG3: Created hashtable size 4 at 0x4a080 >>> of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created >>> hashtable size 4 at 0x4a168 of type 1 [in lttng_ht_new() at >>> hashtable.c:96] DEBUG2: Creating consumer directory: >>> /var/run/lttng/kconsumerd [in set_consumer_sockets() at main.c:4357] >>> FATAL: Module lttng_tracer not found. Error: Unable to load module >>> lttng-tracer DEBUG1: Failed to open /proc/lttng [in >>> init_kernel_tracer() at main.c:1871] Error: Unable to remove module >>> lttng-tracer Warning: No kernel tracer available DEBUG2: Creating >>> consumer directory: /var/run/lttng/ustconsumerd64 [in >>> set_consumer_sockets() at main.c:4357] DEBUG2: Creating consumer >>> directory: /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() at >>> main.c:4357] DEBUG1: Signal handler set for SIGTERM, SIGPIPE and SIGINT >>> [in set_signal_handler() at main.c:4449] DEBUG1: All permissions are >>> set [in set_permissions() at main.c:4250] DEBUG1: poll set max size set >>> to 65535 [in compat_poll_set_max_size() at compat-poll.c:196] DEBUG1: >>> [thread] Manage application started [in thread_manage_apps() at >>> main.c:1179] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at >>> compat-poll.c:91] DEBUG1: fd 12 of 2 added to pollfd [in >>> compat_poll_add() at compat-poll.c:91] DEBUG1: Apps thread polling on 2 >>> fds [in thread_manage_apps() at main.c:1200] DEBUG1: Thread manage >>> kernel started [in thread_manage_kernel() at main.c:876] DEBUG1: fd 3 >>> of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: >>> fd 10 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] >>> DEBUG1: Updating kernel poll set [in update_kernel_poll() at >>> main.c:748] DEBUG1: Thread kernel polling on 2 fds [in >>> thread_manage_kernel() at main.c:905] DEBUG1: [thread] Manage >>> application registration started [in thread_registration_apps() at >>> main.c:1392] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at >>> compat-poll.c:91] DEBUG1: fd 9 of 2 added to pollfd [in >>> compat_poll_add() at compat-poll.c:91] DEBUG1: Notifying applications >>> of session daemon state: 1 [in notify_ust_apps() at main.c:687] DEBUG1: >>> [thread] Dispatch UST command started [in >>> thread_dispatch_ust_registration() at main.c:1324] DEBUG1: Futex n to 1 >>> prepare done [in futex_nto1_prepare() at futex.c:73] DEBUG1: Woken up >>> but nothing in the UST command queue [in >>> thread_dispatch_ust_registration() at main.c:1334] DEBUG1: [thread] >>> Manage client started [in thread_manage_clients() at main.c:3794] >>> DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at >>> compat-poll.c:91] DEBUG1: fd 8 of 2 added to pollfd [in >>> compat_poll_add() at compat-poll.c:91] DEBUG1: Accepting client command >>> ... [in thread_manage_clients() at main.c:3826] DEBUG1: Got the wait >>> shm fd 14 [in get_wait_shm() at shm.c:117] DEBUG1: Futex wait update >>> active 1 [in futex_wait_update() at futex.c:62] DEBUG1: Accepting >>> application registration [in thread_registration_apps() at >>> main.c:1423] >>> >>> >>> Regards, Pavan >>> >>> -----Original Message----- From: Mathieu Desnoyers >>> [mailto:mathieu.desnoyers at efficios.com] Sent: Saturday, June 09, 2012 >>> 6:03 PM To: Pavan Anumula Cc: lttng-dev at lists.lttng.org Subject: Re: >>> [lttng-dev] Lttng start failed >>> >>> * Pavan Anumula (pavan.anumula at sasken.com) wrote: >>>> Hi Mathue, >>>> >>>> Thanks for the quick reply, >>>> >>>> After inserting lttng modules , I had given the command >>>> "arm-none-linux-gnueabi-lttng-sessiond -vvv" as you said, Below is >>>> the output where there are error messages. Please help me in >>>> resolving the issue, SO that I can catch kernel and user traces. >>>> >>>> >>>> root at arago:/usr/lttng/modules# arm-none-linux-gnueabi-lttng-sessiond >>>> -d root at arago:/usr/lttng/modules# >>>> arm-none-linux-gnueabi-lttng-sessiond -vvv DEBUG3: Creating LTTng run >>>> directory: /var/run/lttng [in create_lttng_rundir() at main.c:4315] >>>> DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error >>>> [in main() at main.c:4543] DEBUG2: Kernel consumer cmd path: >>>> /var/run/lttng/kconsumerd/command [in main() at main.c:4545] DEBUG1: >>>> Client socket path /var/run/lttng/client-lttng-sessiond [in main() at >>>> main.c:4592] DEBUG1: Application socket path >>>> /var/run/lttng/apps-lttng-sessiond [in main() at main.c:4593] DEBUG1: >>>> LTTng run directory path: /var/run/lttng [in main() at main.c:4594] >>>> DEBUG2: UST consumer 32 bits err path: >>>> /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] >>>> DEBUG2: UST consumer 32 bits cmd path: >>>> /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] >>>> DEBUG2: UST consumer 64 bits err path: >>>> /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] >>>> DEBUG2: UST consumer 64 bits cmd path: >>>> /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] >>>> Error: Already running daemon. >>> >>> Please kill the lttng-sessiond that is already started (the one with >>> -d). Instead of that one, run the sessiond with: >>> >>> lttng-sessiond -vvv >>> >>> (don't run lttng-sessiond -d before) >>> >>> Thanks, >>> >>> Mathieu >>> >>>> >>>> >>>> >>>> >>>> Thanks in advance, Pavan >>>> >>>> -----Original Message----- From: Mathieu Desnoyers >>>> [mailto:mathieu.desnoyers at efficios.com] Sent: Friday, June 08, 2012 >>>> 11:12 PM To: Pavan Anumula Cc: lttng-dev at lists.lttng.org Subject: Re: >>>> [lttng-dev] Lttng start failed >>>> >>>> * Pavan Anumula (pavan.anumula at sasken.com) wrote: >>>>> Hi , >>>>> >>>>> I am new to LTTng usage, I am trying to use LTTng 2.0.1 and >>>>> LTTng-modules-2.0.2 for ARM(omapL138), with Linux kernel 2.6.33 >>>>> wit RT-29 patch on it. >>>>> >>>>> After enabling all the kernel events I am facing the below errors. >>>>> I loaded all the modules manually by insmod. >>>>> >>>>> >>>>> >>>>> root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng create >>>>> newsessiom Session newsessiom created. Traces will be written in >>>>> /home/root/lttng-traces/newsessiom-20110325-154922 >>>>> >>>>> root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng >>>>> enable-event -a --kernel All kernel events are enabled in channel >>>>> channel0 >>>>> >>>>> root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng start >>>>> LTTng: Failure to write metadata to buffers (timeout) Error: >>>>> Starting kernel trace failed >>>>> >>>>> Please kindly help me on this. >>>> >>>> I think you should look into the lttng-sessiond --help : >>>> >>>> --consumerd32-path PATH Specify path for the 32-bit UST consumer >>>> daemon binary --consumerd32-libdir PATH Specify path for the 32-bit >>>> UST consumer daemon libraries --consumerd64-path PATH Specify >>>> path for the 64-bit UST consumer daemon binary --consumerd64-libdir >>>> PATH Specify path for the 64-bit UST consumer daemon libraries >>>> >>>> options. My guess is that lttng-sessiond is not able to find the >>>> consumerd binary files, maybe due to a rename or because they have >>>> been moved after install. >>>> >>>> One more thing that might help is to launch the lttng-sessiond with >>>> "-vvv" : it will provide verbose output and let us know where things >>>> fall apart. >>>> >>>> Thanks, >>>> >>>> Mathieu >>>> >>>> >>>>> >>>>> >>>>> Thanks in advance, Pavan >>>>> >>>>> ________________________________ SASKEN BUSINESS DISCLAIMER: This >>>>> message may contain confidential, proprietary or legally privileged >>>>> information. In case you are not the original intended Recipient of >>>>> the message, you must not, directly or indirectly, use, disclose, >>>>> distribute, print, or copy any part of this message and you are >>>>> requested to delete it and inform the sender. Any views expressed >>>>> in this message are those of the individual sender unless otherwise >>>>> stated. Nothing contained in this message shall be construed as an >>>>> offer or acceptance of any offer by Sasken Communication >>>>> Technologies Limited ("Sasken") unless sent with that express >>>>> intent and with due authority of Sasken. Sasken has taken enough >>>>> precautions to prevent the spread of viruses. However the company >>>>> accepts no liability for any damage caused by any virus transmitted >>>>> by this email. Read Disclaimer at >>>>> http://www.sasken.com/extras/mail_disclaimer.html >>>> >>>>> _______________________________________________ lttng-dev mailing >>>>> list lttng-dev at lists.lttng.org >>>>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev >>>> >>>> >>>> -- Mathieu Desnoyers Operating System Efficiency R&D Consultant >>>> EfficiOS Inc. http://www.efficios.com >>> >>> -- Mathieu Desnoyers Operating System Efficiency R&D Consultant >>> EfficiOS Inc. http://www.efficios.com >> >> -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS >> Inc. http://www.efficios.com > -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBAgAGBQJP12GMAAoJEELoaioR9I02+q8IAKw/ot4BmP6xibXM/VnNaRdP 60S1KEynFPxDmO9JS3u3x+r4UiDaGWrP4+rEfgAeUeMQcOj3ItHZLyNhp+F8A8+j Dj0q9YFNBLsANzHzAk9VsVlL/myYaeW7ysNGl1yxhL9Vc/Px3cmxaof79smxMGcQ dnMKf6Gk+eNL4IPrXQIxF0cvVK85PI2xaDA1SiiKQaULUiqUBebn2gtGr1l8oFTw FT1xryh4wRw4FUmqngw54SYPiYpZm1y/VUzTWB3Zs9hYPcvEum22wrhAaWNkoUkk ixBjE+HHIVRBzZatDymE35tI4+Ci6z9J3uRmJVC97x2qQz2F4nl5Yw3qRmm1PVM= =3VyT -----END PGP SIGNATURE----- From mburton at ciena.com Tue Jun 12 11:44:53 2012 From: mburton at ciena.com (Burton, Michael) Date: Tue, 12 Jun 2012 11:44:53 -0400 Subject: [lttng-dev] UST Hanging at Tracepoint In-Reply-To: <20120612141344.GC13917@Krystal> References: <3F56B905CF2083408E8482044F20428AEF604D22@ONWVEXCHMB01.ciena.com> <3F56B905CF2083408E8482044F20428AEF78BC79@ONWVEXCHMB01.ciena.com> <20120612140925.GB13917@Krystal> <20120612141344.GC13917@Krystal> Message-ID: <3F56B905CF2083408E8482044F20428AEF7A0AFE@ONWVEXCHMB01.ciena.com> I put a fprintf at the beginning of main() and it never prints. -----Original Message----- From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] Sent: Tuesday, June 12, 2012 10:14 AM To: Burton, Michael; dgoulet at efficios.com Cc: lttng-dev at lists.lttng.org Subject: Re: [lttng-dev] UST Hanging at Tracepoint * Mathieu Desnoyers (mathieu.desnoyers at efficios.com) wrote: > Hi Michael, > > Please note that you should be able to run UST with either (or both) of: > > - root lttng-sessiond, for system-wide tracing > - per-user lttng-sessiond, for per-user tracing. > > It is entirely normal that one of the two UST threads within the > application can't find an active sessiond if, for instance, you have > just a root (and no per-user) sessiond running. > > So although I'm glad that you got things working, I fail to understand > your explanation: it should have worked fine with the root sessiond, > even if no ~/.lttng exists. > > David, can you look into this ? hrm, further insight: is your program _really_ hang ? Try to make sure you put a fprintf(stderr, ".....") at the beginning of your main(). Looking once more at your detailed UST output indicates that the per-user UST thread tells you that it cannot see any running sessiond. So yes, this thread blocks, but the thread talking to the root sessiond can very well be working fine, and your application might also be running fine. This should be double-checked. Thanks, Mathieu > > Thanks, > > Mathieu > > * Burton, Michael (mburton at ciena.com) wrote: > > Mathieu, > > > > I figured out the problem. When we start sessiond on our system as root, the /.lttng/ folder is not created, along with its contents. UST timed out since there was no sock_path. I was able to get UST working once I got sessiond running. > > > > Thanks. > > Michael > > > > ________________________________________ > > From: Mathieu Desnoyers [mathieu.desnoyers at efficios.com] > > Sent: 08 June 2012 21:38 > > To: Burton, Michael > > Cc: lttng-dev at lists.lttng.org > > Subject: Re: [lttng-dev] UST Hanging at Tracepoint > > > > * Burton, Michael (mburton at ciena.com) wrote: > > > For further insight, I've attached the strace from the hanging > > > program. I also upgraded LTTng-UST to 2.0.3 and userspace-rcu to > > > 0.7.3. > > > > Haven't had time to look at it yet, but could you also provide the > > output of: > > > > LTTNG_UST_DEBUG=1 youprogname > > > > (starting your UST instrumented program with LTTNG_UST_DEBUG=1 env. var. > > set) > > > > Thanks! > > > > Mathieu > > > > > > > > Michael > > > > > > From: Burton, Michael [mailto:mburton at ciena.com] > > > Sent: Thursday, June 07, 2012 2:16 PM > > > To: lttng-dev at lists.lttng.org > > > Subject: [lttng-dev] UST Hanging at Tracepoint > > > > > > I am working on getting LTTng 2.0 working in our 2.6.35 kernel. I have kernel tracing working but I'm having problems with UST. > > > > > > I have a program with a dynamic UST tracepoint. When I run the program with all UST events enabled (lttng enable-event -a -u) the program hangs on the tracepoint. The last line from the UST debug is: > > > > > > libust[6184/6185]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) > > > > > > Any insight into why this is happening? I've attached the UST and lttng-sessiond debug generated by the program. > > > > > > I am running the following: > > > lttng-modules-2.6.32 (found through this mailing list) > > > lttng-tools-2.0.1 > > > lttng-ust-2.0.2 > > > userspace-rcu-0.7.0 > > > > > > Thanks, > > > Michael > > > > Content-Description: strace_debug.txt > > > execve("/ciena/bin/idp", ["idp", "help"], [/* 11 vars */]) = 0 > > > brk(0) = 0x804e000 > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb783c000 > > > access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) > > > open("/etc/ld.so.cache", O_RDONLY) = 3 > > > fstat64(3, {st_mode=S_IFREG|0644, st_size=17420, ...}) = 0 > > > mmap2(NULL, 17420, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7837000 > > > close(3) = 0 > > > open("/usr/lib/liblttng-ust.so.0", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220A\0\0004\0\0\0$"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=209876, ...}) = 0 > > > mmap2(NULL, 212940, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7803000 > > > mmap2(0xb7832000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2e) = 0xb7832000 > > > close(3) = 0 > > > open("/usr/lib/liblttng-ust-ctl.so.0", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\200-\0\0004\0\0\0\274"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=140020, ...}) = 0 > > > mmap2(NULL, 142828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77e0000 > > > mmap2(0xb7802000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x21) = 0xb7802000 > > > close(3) = 0 > > > open("/usr/lib/liblttng-ust-fork.so.0", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\0\5\0\0004\0\0\0X"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=3864, ...}) = 0 > > > mmap2(NULL, 6820, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77de000 > > > mmap2(0xb77df000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77df000 > > > close(3) = 0 > > > open("/usr/lib/liblttng-ust-tracepoint.so.0", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220\v\0\0004\0\0\0008"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=16632, ...}) = 0 > > > mmap2(NULL, 19944, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d9000 > > > mmap2(0xb77dd000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3) = 0xb77dd000 > > > close(3) = 0 > > > open("/usr/lib/liblttng-ust-libc-wrapper.so.0", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\t\0\0004\0\0\0\364"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=9300, ...}) = 0 > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb77d8000 > > > mmap2(NULL, 12104, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d5000 > > > mmap2(0xb77d7000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77d7000 > > > close(3) = 0 > > > open("/usr/lib/liburcu.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d0000 > > > mmap2(0xb77d4000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77d4000 > > > close(3) = 0 > > > open("/usr/lib/liburcu-common.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340\5\0\0004\0\0\0\34"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=5084, ...}) = 0 > > > mmap2(NULL, 8032, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77ce000 > > > mmap2(0xb77cf000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77cf000 > > > close(3) = 0 > > > open("/usr/lib/liburcu-qsbr.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\22\0\0004\0\0\0\374"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77c9000 > > > mmap2(0xb77cd000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77cd000 > > > close(3) = 0 > > > open("/usr/lib/liburcu-mb.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77c4000 > > > mmap2(0xb77c8000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c8000 > > > close(3) = 0 > > > open("/usr/lib/liburcu-signal.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\23\0\0004\0\0\0\10"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18712, ...}) = 0 > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb77c3000 > > > mmap2(NULL, 21704, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77bd000 > > > mmap2(0xb77c2000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c2000 > > > close(3) = 0 > > > open("/usr/lib/liburcu-cds.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\300\f\0\0004\0\0\0t"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=26204, ...}) = 0 > > > mmap2(NULL, 25004, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77b6000 > > > mmap2(0xb77bc000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb77bc000 > > > close(3) = 0 > > > open("/usr/lib/liburcu-bp.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320\24\0\0004\0\0\0\360"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=19968, ...}) = 0 > > > mmap2(NULL, 23128, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77b0000 > > > mmap2(0xb77b5000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77b5000 > > > close(3) = 0 > > > open("/lib/libdl.so.2", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0 \n\0\0004\0\0\0<"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=9668, ...}) = 0 > > > mmap2(NULL, 12408, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77ac000 > > > mmap2(0xb77ae000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77ae000 > > > close(3) = 0 > > > open("/lib/libuuid.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0@\r\0\0004\0\0\0\350"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=12872, ...}) = 0 > > > mmap2(NULL, 15632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77a8000 > > > mmap2(0xb77ab000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2) = 0xb77ab000 > > > close(3) = 0 > > > open("/lib/libc.so.6", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220n\1\0004\0\0\0L"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=1347780, ...}) = 0 > > > mmap2(NULL, 1358120, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb765c000 > > > mprotect(0xb77a1000, 4096, PROT_NONE) = 0 > > > mmap2(0xb77a2000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x145) = 0xb77a2000 > > > mmap2(0xb77a5000, 10536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb77a5000 > > > close(3) = 0 > > > open("/lib/librt.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0p\30\0\0004\0\0\0\230"..., 512) = 512 > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb765b000 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=30616, ...}) = 0 > > > mmap2(NULL, 33364, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7652000 > > > mmap2(0xb7659000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb7659000 > > > close(3) = 0 > > > open("/lib/libpthread.so.0", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360J\0\0004\0\0\0\354"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=88420, ...}) = 0 > > > mmap2(NULL, 98828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7639000 > > > mmap2(0xb764e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14) = 0xb764e000 > > > mmap2(0xb7650000, 4620, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7650000 > > > close(3) = 0 > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7638000 > > > mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7636000 > > > set_thread_area({entry_number:-1 -> 6, base_addr:0xb7636d80, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 > > > mprotect(0xb764e000, 4096, PROT_READ) = 0 > > > mprotect(0xb7659000, 4096, PROT_READ) = 0 > > > mprotect(0xb77a2000, 8192, PROT_READ) = 0 > > > mprotect(0xb77ae000, 4096, PROT_READ) = 0 > > > mprotect(0xb785a000, 4096, PROT_READ) = 0 > > > munmap(0xb7837000, 17420) = 0 > > > set_tid_address(0xb7636de8) = 3050 > > > set_robust_list(0xb7636df0, 0xc) = 0 > > > futex(0xbfaaf560, FUTEX_WAKE_PRIVATE, 1) = 0 > > > futex(0xbfaaf560, 0x189 /* FUTEX_??? */, 1, NULL, bfaaf570) = -1 EAGAIN (Resource temporarily unavailable) > > > rt_sigaction(SIGRTMIN, {0xb763d4f0, [], SA_SIGINFO}, NULL, 8) = 0 > > > rt_sigaction(SIGRT_1, {0xb763d9c0, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0 > > > rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0 > > > getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0 > > > uname({sys="Linux", node="5410", ...}) = 0 > > > rt_sigaction(SIGUSR1, {0xb77bec0a, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0 > > > futex(0xb77af06c, FUTEX_WAKE_PRIVATE, 2147483647) = 0 > > > brk(0) = 0x804e000 > > > brk(0x806f000) = 0x806f000 > > > gettimeofday({1339187216, 880428}, NULL) = 0 > > > open("/dev/urandom", O_RDONLY|O_LARGEFILE) = 3 > > > fcntl64(3, F_GETFD) = 0 > > > fcntl64(3, F_SETFD, FD_CLOEXEC) = 0 > > > getuid32() = 0 > > > gettimeofday({1339187216, 889603}, NULL) = 0 > > > gettimeofday({1339187216, 894471}, NULL) = 0 > > > read(3, "\275\0051F\215\373\371\202\377kfb/\362\303N"..., 16) = 16 > > > clock_gettime(CLOCK_REALTIME, {1339187216, 902329405}) = 0 > > > getuid32() = 0 > > > geteuid32() = 0 > > > rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0 > > > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6e36000 > > > mprotect(0xb6e36000, 4096, PROT_NONE) = 0 > > > clone(child_stack=0xb7634d64, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0xb7635bd8, {entry_number:6, base_addr:0xb7635b70, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xb7635bd8) = 3051 > > > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6636000 > > > mprotect(0xb6636000, 4096, PROT_NONE) = 0 > > > clone(child_stack=0xb6e34d64, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0xb6e35bd8, {entry_number:6, base_addr:0xb6e35b70, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xb6e35bd8) = 3052 > > > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > > > gettimeofday({1339187216, 974869}, NULL) = 0 > > > futex(0xb7836e30, FUTEX_WAIT_PRIVATE, 0, {2, 927460405}) = -1 ETIMEDOUT (Connection timed out) > > > gettid() = 3050 > > > write(2, "libust[3050/3050]: Error: Timed o"..., 107libust[3050/3050]: Error: Timed out waiting for ltt-sessiond (in lttng_ust_init() at lttng-ust-comm.c:912) > > > ) = 107 > > > futex(0xb7836e80, FUTEX_WAIT_PRIVATE, 2, NULL > > > _______________________________________________ > > > lttng-dev mailing list > > > lttng-dev at lists.lttng.org > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > > > > > -- > > Mathieu Desnoyers > > Operating System Efficiency R&D Consultant > > EfficiOS Inc. > > http://www.efficios.com > > -- > Mathieu Desnoyers > Operating System Efficiency R&D Consultant > EfficiOS Inc. > http://www.efficios.com > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From dgoulet at efficios.com Tue Jun 12 11:51:26 2012 From: dgoulet at efficios.com (David Goulet) Date: Tue, 12 Jun 2012 11:51:26 -0400 Subject: [lttng-dev] UST Hanging at Tracepoint In-Reply-To: <20120612140925.GB13917@Krystal> References: <3F56B905CF2083408E8482044F20428AEF604D22@ONWVEXCHMB01.ciena.com> <3F56B905CF2083408E8482044F20428AEF78BC79@ONWVEXCHMB01.ciena.com> <20120612140925.GB13917@Krystal> Message-ID: <4FD7657E.8080603@efficios.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hey! On 12/06/12 10:09 AM, Mathieu Desnoyers wrote: > Hi Michael, > > Please note that you should be able to run UST with either (or both) of: > > - root lttng-sessiond, for system-wide tracing - per-user lttng-sessiond, > for per-user tracing. > > It is entirely normal that one of the two UST threads within the > application can't find an active sessiond if, for instance, you have just a > root (and no per-user) sessiond running. > > So although I'm glad that you got things working, I fail to understand your > explanation: it should have worked fine with the root sessiond, even if no > ~/.lttng exists. A root session daemon does NOT create a ".lttng" directory. For system wide, we use /var/run/lttng/*. However, UST tries the $HOME/.lttng/ and /var/run/lttng for the global session daemon. - From the strace output, the timeout occurs on the FUTEX_WAIT and not the socket... However, there might be a missing thread in that output I think... futex(0xb7836e30, FUTEX_WAIT_PRIVATE, 0, {2, 927460405}) = -1 ETIMEDOUT (Connection timed out) David > > David, can you look into this ? > > Thanks, > > Mathieu > > * Burton, Michael (mburton at ciena.com) wrote: >> Mathieu, >> >> I figured out the problem. When we start sessiond on our system as root, >> the /.lttng/ folder is not created, along with its contents. UST timed >> out since there was no sock_path. I was able to get UST working once I >> got sessiond running. >> >> Thanks. Michael >> >> ________________________________________ From: Mathieu Desnoyers >> [mathieu.desnoyers at efficios.com] Sent: 08 June 2012 21:38 To: Burton, >> Michael Cc: lttng-dev at lists.lttng.org Subject: Re: [lttng-dev] UST >> Hanging at Tracepoint >> >> * Burton, Michael (mburton at ciena.com) wrote: >>> For further insight, I've attached the strace from the hanging program. >>> I also upgraded LTTng-UST to 2.0.3 and userspace-rcu to 0.7.3. >> >> Haven't had time to look at it yet, but could you also provide the output >> of: >> >> LTTNG_UST_DEBUG=1 youprogname >> >> (starting your UST instrumented program with LTTNG_UST_DEBUG=1 env. var. >> set) >> >> Thanks! >> >> Mathieu >> >>> >>> Michael >>> >>> From: Burton, Michael [mailto:mburton at ciena.com] Sent: Thursday, June >>> 07, 2012 2:16 PM To: lttng-dev at lists.lttng.org Subject: [lttng-dev] UST >>> Hanging at Tracepoint >>> >>> I am working on getting LTTng 2.0 working in our 2.6.35 kernel. I have >>> kernel tracing working but I'm having problems with UST. >>> >>> I have a program with a dynamic UST tracepoint. When I run the program >>> with all UST events enabled (lttng enable-event -a -u) the program >>> hangs on the tracepoint. The last line from the UST debug is: >>> >>> libust[6184/6185]: Info: sessiond not accepting connections to local >>> apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) >>> >>> Any insight into why this is happening? I've attached the UST and >>> lttng-sessiond debug generated by the program. >>> >>> I am running the following: lttng-modules-2.6.32 (found through this >>> mailing list) lttng-tools-2.0.1 lttng-ust-2.0.2 userspace-rcu-0.7.0 >>> >>> Thanks, Michael >> >> Content-Description: strace_debug.txt >>> execve("/ciena/bin/idp", ["idp", "help"], [/* 11 vars */]) = 0 brk(0) >>> = 0x804e000 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, >>> MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb783c000 >>> access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or >>> directory) open("/etc/ld.so.cache", O_RDONLY) = 3 fstat64(3, >>> {st_mode=S_IFREG|0644, st_size=17420, ...}) = 0 mmap2(NULL, 17420, >>> PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7837000 close(3) >>> = 0 open("/usr/lib/liblttng-ust.so.0", O_RDONLY) = 3 read(3, >>> "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220A\0\0004\0\0\0$"..., >>> 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=209876, ...}) = 0 >>> mmap2(NULL, 212940, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, >>> 0) = 0xb7803000 mmap2(0xb7832000, 20480, PROT_READ|PROT_WRITE, >>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2e) = 0xb7832000 close(3) >>> = 0 open("/usr/lib/liblttng-ust-ctl.so.0", O_RDONLY) = 3 read(3, >>> "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\200-\0\0004\0\0\0\274"..., >>> 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=140020, ...}) = 0 >>> mmap2(NULL, 142828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, >>> 0) = 0xb77e0000 mmap2(0xb7802000, 4096, PROT_READ|PROT_WRITE, >>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x21) = 0xb7802000 close(3) >>> = 0 open("/usr/lib/liblttng-ust-fork.so.0", O_RDONLY) = 3 read(3, >>> "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\0\5\0\0004\0\0\0X"..., >>> 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=3864, ...}) = 0 >>> mmap2(NULL, 6820, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) >>> = 0xb77de000 mmap2(0xb77df000, 4096, PROT_READ|PROT_WRITE, >>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77df000 close(3) >>> = 0 open("/usr/lib/liblttng-ust-tracepoint.so.0", O_RDONLY) = 3 read(3, >>> "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220\v\0\0004\0\0\0008"..., >>> 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=16632, ...}) = 0 >>> mmap2(NULL, 19944, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, >>> 0) = 0xb77d9000 mmap2(0xb77dd000, 4096, PROT_READ|PROT_WRITE, >>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3) = 0xb77dd000 close(3) >>> = 0 open("/usr/lib/liblttng-ust-libc-wrapper.so.0", O_RDONLY) = 3 >>> read(3, >>> "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\t\0\0004\0\0\0\364"..., >>> 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=9300, ...}) = 0 >>> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, >>> 0) = 0xb77d8000 mmap2(NULL, 12104, PROT_READ|PROT_EXEC, >>> MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d5000 mmap2(0xb77d7000, 4096, >>> PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = >>> 0xb77d7000 close(3) = 0 >>> open("/usr/lib/liburcu.so.1", O_RDONLY) = 3 read(3, >>> "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"..., >>> 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 >>> mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, >>> 0) = 0xb77d0000 mmap2(0xb77d4000, 4096, PROT_READ|PROT_WRITE, >>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77d4000 close(3) >>> = 0 open("/usr/lib/liburcu-common.so.1", O_RDONLY) = 3 read(3, >>> "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340\5\0\0004\0\0\0\34"..., >>> 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=5084, ...}) = 0 >>> mmap2(NULL, 8032, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) >>> = 0xb77ce000 mmap2(0xb77cf000, 4096, PROT_READ|PROT_WRITE, >>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77cf000 close(3) >>> = 0 open("/usr/lib/liburcu-qsbr.so.1", O_RDONLY) = 3 read(3, >>> "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\22\0\0004\0\0\0\374"..., >>> 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 >>> mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, >>> 0) = 0xb77c9000 mmap2(0xb77cd000, 4096, PROT_READ|PROT_WRITE, >>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77cd000 close(3) >>> = 0 open("/usr/lib/liburcu-mb.so.1", O_RDONLY) = 3 read(3, >>> "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"..., >>> 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 >>> mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, >>> 0) = 0xb77c4000 mmap2(0xb77c8000, 4096, PROT_READ|PROT_WRITE, >>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c8000 close(3) >>> = 0 open("/usr/lib/liburcu-signal.so.1", O_RDONLY) = 3 read(3, >>> "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\23\0\0004\0\0\0\10"..., >>> 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=18712, ...}) = 0 >>> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, >>> 0) = 0xb77c3000 mmap2(NULL, 21704, PROT_READ|PROT_EXEC, >>> MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77bd000 mmap2(0xb77c2000, 4096, >>> PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = >>> 0xb77c2000 close(3) = 0 >>> open("/usr/lib/liburcu-cds.so.1", O_RDONLY) = 3 read(3, >>> "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\300\f\0\0004\0\0\0t"..., >>> 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=26204, ...}) = 0 >>> mmap2(NULL, 25004, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, >>> 0) = 0xb77b6000 mmap2(0xb77bc000, 4096, PROT_READ|PROT_WRITE, >>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb77bc000 close(3) >>> = 0 open("/usr/lib/liburcu-bp.so.1", O_RDONLY) = 3 read(3, >>> "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320\24\0\0004\0\0\0\360"..., >>> 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=19968, ...}) = 0 >>> mmap2(NULL, 23128, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, >>> 0) = 0xb77b0000 mmap2(0xb77b5000, 4096, PROT_READ|PROT_WRITE, >>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77b5000 close(3) >>> = 0 open("/lib/libdl.so.2", O_RDONLY) = 3 read(3, >>> "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0 \n\0\0004\0\0\0<"..., >>> 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=9668, ...}) = 0 >>> mmap2(NULL, 12408, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, >>> 0) = 0xb77ac000 mmap2(0xb77ae000, 8192, PROT_READ|PROT_WRITE, >>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77ae000 close(3) >>> = 0 open("/lib/libuuid.so.1", O_RDONLY) = 3 read(3, >>> "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0@\r\0\0004\0\0\0\350"..., >>> 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=12872, ...}) = 0 >>> mmap2(NULL, 15632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, >>> 0) = 0xb77a8000 mmap2(0xb77ab000, 4096, PROT_READ|PROT_WRITE, >>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2) = 0xb77ab000 close(3) >>> = 0 open("/lib/libc.so.6", O_RDONLY) = 3 read(3, >>> "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220n\1\0004\0\0\0L"..., >>> 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=1347780, ...}) = >>> 0 mmap2(NULL, 1358120, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, >>> 3, 0) = 0xb765c000 mprotect(0xb77a1000, 4096, PROT_NONE) = 0 >>> mmap2(0xb77a2000, 12288, PROT_READ|PROT_WRITE, >>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x145) = 0xb77a2000 >>> mmap2(0xb77a5000, 10536, PROT_READ|PROT_WRITE, >>> MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb77a5000 close(3) >>> = 0 open("/lib/librt.so.1", O_RDONLY) = 3 read(3, >>> "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0p\30\0\0004\0\0\0\230"..., >>> 512) = 512 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, >>> MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb765b000 fstat64(3, >>> {st_mode=S_IFREG|0755, st_size=30616, ...}) = 0 mmap2(NULL, 33364, >>> PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7652000 >>> mmap2(0xb7659000, 8192, PROT_READ|PROT_WRITE, >>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb7659000 close(3) >>> = 0 open("/lib/libpthread.so.0", O_RDONLY) = 3 read(3, >>> "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360J\0\0004\0\0\0\354"..., >>> 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=88420, ...}) = 0 >>> mmap2(NULL, 98828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, >>> 0) = 0xb7639000 mmap2(0xb764e000, 8192, PROT_READ|PROT_WRITE, >>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14) = 0xb764e000 >>> mmap2(0xb7650000, 4620, PROT_READ|PROT_WRITE, >>> MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7650000 close(3) >>> = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, >>> -1, 0) = 0xb7638000 mmap2(NULL, 8192, PROT_READ|PROT_WRITE, >>> MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7636000 >>> set_thread_area({entry_number:-1 -> 6, base_addr:0xb7636d80, >>> limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, >>> limit_in_pages:1, seg_not_present:0, useable:1}) = 0 >>> mprotect(0xb764e000, 4096, PROT_READ) = 0 mprotect(0xb7659000, 4096, >>> PROT_READ) = 0 mprotect(0xb77a2000, 8192, PROT_READ) = 0 >>> mprotect(0xb77ae000, 4096, PROT_READ) = 0 mprotect(0xb785a000, 4096, >>> PROT_READ) = 0 munmap(0xb7837000, 17420) = 0 >>> set_tid_address(0xb7636de8) = 3050 >>> set_robust_list(0xb7636df0, 0xc) = 0 futex(0xbfaaf560, >>> FUTEX_WAKE_PRIVATE, 1) = 0 futex(0xbfaaf560, 0x189 /* FUTEX_??? */, 1, >>> NULL, bfaaf570) = -1 EAGAIN (Resource temporarily unavailable) >>> rt_sigaction(SIGRTMIN, {0xb763d4f0, [], SA_SIGINFO}, NULL, 8) = 0 >>> rt_sigaction(SIGRT_1, {0xb763d9c0, [], SA_RESTART|SA_SIGINFO}, NULL, 8) >>> = 0 rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0 >>> getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = >>> 0 uname({sys="Linux", node="5410", ...}) = 0 rt_sigaction(SIGUSR1, >>> {0xb77bec0a, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0 futex(0xb77af06c, >>> FUTEX_WAKE_PRIVATE, 2147483647) = 0 brk(0) >>> = 0x804e000 brk(0x806f000) = 0x806f000 >>> gettimeofday({1339187216, 880428}, NULL) = 0 open("/dev/urandom", >>> O_RDONLY|O_LARGEFILE) = 3 fcntl64(3, F_GETFD) = 0 >>> fcntl64(3, F_SETFD, FD_CLOEXEC) = 0 getuid32() >>> = 0 gettimeofday({1339187216, 889603}, NULL) = 0 >>> gettimeofday({1339187216, 894471}, NULL) = 0 read(3, >>> "\275\0051F\215\373\371\202\377kfb/\362\303N"..., 16) = 16 >>> clock_gettime(CLOCK_REALTIME, {1339187216, 902329405}) = 0 getuid32() >>> = 0 geteuid32() = 0 >>> rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0 mmap2(NULL, >>> 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, >>> 0) = 0xb6e36000 mprotect(0xb6e36000, 4096, PROT_NONE) = 0 >>> clone(child_stack=0xb7634d64, >>> flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, >>> parent_tidptr=0xb7635bd8, {entry_number:6, base_addr:0xb7635b70, >>> limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, >>> limit_in_pages:1, seg_not_present:0, useable:1}, >>> child_tidptr=0xb7635bd8) = 3051 mmap2(NULL, 8388608, >>> PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = >>> 0xb6636000 mprotect(0xb6636000, 4096, PROT_NONE) = 0 >>> clone(child_stack=0xb6e34d64, >>> flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, >>> parent_tidptr=0xb6e35bd8, {entry_number:6, base_addr:0xb6e35b70, >>> limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, >>> limit_in_pages:1, seg_not_present:0, useable:1}, >>> child_tidptr=0xb6e35bd8) = 3052 rt_sigprocmask(SIG_SETMASK, [], NULL, >>> 8) = 0 gettimeofday({1339187216, 974869}, NULL) = 0 futex(0xb7836e30, >>> FUTEX_WAIT_PRIVATE, 0, {2, 927460405}) = -1 ETIMEDOUT (Connection timed >>> out) gettid() = 3050 write(2, >>> "libust[3050/3050]: Error: Timed o"..., 107libust[3050/3050]: Error: >>> Timed out waiting for ltt-sessiond (in lttng_ust_init() at >>> lttng-ust-comm.c:912) ) = 107 futex(0xb7836e80, FUTEX_WAIT_PRIVATE, 2, >>> NULL _______________________________________________ lttng-dev mailing >>> list lttng-dev at lists.lttng.org >>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev >> >> >> -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS >> Inc. http://www.efficios.com > -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBAgAGBQJP12V+AAoJEELoaioR9I02v3YH/A8x6NpChh1v3P0IxhHyMyQC sJnfeejvCAxLWCSRm8hLgvaX65HfZA8nOcS4oPUEtonZU+QZS51no1W4biDUePKx IPIzMy0+Fv5W/j6N0ocXgdP5lup2JiyPwqf13cj2ZC/n0BbGuuWrfpG4fYgwUaYG TJSB5FrTF4KB5aQ6cYaMbsfv25mYGUWPGTvFjI3PFYPQ9MA6oimxBSYDMONM5IeI 5cZDEI7sd3BEO5Vf3YmymdxCWNwbvEOfyEJZo/TgsCV4ljyxthGWNvB1/meNdOqg v8gaRyU29jXp/u58RP5fSuWrcpfQ4b4rRZ1dc1AXuUDD1vYBKrAY0t+3a1THJjE= =ghI+ -----END PGP SIGNATURE----- From mathieu.desnoyers at efficios.com Tue Jun 12 11:55:39 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 12 Jun 2012 11:55:39 -0400 Subject: [lttng-dev] lttng-ust with --std=c99 -pedantic In-Reply-To: <20120612153257.GB14189@Krystal> References: <20120612153257.GB14189@Krystal> Message-ID: <20120612155539.GA15859@Krystal> * Mathieu Desnoyers (mathieu.desnoyers at efficios.com) wrote: > Hi John, > > * John Steele Scott (toojays at toojays.net) wrote: > > I want to add lttng-ust tracepoints to a program which builds with "--std=c99 -pedantic". Right now this does not work. > > > > Using the demo program as an example, if you enable --std=c99, the first issue looks like: > > > > jscott at saaz:~/src/lttng-ust/tests/demo$ ccache gcc -std=c99 -DHAVE_CONFIG_H -I. -I../.. -I../../include/lttng -I../../include -Wall -g -O2 -MT demo.o -MD -MP -MF .deps/demo.Tpo -c -o demo.o demo.c > > In file included from demo.c:34:0: > > ust_tests_demo.h: In function ?__tracepoint_cb_ust_tests_demo___starting?: > > ust_tests_demo.h:27:23: warning: implicit declaration of function ?typeof? [-Wimplicit-function-declaration] > > ust_tests_demo.h:27:218: error: expected ?;? before ?_________p1? > > ust_tests_demo.h:27:395: error: ?_________p1? undeclared (first use in this function) > > ust_tests_demo.h:27:395: note: each undeclared identifier is reported only once for each function it appears in > > In file included from demo.c:34:0: > > ust_tests_demo.h: In function ?__tracepoint_cb_ust_tests_demo___done?: > > ust_tests_demo.h:35:214: error: expected ?;? before ?_________p1? > > ust_tests_demo.h:35:383: error: ?_________p1? undeclared (first use in this function) > > In file included from demo.c:35:0: > > ust_tests_demo2.h: In function ?__tracepoint_cb_ust_tests_demo2___loop?: > > ust_tests_demo2.h:27:299: error: expected ?;? before ?_________p1? > > ust_tests_demo2.h:27:470: error: ?_________p1? undeclared (first use in this function) > > In file included from demo.c:36:0: > > ust_tests_demo3.h: In function ?__tracepoint_cb_ust_tests_demo3___done?: > > ust_tests_demo3.h:27:215: error: expected ?;? before ?_________p1? > > ust_tests_demo3.h:27:386: error: ?_________p1? undeclared (first use in this function) > > > > This can be easily resolved by using __typeof__() instead of typeof(). Then I can build with --std=c99. But adding -pedantic still fails: > > I pushed a fix for this: > > commit 6423c3134bf07d4a7db56f69f2c79b540a79c4f1 > Author: Mathieu Desnoyers > Date: Mon Jun 11 10:15:25 2012 -0400 > > Fix c99 compatibility: use __typeof__ instead of typeof in public headers > > Reported-by: John Steele Scott > Signed-off-by: Mathieu Desnoyers > > I also had issues with (void) arg parameters with tracepoints, so > pushed: > > commit 4495dd39c05739d0fb2bc463b7c093d2459ce2b6 > Author: Mathieu Desnoyers > Date: Tue Jun 12 11:22:46 2012 -0400 > > Fix: support -std=c99 in tracepoint macros > > Signed-off-by: Mathieu Desnoyers > > I had also to fix userspace RCU library, with: > > > commit e51500edbd9919cee53bc85cbb4b22cd4786fc42 > Author: Mathieu Desnoyers > Date: Tue Jun 12 11:24:31 2012 -0400 > > Fix c99 compatibility: use __asm__ and __volatile__ in public headers > > Signed-off-by: Mathieu Desnoyers > > commit bdffa73aa208ad5f1e5b3a3cb6cbf86ac6996559 > Author: Mathieu Desnoyers > Date: Mon Jun 11 10:16:35 2012 -0400 > > Fix c99 compatibility: use __typeof__ instead of typeof in public headers > > Reported-by: John Steele Scott > Signed-off-by: Mathieu Desnoyers > > > > > > jscott at saaz:~/src/lttng-ust/tests/demo$ ccache gcc -std=c99 -Dtypeof=__typeof__ -pedantic -DHAVE_CONFIG_H -I. -I../.. -I../../include/lttng -I../../include -Wall -g -O2 -MT demo.o -MD -MP -MF .deps/demo.Tpo -c -o demo.o demo.c > > In file included from ust_tests_demo.h:25:0, > > from demo.c:34: > > ../../include/lttng/tracepoint.h: In function ?__tracepoints__init?: > > ../../include/lttng/tracepoint.h:251:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > > ../../include/lttng/tracepoint.h:255:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > > ../../include/lttng/tracepoint.h:260:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > > ../../include/lttng/tracepoint.h:264:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > > ../../include/lttng/tracepoint.h:268:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > > In file included from demo.c:34:0: > > ust_tests_demo.h: In function ?__tracepoint_cb_ust_tests_demo___starting?: > > ust_tests_demo.h:27:161: warning: ISO C forbids braced-groups within expressions [-pedantic] > > ust_tests_demo.h:27:549: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > > In file included from demo.c:34:0: > > ust_tests_demo.h: In function ?__tracepoint_cb_ust_tests_demo___done?: > > ust_tests_demo.h:35:161: warning: ISO C forbids braced-groups within expressions [-pedantic] > > ust_tests_demo.h:35:537: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > > In file included from demo.c:35:0: > > ust_tests_demo2.h: In function ?__tracepoint_cb_ust_tests_demo2___loop?: > > ust_tests_demo2.h:27:245: warning: ISO C forbids braced-groups within expressions [-pedantic] > > ust_tests_demo2.h:27:624: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > > In file included from demo.c:36:0: > > ust_tests_demo3.h: In function ?__tracepoint_cb_ust_tests_demo3___done?: > > ust_tests_demo3.h:27:161: warning: ISO C forbids braced-groups within expressions [-pedantic] > > ust_tests_demo3.h:27:540: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > > I see these warnings, the only thing that currently fails is due to > BYTE_ORDER and BIG_ENDIAN not being defined. By adding: > > -DBYTE_ORDER=__BYTE_ORDER -DBIG_ENDIAN=__BIG_ENDIAN > > my tests/hello program, with a Makefile.am modified to do: > > hello_CFLAGS = -Werror=old-style-definition --std=c99 -pedantic > > prints many pedantic warnings, and fails with: > > ././ust_tests_hello.h:55:1: error: zero or negative size array ?__event_fields___ust_tests_hello___tptest_sighandler? > > which seems to be caused by my event with 0 fields, which try to create > an array of length 0. > > I'll look into this one. Please try again with lttng-ust master HEAD, userspace-rcu master HEAD, and let me know if you experience problems compiling your instrumented application with --std=c99 -pedantic. Please note that the probe object (e.g. tp.c in the tests/hello program) needs to be compiled with gnu extensions, so you should not use --std=c99 for this specific object. Thanks, Mathieu > > Thanks, > > Mathieu > > > > > This is with 4.6.1-9ubuntu3 on Ubuntu 11.10, lttng-ust master 5a821c. > > > > Would it be particularly difficult to make the lttng-ust tracepoints compatible with programs built with -pedantic? > > > > cheers, > > > > John > > > > > > _______________________________________________ > > lttng-dev mailing list > > lttng-dev at lists.lttng.org > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > -- > Mathieu Desnoyers > Operating System Efficiency R&D Consultant > EfficiOS Inc. > http://www.efficios.com > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mburton at ciena.com Tue Jun 12 12:09:12 2012 From: mburton at ciena.com (Burton, Michael) Date: Tue, 12 Jun 2012 12:09:12 -0400 Subject: [lttng-dev] UST Hanging at Tracepoint In-Reply-To: <3F56B905CF2083408E8482044F20428AEF7A0AFE@ONWVEXCHMB01.ciena.com> References: <3F56B905CF2083408E8482044F20428AEF604D22@ONWVEXCHMB01.ciena.com> <3F56B905CF2083408E8482044F20428AEF78BC79@ONWVEXCHMB01.ciena.com> <20120612140925.GB13917@Krystal> <20120612141344.GC13917@Krystal> <3F56B905CF2083408E8482044F20428AEF7A0AFE@ONWVEXCHMB01.ciena.com> Message-ID: <3F56B905CF2083408E8482044F20428AEF7A0B4C@ONWVEXCHMB01.ciena.com> Also, I stepped through the code with gdb and things hang within the UST constructors and the main function of my application is never reached. -----Original Message----- From: Burton, Michael [mailto:mburton at ciena.com] Sent: Tuesday, June 12, 2012 11:45 AM To: Mathieu Desnoyers; dgoulet at efficios.com Cc: lttng-dev at lists.lttng.org Subject: Re: [lttng-dev] UST Hanging at Tracepoint I put a fprintf at the beginning of main() and it never prints. -----Original Message----- From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] Sent: Tuesday, June 12, 2012 10:14 AM To: Burton, Michael; dgoulet at efficios.com Cc: lttng-dev at lists.lttng.org Subject: Re: [lttng-dev] UST Hanging at Tracepoint * Mathieu Desnoyers (mathieu.desnoyers at efficios.com) wrote: > Hi Michael, > > Please note that you should be able to run UST with either (or both) of: > > - root lttng-sessiond, for system-wide tracing > - per-user lttng-sessiond, for per-user tracing. > > It is entirely normal that one of the two UST threads within the > application can't find an active sessiond if, for instance, you have > just a root (and no per-user) sessiond running. > > So although I'm glad that you got things working, I fail to understand > your explanation: it should have worked fine with the root sessiond, > even if no ~/.lttng exists. > > David, can you look into this ? hrm, further insight: is your program _really_ hang ? Try to make sure you put a fprintf(stderr, ".....") at the beginning of your main(). Looking once more at your detailed UST output indicates that the per-user UST thread tells you that it cannot see any running sessiond. So yes, this thread blocks, but the thread talking to the root sessiond can very well be working fine, and your application might also be running fine. This should be double-checked. Thanks, Mathieu > > Thanks, > > Mathieu > > * Burton, Michael (mburton at ciena.com) wrote: > > Mathieu, > > > > I figured out the problem. When we start sessiond on our system as root, the /.lttng/ folder is not created, along with its contents. UST timed out since there was no sock_path. I was able to get UST working once I got sessiond running. > > > > Thanks. > > Michael > > > > ________________________________________ > > From: Mathieu Desnoyers [mathieu.desnoyers at efficios.com] > > Sent: 08 June 2012 21:38 > > To: Burton, Michael > > Cc: lttng-dev at lists.lttng.org > > Subject: Re: [lttng-dev] UST Hanging at Tracepoint > > > > * Burton, Michael (mburton at ciena.com) wrote: > > > For further insight, I've attached the strace from the hanging > > > program. I also upgraded LTTng-UST to 2.0.3 and userspace-rcu to > > > 0.7.3. > > > > Haven't had time to look at it yet, but could you also provide the > > output of: > > > > LTTNG_UST_DEBUG=1 youprogname > > > > (starting your UST instrumented program with LTTNG_UST_DEBUG=1 env. var. > > set) > > > > Thanks! > > > > Mathieu > > > > > > > > Michael > > > > > > From: Burton, Michael [mailto:mburton at ciena.com] > > > Sent: Thursday, June 07, 2012 2:16 PM > > > To: lttng-dev at lists.lttng.org > > > Subject: [lttng-dev] UST Hanging at Tracepoint > > > > > > I am working on getting LTTng 2.0 working in our 2.6.35 kernel. I have kernel tracing working but I'm having problems with UST. > > > > > > I have a program with a dynamic UST tracepoint. When I run the program with all UST events enabled (lttng enable-event -a -u) the program hangs on the tracepoint. The last line from the UST debug is: > > > > > > libust[6184/6185]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) > > > > > > Any insight into why this is happening? I've attached the UST and lttng-sessiond debug generated by the program. > > > > > > I am running the following: > > > lttng-modules-2.6.32 (found through this mailing list) > > > lttng-tools-2.0.1 > > > lttng-ust-2.0.2 > > > userspace-rcu-0.7.0 > > > > > > Thanks, > > > Michael > > > > Content-Description: strace_debug.txt > > > execve("/ciena/bin/idp", ["idp", "help"], [/* 11 vars */]) = 0 > > > brk(0) = 0x804e000 > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb783c000 > > > access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) > > > open("/etc/ld.so.cache", O_RDONLY) = 3 > > > fstat64(3, {st_mode=S_IFREG|0644, st_size=17420, ...}) = 0 > > > mmap2(NULL, 17420, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7837000 > > > close(3) = 0 > > > open("/usr/lib/liblttng-ust.so.0", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220A\0\0004\0\0\0$"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=209876, ...}) = 0 > > > mmap2(NULL, 212940, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7803000 > > > mmap2(0xb7832000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2e) = 0xb7832000 > > > close(3) = 0 > > > open("/usr/lib/liblttng-ust-ctl.so.0", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\200-\0\0004\0\0\0\274"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=140020, ...}) = 0 > > > mmap2(NULL, 142828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77e0000 > > > mmap2(0xb7802000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x21) = 0xb7802000 > > > close(3) = 0 > > > open("/usr/lib/liblttng-ust-fork.so.0", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\0\5\0\0004\0\0\0X"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=3864, ...}) = 0 > > > mmap2(NULL, 6820, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77de000 > > > mmap2(0xb77df000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77df000 > > > close(3) = 0 > > > open("/usr/lib/liblttng-ust-tracepoint.so.0", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220\v\0\0004\0\0\0008"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=16632, ...}) = 0 > > > mmap2(NULL, 19944, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d9000 > > > mmap2(0xb77dd000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3) = 0xb77dd000 > > > close(3) = 0 > > > open("/usr/lib/liblttng-ust-libc-wrapper.so.0", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\t\0\0004\0\0\0\364"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=9300, ...}) = 0 > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb77d8000 > > > mmap2(NULL, 12104, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d5000 > > > mmap2(0xb77d7000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77d7000 > > > close(3) = 0 > > > open("/usr/lib/liburcu.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d0000 > > > mmap2(0xb77d4000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77d4000 > > > close(3) = 0 > > > open("/usr/lib/liburcu-common.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340\5\0\0004\0\0\0\34"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=5084, ...}) = 0 > > > mmap2(NULL, 8032, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77ce000 > > > mmap2(0xb77cf000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77cf000 > > > close(3) = 0 > > > open("/usr/lib/liburcu-qsbr.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\22\0\0004\0\0\0\374"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77c9000 > > > mmap2(0xb77cd000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77cd000 > > > close(3) = 0 > > > open("/usr/lib/liburcu-mb.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77c4000 > > > mmap2(0xb77c8000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c8000 > > > close(3) = 0 > > > open("/usr/lib/liburcu-signal.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\23\0\0004\0\0\0\10"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18712, ...}) = 0 > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb77c3000 > > > mmap2(NULL, 21704, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77bd000 > > > mmap2(0xb77c2000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c2000 > > > close(3) = 0 > > > open("/usr/lib/liburcu-cds.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\300\f\0\0004\0\0\0t"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=26204, ...}) = 0 > > > mmap2(NULL, 25004, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77b6000 > > > mmap2(0xb77bc000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb77bc000 > > > close(3) = 0 > > > open("/usr/lib/liburcu-bp.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320\24\0\0004\0\0\0\360"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=19968, ...}) = 0 > > > mmap2(NULL, 23128, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77b0000 > > > mmap2(0xb77b5000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77b5000 > > > close(3) = 0 > > > open("/lib/libdl.so.2", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0 \n\0\0004\0\0\0<"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=9668, ...}) = 0 > > > mmap2(NULL, 12408, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77ac000 > > > mmap2(0xb77ae000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77ae000 > > > close(3) = 0 > > > open("/lib/libuuid.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0@\r\0\0004\0\0\0\350"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=12872, ...}) = 0 > > > mmap2(NULL, 15632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77a8000 > > > mmap2(0xb77ab000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2) = 0xb77ab000 > > > close(3) = 0 > > > open("/lib/libc.so.6", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220n\1\0004\0\0\0L"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=1347780, ...}) = 0 > > > mmap2(NULL, 1358120, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb765c000 > > > mprotect(0xb77a1000, 4096, PROT_NONE) = 0 > > > mmap2(0xb77a2000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x145) = 0xb77a2000 > > > mmap2(0xb77a5000, 10536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb77a5000 > > > close(3) = 0 > > > open("/lib/librt.so.1", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0p\30\0\0004\0\0\0\230"..., 512) = 512 > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb765b000 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=30616, ...}) = 0 > > > mmap2(NULL, 33364, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7652000 > > > mmap2(0xb7659000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb7659000 > > > close(3) = 0 > > > open("/lib/libpthread.so.0", O_RDONLY) = 3 > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360J\0\0004\0\0\0\354"..., 512) = 512 > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=88420, ...}) = 0 > > > mmap2(NULL, 98828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7639000 > > > mmap2(0xb764e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14) = 0xb764e000 > > > mmap2(0xb7650000, 4620, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7650000 > > > close(3) = 0 > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7638000 > > > mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7636000 > > > set_thread_area({entry_number:-1 -> 6, base_addr:0xb7636d80, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 > > > mprotect(0xb764e000, 4096, PROT_READ) = 0 > > > mprotect(0xb7659000, 4096, PROT_READ) = 0 > > > mprotect(0xb77a2000, 8192, PROT_READ) = 0 > > > mprotect(0xb77ae000, 4096, PROT_READ) = 0 > > > mprotect(0xb785a000, 4096, PROT_READ) = 0 > > > munmap(0xb7837000, 17420) = 0 > > > set_tid_address(0xb7636de8) = 3050 > > > set_robust_list(0xb7636df0, 0xc) = 0 > > > futex(0xbfaaf560, FUTEX_WAKE_PRIVATE, 1) = 0 > > > futex(0xbfaaf560, 0x189 /* FUTEX_??? */, 1, NULL, bfaaf570) = -1 EAGAIN (Resource temporarily unavailable) > > > rt_sigaction(SIGRTMIN, {0xb763d4f0, [], SA_SIGINFO}, NULL, 8) = 0 > > > rt_sigaction(SIGRT_1, {0xb763d9c0, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0 > > > rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0 > > > getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0 > > > uname({sys="Linux", node="5410", ...}) = 0 > > > rt_sigaction(SIGUSR1, {0xb77bec0a, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0 > > > futex(0xb77af06c, FUTEX_WAKE_PRIVATE, 2147483647) = 0 > > > brk(0) = 0x804e000 > > > brk(0x806f000) = 0x806f000 > > > gettimeofday({1339187216, 880428}, NULL) = 0 > > > open("/dev/urandom", O_RDONLY|O_LARGEFILE) = 3 > > > fcntl64(3, F_GETFD) = 0 > > > fcntl64(3, F_SETFD, FD_CLOEXEC) = 0 > > > getuid32() = 0 > > > gettimeofday({1339187216, 889603}, NULL) = 0 > > > gettimeofday({1339187216, 894471}, NULL) = 0 > > > read(3, "\275\0051F\215\373\371\202\377kfb/\362\303N"..., 16) = 16 > > > clock_gettime(CLOCK_REALTIME, {1339187216, 902329405}) = 0 > > > getuid32() = 0 > > > geteuid32() = 0 > > > rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0 > > > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6e36000 > > > mprotect(0xb6e36000, 4096, PROT_NONE) = 0 > > > clone(child_stack=0xb7634d64, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0xb7635bd8, {entry_number:6, base_addr:0xb7635b70, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xb7635bd8) = 3051 > > > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6636000 > > > mprotect(0xb6636000, 4096, PROT_NONE) = 0 > > > clone(child_stack=0xb6e34d64, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0xb6e35bd8, {entry_number:6, base_addr:0xb6e35b70, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xb6e35bd8) = 3052 > > > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > > > gettimeofday({1339187216, 974869}, NULL) = 0 > > > futex(0xb7836e30, FUTEX_WAIT_PRIVATE, 0, {2, 927460405}) = -1 ETIMEDOUT (Connection timed out) > > > gettid() = 3050 > > > write(2, "libust[3050/3050]: Error: Timed o"..., 107libust[3050/3050]: Error: Timed out waiting for ltt-sessiond (in lttng_ust_init() at lttng-ust-comm.c:912) > > > ) = 107 > > > futex(0xb7836e80, FUTEX_WAIT_PRIVATE, 2, NULL > > > _______________________________________________ > > > lttng-dev mailing list > > > lttng-dev at lists.lttng.org > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > > > > > -- > > Mathieu Desnoyers > > Operating System Efficiency R&D Consultant > > EfficiOS Inc. > > http://www.efficios.com > > -- > Mathieu Desnoyers > Operating System Efficiency R&D Consultant > EfficiOS Inc. > http://www.efficios.com > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com _______________________________________________ lttng-dev mailing list lttng-dev at lists.lttng.org http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev From mathieu.desnoyers at efficios.com Tue Jun 12 12:25:26 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 12 Jun 2012 12:25:26 -0400 Subject: [lttng-dev] UST Hanging at Tracepoint In-Reply-To: <3F56B905CF2083408E8482044F20428AEF7A0B4C@ONWVEXCHMB01.ciena.com> References: <3F56B905CF2083408E8482044F20428AEF604D22@ONWVEXCHMB01.ciena.com> <3F56B905CF2083408E8482044F20428AEF78BC79@ONWVEXCHMB01.ciena.com> <20120612140925.GB13917@Krystal> <20120612141344.GC13917@Krystal> <3F56B905CF2083408E8482044F20428AEF7A0AFE@ONWVEXCHMB01.ciena.com> <3F56B905CF2083408E8482044F20428AEF7A0B4C@ONWVEXCHMB01.ciena.com> Message-ID: <20120612162526.GB16098@Krystal> Please build you app and lttng-ust with: make CFLAGS=-g (so with -O0 optimisation, and debug info) then, under gdb, please provide the full backtrace of each thread when stucked with: thread apply all bt full Thanks, Mathieu * Burton, Michael (mburton at ciena.com) wrote: > Also, I stepped through the code with gdb and things hang within the UST constructors and the main function of my application is never reached. > > -----Original Message----- > From: Burton, Michael [mailto:mburton at ciena.com] > Sent: Tuesday, June 12, 2012 11:45 AM > To: Mathieu Desnoyers; dgoulet at efficios.com > Cc: lttng-dev at lists.lttng.org > Subject: Re: [lttng-dev] UST Hanging at Tracepoint > > I put a fprintf at the beginning of main() and it never prints. > > -----Original Message----- > From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] > Sent: Tuesday, June 12, 2012 10:14 AM > To: Burton, Michael; dgoulet at efficios.com > Cc: lttng-dev at lists.lttng.org > Subject: Re: [lttng-dev] UST Hanging at Tracepoint > > * Mathieu Desnoyers (mathieu.desnoyers at efficios.com) wrote: > > Hi Michael, > > > > Please note that you should be able to run UST with either (or both) of: > > > > - root lttng-sessiond, for system-wide tracing > > - per-user lttng-sessiond, for per-user tracing. > > > > It is entirely normal that one of the two UST threads within the > > application can't find an active sessiond if, for instance, you have > > just a root (and no per-user) sessiond running. > > > > So although I'm glad that you got things working, I fail to understand > > your explanation: it should have worked fine with the root sessiond, > > even if no ~/.lttng exists. > > > > David, can you look into this ? > > hrm, further insight: is your program _really_ hang ? Try to make sure > you put a fprintf(stderr, ".....") at the beginning of your main(). > Looking once more at your detailed UST output indicates that the > per-user UST thread tells you that it cannot see any running sessiond. > So yes, this thread blocks, but the thread talking to the root sessiond > can very well be working fine, and your application might also be > running fine. This should be double-checked. > > Thanks, > > Mathieu > > > > > Thanks, > > > > Mathieu > > > > * Burton, Michael (mburton at ciena.com) wrote: > > > Mathieu, > > > > > > I figured out the problem. When we start sessiond on our system as root, the /.lttng/ folder is not created, along with its contents. UST timed out since there was no sock_path. I was able to get UST working once I got sessiond running. > > > > > > Thanks. > > > Michael > > > > > > ________________________________________ > > > From: Mathieu Desnoyers [mathieu.desnoyers at efficios.com] > > > Sent: 08 June 2012 21:38 > > > To: Burton, Michael > > > Cc: lttng-dev at lists.lttng.org > > > Subject: Re: [lttng-dev] UST Hanging at Tracepoint > > > > > > * Burton, Michael (mburton at ciena.com) wrote: > > > > For further insight, I've attached the strace from the hanging > > > > program. I also upgraded LTTng-UST to 2.0.3 and userspace-rcu to > > > > 0.7.3. > > > > > > Haven't had time to look at it yet, but could you also provide the > > > output of: > > > > > > LTTNG_UST_DEBUG=1 youprogname > > > > > > (starting your UST instrumented program with LTTNG_UST_DEBUG=1 env. var. > > > set) > > > > > > Thanks! > > > > > > Mathieu > > > > > > > > > > > Michael > > > > > > > > From: Burton, Michael [mailto:mburton at ciena.com] > > > > Sent: Thursday, June 07, 2012 2:16 PM > > > > To: lttng-dev at lists.lttng.org > > > > Subject: [lttng-dev] UST Hanging at Tracepoint > > > > > > > > I am working on getting LTTng 2.0 working in our 2.6.35 kernel. I have kernel tracing working but I'm having problems with UST. > > > > > > > > I have a program with a dynamic UST tracepoint. When I run the program with all UST events enabled (lttng enable-event -a -u) the program hangs on the tracepoint. The last line from the UST debug is: > > > > > > > > libust[6184/6185]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) > > > > > > > > Any insight into why this is happening? I've attached the UST and lttng-sessiond debug generated by the program. > > > > > > > > I am running the following: > > > > lttng-modules-2.6.32 (found through this mailing list) > > > > lttng-tools-2.0.1 > > > > lttng-ust-2.0.2 > > > > userspace-rcu-0.7.0 > > > > > > > > Thanks, > > > > Michael > > > > > > Content-Description: strace_debug.txt > > > > execve("/ciena/bin/idp", ["idp", "help"], [/* 11 vars */]) = 0 > > > > brk(0) = 0x804e000 > > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb783c000 > > > > access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) > > > > open("/etc/ld.so.cache", O_RDONLY) = 3 > > > > fstat64(3, {st_mode=S_IFREG|0644, st_size=17420, ...}) = 0 > > > > mmap2(NULL, 17420, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7837000 > > > > close(3) = 0 > > > > open("/usr/lib/liblttng-ust.so.0", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220A\0\0004\0\0\0$"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=209876, ...}) = 0 > > > > mmap2(NULL, 212940, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7803000 > > > > mmap2(0xb7832000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2e) = 0xb7832000 > > > > close(3) = 0 > > > > open("/usr/lib/liblttng-ust-ctl.so.0", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\200-\0\0004\0\0\0\274"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=140020, ...}) = 0 > > > > mmap2(NULL, 142828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77e0000 > > > > mmap2(0xb7802000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x21) = 0xb7802000 > > > > close(3) = 0 > > > > open("/usr/lib/liblttng-ust-fork.so.0", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\0\5\0\0004\0\0\0X"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=3864, ...}) = 0 > > > > mmap2(NULL, 6820, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77de000 > > > > mmap2(0xb77df000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77df000 > > > > close(3) = 0 > > > > open("/usr/lib/liblttng-ust-tracepoint.so.0", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220\v\0\0004\0\0\0008"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=16632, ...}) = 0 > > > > mmap2(NULL, 19944, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d9000 > > > > mmap2(0xb77dd000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3) = 0xb77dd000 > > > > close(3) = 0 > > > > open("/usr/lib/liblttng-ust-libc-wrapper.so.0", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\t\0\0004\0\0\0\364"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=9300, ...}) = 0 > > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb77d8000 > > > > mmap2(NULL, 12104, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d5000 > > > > mmap2(0xb77d7000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77d7000 > > > > close(3) = 0 > > > > open("/usr/lib/liburcu.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > > > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d0000 > > > > mmap2(0xb77d4000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77d4000 > > > > close(3) = 0 > > > > open("/usr/lib/liburcu-common.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340\5\0\0004\0\0\0\34"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=5084, ...}) = 0 > > > > mmap2(NULL, 8032, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77ce000 > > > > mmap2(0xb77cf000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77cf000 > > > > close(3) = 0 > > > > open("/usr/lib/liburcu-qsbr.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\22\0\0004\0\0\0\374"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > > > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77c9000 > > > > mmap2(0xb77cd000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77cd000 > > > > close(3) = 0 > > > > open("/usr/lib/liburcu-mb.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > > > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77c4000 > > > > mmap2(0xb77c8000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c8000 > > > > close(3) = 0 > > > > open("/usr/lib/liburcu-signal.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\23\0\0004\0\0\0\10"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18712, ...}) = 0 > > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb77c3000 > > > > mmap2(NULL, 21704, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77bd000 > > > > mmap2(0xb77c2000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c2000 > > > > close(3) = 0 > > > > open("/usr/lib/liburcu-cds.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\300\f\0\0004\0\0\0t"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=26204, ...}) = 0 > > > > mmap2(NULL, 25004, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77b6000 > > > > mmap2(0xb77bc000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb77bc000 > > > > close(3) = 0 > > > > open("/usr/lib/liburcu-bp.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320\24\0\0004\0\0\0\360"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=19968, ...}) = 0 > > > > mmap2(NULL, 23128, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77b0000 > > > > mmap2(0xb77b5000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77b5000 > > > > close(3) = 0 > > > > open("/lib/libdl.so.2", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0 \n\0\0004\0\0\0<"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=9668, ...}) = 0 > > > > mmap2(NULL, 12408, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77ac000 > > > > mmap2(0xb77ae000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77ae000 > > > > close(3) = 0 > > > > open("/lib/libuuid.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0@\r\0\0004\0\0\0\350"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=12872, ...}) = 0 > > > > mmap2(NULL, 15632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77a8000 > > > > mmap2(0xb77ab000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2) = 0xb77ab000 > > > > close(3) = 0 > > > > open("/lib/libc.so.6", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220n\1\0004\0\0\0L"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=1347780, ...}) = 0 > > > > mmap2(NULL, 1358120, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb765c000 > > > > mprotect(0xb77a1000, 4096, PROT_NONE) = 0 > > > > mmap2(0xb77a2000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x145) = 0xb77a2000 > > > > mmap2(0xb77a5000, 10536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb77a5000 > > > > close(3) = 0 > > > > open("/lib/librt.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0p\30\0\0004\0\0\0\230"..., 512) = 512 > > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb765b000 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=30616, ...}) = 0 > > > > mmap2(NULL, 33364, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7652000 > > > > mmap2(0xb7659000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb7659000 > > > > close(3) = 0 > > > > open("/lib/libpthread.so.0", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360J\0\0004\0\0\0\354"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=88420, ...}) = 0 > > > > mmap2(NULL, 98828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7639000 > > > > mmap2(0xb764e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14) = 0xb764e000 > > > > mmap2(0xb7650000, 4620, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7650000 > > > > close(3) = 0 > > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7638000 > > > > mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7636000 > > > > set_thread_area({entry_number:-1 -> 6, base_addr:0xb7636d80, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 > > > > mprotect(0xb764e000, 4096, PROT_READ) = 0 > > > > mprotect(0xb7659000, 4096, PROT_READ) = 0 > > > > mprotect(0xb77a2000, 8192, PROT_READ) = 0 > > > > mprotect(0xb77ae000, 4096, PROT_READ) = 0 > > > > mprotect(0xb785a000, 4096, PROT_READ) = 0 > > > > munmap(0xb7837000, 17420) = 0 > > > > set_tid_address(0xb7636de8) = 3050 > > > > set_robust_list(0xb7636df0, 0xc) = 0 > > > > futex(0xbfaaf560, FUTEX_WAKE_PRIVATE, 1) = 0 > > > > futex(0xbfaaf560, 0x189 /* FUTEX_??? */, 1, NULL, bfaaf570) = -1 EAGAIN (Resource temporarily unavailable) > > > > rt_sigaction(SIGRTMIN, {0xb763d4f0, [], SA_SIGINFO}, NULL, 8) = 0 > > > > rt_sigaction(SIGRT_1, {0xb763d9c0, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0 > > > > rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0 > > > > getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0 > > > > uname({sys="Linux", node="5410", ...}) = 0 > > > > rt_sigaction(SIGUSR1, {0xb77bec0a, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0 > > > > futex(0xb77af06c, FUTEX_WAKE_PRIVATE, 2147483647) = 0 > > > > brk(0) = 0x804e000 > > > > brk(0x806f000) = 0x806f000 > > > > gettimeofday({1339187216, 880428}, NULL) = 0 > > > > open("/dev/urandom", O_RDONLY|O_LARGEFILE) = 3 > > > > fcntl64(3, F_GETFD) = 0 > > > > fcntl64(3, F_SETFD, FD_CLOEXEC) = 0 > > > > getuid32() = 0 > > > > gettimeofday({1339187216, 889603}, NULL) = 0 > > > > gettimeofday({1339187216, 894471}, NULL) = 0 > > > > read(3, "\275\0051F\215\373\371\202\377kfb/\362\303N"..., 16) = 16 > > > > clock_gettime(CLOCK_REALTIME, {1339187216, 902329405}) = 0 > > > > getuid32() = 0 > > > > geteuid32() = 0 > > > > rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0 > > > > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6e36000 > > > > mprotect(0xb6e36000, 4096, PROT_NONE) = 0 > > > > clone(child_stack=0xb7634d64, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0xb7635bd8, {entry_number:6, base_addr:0xb7635b70, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xb7635bd8) = 3051 > > > > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6636000 > > > > mprotect(0xb6636000, 4096, PROT_NONE) = 0 > > > > clone(child_stack=0xb6e34d64, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0xb6e35bd8, {entry_number:6, base_addr:0xb6e35b70, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xb6e35bd8) = 3052 > > > > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > > > > gettimeofday({1339187216, 974869}, NULL) = 0 > > > > futex(0xb7836e30, FUTEX_WAIT_PRIVATE, 0, {2, 927460405}) = -1 ETIMEDOUT (Connection timed out) > > > > gettid() = 3050 > > > > write(2, "libust[3050/3050]: Error: Timed o"..., 107libust[3050/3050]: Error: Timed out waiting for ltt-sessiond (in lttng_ust_init() at lttng-ust-comm.c:912) > > > > ) = 107 > > > > futex(0xb7836e80, FUTEX_WAIT_PRIVATE, 2, NULL > > > > _______________________________________________ > > > > lttng-dev mailing list > > > > lttng-dev at lists.lttng.org > > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > > > > > > > > -- > > > Mathieu Desnoyers > > > Operating System Efficiency R&D Consultant > > > EfficiOS Inc. > > > http://www.efficios.com > > > > -- > > Mathieu Desnoyers > > Operating System Efficiency R&D Consultant > > EfficiOS Inc. > > http://www.efficios.com > > > > _______________________________________________ > > lttng-dev mailing list > > lttng-dev at lists.lttng.org > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > -- > Mathieu Desnoyers > Operating System Efficiency R&D Consultant > EfficiOS Inc. > http://www.efficios.com > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mburton at ciena.com Tue Jun 12 14:24:37 2012 From: mburton at ciena.com (Burton, Michael) Date: Tue, 12 Jun 2012 14:24:37 -0400 Subject: [lttng-dev] UST Hanging at Tracepoint In-Reply-To: <20120612162526.GB16098@Krystal> References: <3F56B905CF2083408E8482044F20428AEF604D22@ONWVEXCHMB01.ciena.com> <3F56B905CF2083408E8482044F20428AEF78BC79@ONWVEXCHMB01.ciena.com> <20120612140925.GB13917@Krystal> <20120612141344.GC13917@Krystal> <3F56B905CF2083408E8482044F20428AEF7A0AFE@ONWVEXCHMB01.ciena.com> <3F56B905CF2083408E8482044F20428AEF7A0B4C@ONWVEXCHMB01.ciena.com> <20120612162526.GB16098@Krystal> Message-ID: <3F56B905CF2083408E8482044F20428AEF7A0CE0@ONWVEXCHMB01.ciena.com> I've attached the full bt. Michael -----Original Message----- From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] Sent: Tuesday, June 12, 2012 12:25 PM To: Burton, Michael Cc: dgoulet at efficios.com; lttng-dev at lists.lttng.org Subject: Re: [lttng-dev] UST Hanging at Tracepoint Please build you app and lttng-ust with: make CFLAGS=-g (so with -O0 optimisation, and debug info) then, under gdb, please provide the full backtrace of each thread when stucked with: thread apply all bt full Thanks, Mathieu * Burton, Michael (mburton at ciena.com) wrote: > Also, I stepped through the code with gdb and things hang within the UST constructors and the main function of my application is never reached. > > -----Original Message----- > From: Burton, Michael [mailto:mburton at ciena.com] > Sent: Tuesday, June 12, 2012 11:45 AM > To: Mathieu Desnoyers; dgoulet at efficios.com > Cc: lttng-dev at lists.lttng.org > Subject: Re: [lttng-dev] UST Hanging at Tracepoint > > I put a fprintf at the beginning of main() and it never prints. > > -----Original Message----- > From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] > Sent: Tuesday, June 12, 2012 10:14 AM > To: Burton, Michael; dgoulet at efficios.com > Cc: lttng-dev at lists.lttng.org > Subject: Re: [lttng-dev] UST Hanging at Tracepoint > > * Mathieu Desnoyers (mathieu.desnoyers at efficios.com) wrote: > > Hi Michael, > > > > Please note that you should be able to run UST with either (or both) of: > > > > - root lttng-sessiond, for system-wide tracing > > - per-user lttng-sessiond, for per-user tracing. > > > > It is entirely normal that one of the two UST threads within the > > application can't find an active sessiond if, for instance, you have > > just a root (and no per-user) sessiond running. > > > > So although I'm glad that you got things working, I fail to understand > > your explanation: it should have worked fine with the root sessiond, > > even if no ~/.lttng exists. > > > > David, can you look into this ? > > hrm, further insight: is your program _really_ hang ? Try to make sure > you put a fprintf(stderr, ".....") at the beginning of your main(). > Looking once more at your detailed UST output indicates that the > per-user UST thread tells you that it cannot see any running sessiond. > So yes, this thread blocks, but the thread talking to the root sessiond > can very well be working fine, and your application might also be > running fine. This should be double-checked. > > Thanks, > > Mathieu > > > > > Thanks, > > > > Mathieu > > > > * Burton, Michael (mburton at ciena.com) wrote: > > > Mathieu, > > > > > > I figured out the problem. When we start sessiond on our system as root, the /.lttng/ folder is not created, along with its contents. UST timed out since there was no sock_path. I was able to get UST working once I got sessiond running. > > > > > > Thanks. > > > Michael > > > > > > ________________________________________ > > > From: Mathieu Desnoyers [mathieu.desnoyers at efficios.com] > > > Sent: 08 June 2012 21:38 > > > To: Burton, Michael > > > Cc: lttng-dev at lists.lttng.org > > > Subject: Re: [lttng-dev] UST Hanging at Tracepoint > > > > > > * Burton, Michael (mburton at ciena.com) wrote: > > > > For further insight, I've attached the strace from the hanging > > > > program. I also upgraded LTTng-UST to 2.0.3 and userspace-rcu to > > > > 0.7.3. > > > > > > Haven't had time to look at it yet, but could you also provide the > > > output of: > > > > > > LTTNG_UST_DEBUG=1 youprogname > > > > > > (starting your UST instrumented program with LTTNG_UST_DEBUG=1 env. var. > > > set) > > > > > > Thanks! > > > > > > Mathieu > > > > > > > > > > > Michael > > > > > > > > From: Burton, Michael [mailto:mburton at ciena.com] > > > > Sent: Thursday, June 07, 2012 2:16 PM > > > > To: lttng-dev at lists.lttng.org > > > > Subject: [lttng-dev] UST Hanging at Tracepoint > > > > > > > > I am working on getting LTTng 2.0 working in our 2.6.35 kernel. I have kernel tracing working but I'm having problems with UST. > > > > > > > > I have a program with a dynamic UST tracepoint. When I run the program with all UST events enabled (lttng enable-event -a -u) the program hangs on the tracepoint. The last line from the UST debug is: > > > > > > > > libust[6184/6185]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) > > > > > > > > Any insight into why this is happening? I've attached the UST and lttng-sessiond debug generated by the program. > > > > > > > > I am running the following: > > > > lttng-modules-2.6.32 (found through this mailing list) > > > > lttng-tools-2.0.1 > > > > lttng-ust-2.0.2 > > > > userspace-rcu-0.7.0 > > > > > > > > Thanks, > > > > Michael > > > > > > Content-Description: strace_debug.txt > > > > execve("/ciena/bin/idp", ["idp", "help"], [/* 11 vars */]) = 0 > > > > brk(0) = 0x804e000 > > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb783c000 > > > > access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) > > > > open("/etc/ld.so.cache", O_RDONLY) = 3 > > > > fstat64(3, {st_mode=S_IFREG|0644, st_size=17420, ...}) = 0 > > > > mmap2(NULL, 17420, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7837000 > > > > close(3) = 0 > > > > open("/usr/lib/liblttng-ust.so.0", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220A\0\0004\0\0\0$"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=209876, ...}) = 0 > > > > mmap2(NULL, 212940, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7803000 > > > > mmap2(0xb7832000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2e) = 0xb7832000 > > > > close(3) = 0 > > > > open("/usr/lib/liblttng-ust-ctl.so.0", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\200-\0\0004\0\0\0\274"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=140020, ...}) = 0 > > > > mmap2(NULL, 142828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77e0000 > > > > mmap2(0xb7802000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x21) = 0xb7802000 > > > > close(3) = 0 > > > > open("/usr/lib/liblttng-ust-fork.so.0", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\0\5\0\0004\0\0\0X"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=3864, ...}) = 0 > > > > mmap2(NULL, 6820, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77de000 > > > > mmap2(0xb77df000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77df000 > > > > close(3) = 0 > > > > open("/usr/lib/liblttng-ust-tracepoint.so.0", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220\v\0\0004\0\0\0008"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=16632, ...}) = 0 > > > > mmap2(NULL, 19944, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d9000 > > > > mmap2(0xb77dd000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3) = 0xb77dd000 > > > > close(3) = 0 > > > > open("/usr/lib/liblttng-ust-libc-wrapper.so.0", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\t\0\0004\0\0\0\364"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=9300, ...}) = 0 > > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb77d8000 > > > > mmap2(NULL, 12104, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d5000 > > > > mmap2(0xb77d7000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77d7000 > > > > close(3) = 0 > > > > open("/usr/lib/liburcu.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > > > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d0000 > > > > mmap2(0xb77d4000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77d4000 > > > > close(3) = 0 > > > > open("/usr/lib/liburcu-common.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340\5\0\0004\0\0\0\34"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=5084, ...}) = 0 > > > > mmap2(NULL, 8032, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77ce000 > > > > mmap2(0xb77cf000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77cf000 > > > > close(3) = 0 > > > > open("/usr/lib/liburcu-qsbr.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\22\0\0004\0\0\0\374"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > > > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77c9000 > > > > mmap2(0xb77cd000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77cd000 > > > > close(3) = 0 > > > > open("/usr/lib/liburcu-mb.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > > > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77c4000 > > > > mmap2(0xb77c8000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c8000 > > > > close(3) = 0 > > > > open("/usr/lib/liburcu-signal.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\23\0\0004\0\0\0\10"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18712, ...}) = 0 > > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb77c3000 > > > > mmap2(NULL, 21704, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77bd000 > > > > mmap2(0xb77c2000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c2000 > > > > close(3) = 0 > > > > open("/usr/lib/liburcu-cds.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\300\f\0\0004\0\0\0t"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=26204, ...}) = 0 > > > > mmap2(NULL, 25004, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77b6000 > > > > mmap2(0xb77bc000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb77bc000 > > > > close(3) = 0 > > > > open("/usr/lib/liburcu-bp.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320\24\0\0004\0\0\0\360"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=19968, ...}) = 0 > > > > mmap2(NULL, 23128, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77b0000 > > > > mmap2(0xb77b5000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77b5000 > > > > close(3) = 0 > > > > open("/lib/libdl.so.2", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0 \n\0\0004\0\0\0<"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=9668, ...}) = 0 > > > > mmap2(NULL, 12408, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77ac000 > > > > mmap2(0xb77ae000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77ae000 > > > > close(3) = 0 > > > > open("/lib/libuuid.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0@\r\0\0004\0\0\0\350"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=12872, ...}) = 0 > > > > mmap2(NULL, 15632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77a8000 > > > > mmap2(0xb77ab000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2) = 0xb77ab000 > > > > close(3) = 0 > > > > open("/lib/libc.so.6", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220n\1\0004\0\0\0L"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=1347780, ...}) = 0 > > > > mmap2(NULL, 1358120, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb765c000 > > > > mprotect(0xb77a1000, 4096, PROT_NONE) = 0 > > > > mmap2(0xb77a2000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x145) = 0xb77a2000 > > > > mmap2(0xb77a5000, 10536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb77a5000 > > > > close(3) = 0 > > > > open("/lib/librt.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0p\30\0\0004\0\0\0\230"..., 512) = 512 > > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb765b000 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=30616, ...}) = 0 > > > > mmap2(NULL, 33364, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7652000 > > > > mmap2(0xb7659000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb7659000 > > > > close(3) = 0 > > > > open("/lib/libpthread.so.0", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360J\0\0004\0\0\0\354"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=88420, ...}) = 0 > > > > mmap2(NULL, 98828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7639000 > > > > mmap2(0xb764e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14) = 0xb764e000 > > > > mmap2(0xb7650000, 4620, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7650000 > > > > close(3) = 0 > > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7638000 > > > > mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7636000 > > > > set_thread_area({entry_number:-1 -> 6, base_addr:0xb7636d80, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 > > > > mprotect(0xb764e000, 4096, PROT_READ) = 0 > > > > mprotect(0xb7659000, 4096, PROT_READ) = 0 > > > > mprotect(0xb77a2000, 8192, PROT_READ) = 0 > > > > mprotect(0xb77ae000, 4096, PROT_READ) = 0 > > > > mprotect(0xb785a000, 4096, PROT_READ) = 0 > > > > munmap(0xb7837000, 17420) = 0 > > > > set_tid_address(0xb7636de8) = 3050 > > > > set_robust_list(0xb7636df0, 0xc) = 0 > > > > futex(0xbfaaf560, FUTEX_WAKE_PRIVATE, 1) = 0 > > > > futex(0xbfaaf560, 0x189 /* FUTEX_??? */, 1, NULL, bfaaf570) = -1 EAGAIN (Resource temporarily unavailable) > > > > rt_sigaction(SIGRTMIN, {0xb763d4f0, [], SA_SIGINFO}, NULL, 8) = 0 > > > > rt_sigaction(SIGRT_1, {0xb763d9c0, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0 > > > > rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0 > > > > getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0 > > > > uname({sys="Linux", node="5410", ...}) = 0 > > > > rt_sigaction(SIGUSR1, {0xb77bec0a, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0 > > > > futex(0xb77af06c, FUTEX_WAKE_PRIVATE, 2147483647) = 0 > > > > brk(0) = 0x804e000 > > > > brk(0x806f000) = 0x806f000 > > > > gettimeofday({1339187216, 880428}, NULL) = 0 > > > > open("/dev/urandom", O_RDONLY|O_LARGEFILE) = 3 > > > > fcntl64(3, F_GETFD) = 0 > > > > fcntl64(3, F_SETFD, FD_CLOEXEC) = 0 > > > > getuid32() = 0 > > > > gettimeofday({1339187216, 889603}, NULL) = 0 > > > > gettimeofday({1339187216, 894471}, NULL) = 0 > > > > read(3, "\275\0051F\215\373\371\202\377kfb/\362\303N"..., 16) = 16 > > > > clock_gettime(CLOCK_REALTIME, {1339187216, 902329405}) = 0 > > > > getuid32() = 0 > > > > geteuid32() = 0 > > > > rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0 > > > > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6e36000 > > > > mprotect(0xb6e36000, 4096, PROT_NONE) = 0 > > > > clone(child_stack=0xb7634d64, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0xb7635bd8, {entry_number:6, base_addr:0xb7635b70, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xb7635bd8) = 3051 > > > > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6636000 > > > > mprotect(0xb6636000, 4096, PROT_NONE) = 0 > > > > clone(child_stack=0xb6e34d64, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0xb6e35bd8, {entry_number:6, base_addr:0xb6e35b70, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xb6e35bd8) = 3052 > > > > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > > > > gettimeofday({1339187216, 974869}, NULL) = 0 > > > > futex(0xb7836e30, FUTEX_WAIT_PRIVATE, 0, {2, 927460405}) = -1 ETIMEDOUT (Connection timed out) > > > > gettid() = 3050 > > > > write(2, "libust[3050/3050]: Error: Timed o"..., 107libust[3050/3050]: Error: Timed out waiting for ltt-sessiond (in lttng_ust_init() at lttng-ust-comm.c:912) > > > > ) = 107 > > > > futex(0xb7836e80, FUTEX_WAIT_PRIVATE, 2, NULL > > > > _______________________________________________ > > > > lttng-dev mailing list > > > > lttng-dev at lists.lttng.org > > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > > > > > > > > -- > > > Mathieu Desnoyers > > > Operating System Efficiency R&D Consultant > > > EfficiOS Inc. > > > http://www.efficios.com > > > > -- > > Mathieu Desnoyers > > Operating System Efficiency R&D Consultant > > EfficiOS Inc. > > http://www.efficios.com > > > > _______________________________________________ > > lttng-dev mailing list > > lttng-dev at lists.lttng.org > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > -- > Mathieu Desnoyers > Operating System Efficiency R&D Consultant > EfficiOS Inc. > http://www.efficios.com > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: full_bt.txt URL: From mburton at ciena.com Tue Jun 12 18:25:42 2012 From: mburton at ciena.com (Burton, Michael) Date: Tue, 12 Jun 2012 18:25:42 -0400 Subject: [lttng-dev] UST Hanging at Tracepoint In-Reply-To: <20120612162526.GB16098@Krystal> References: <3F56B905CF2083408E8482044F20428AEF604D22@ONWVEXCHMB01.ciena.com> <3F56B905CF2083408E8482044F20428AEF78BC79@ONWVEXCHMB01.ciena.com> <20120612140925.GB13917@Krystal> <20120612141344.GC13917@Krystal> <3F56B905CF2083408E8482044F20428AEF7A0AFE@ONWVEXCHMB01.ciena.com> <3F56B905CF2083408E8482044F20428AEF7A0B4C@ONWVEXCHMB01.ciena.com> <20120612162526.GB16098@Krystal> Message-ID: <3F56B905CF2083408E8482044F20428AEF7A0ED6@ONWVEXCHMB01.ciena.com> Mathieu, I think there is a deadlock scenario in UST, which has been causing my problem. sessiond is started as root: - creates global sockets ONLY - DOES NOT CREATE shm in $HOME/lttng-ust-wait- application linked against ust is run as root: - in lttng_ust_init constructor - ust_listener_thread (local_apps) - fails to connect to local_apps in $HOME/.lttng (as expected) - prev_connect_failed=1 - ust_unlock() - restart - wait_for_sessiond() --> - ust_lock() | - get_map_shm() | - get_wait_shm() DEADLOCK - shm_open() FAILS (not created by sessiond when run by root) | - fork() (trying to create shared memory itself) | - ust_before_fork() ------------> - ust_lock() You should be able to create this with an empty main, with no tracepoints. As long as sessiond is started as root so $HOME/lttng-ust-wait- is not created. You can also make the lttng-ust constructor (lttng_ust_init) wait forever and then you'll be able to see the deadlock in gdb without even leaving the lttng_ust_init constructor. Also, a thing that is confusing in the lttng_ust_init constructor. Why is the local ust listener thread always started, but the global one is only started if local_apps.allowed=1? Shouldn't that be reversed? The code would end up like this: ret = pthread_create(&global_apps.ust_listener, NULL, ust_listener_thread, &global_apps); if (local_apps.allowed) { ret = pthread_create(&local_apps.ust_listener, NULL, ust_listener_thread, &local_apps); } else { handle_register_done(&local_apps); } But this is unrelated to the deadlock scenario. Michael -----Original Message----- From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] Sent: Tuesday, June 12, 2012 12:25 PM To: Burton, Michael Cc: dgoulet at efficios.com; lttng-dev at lists.lttng.org Subject: Re: [lttng-dev] UST Hanging at Tracepoint Please build you app and lttng-ust with: make CFLAGS=-g (so with -O0 optimisation, and debug info) then, under gdb, please provide the full backtrace of each thread when stucked with: thread apply all bt full Thanks, Mathieu * Burton, Michael (mburton at ciena.com) wrote: > Also, I stepped through the code with gdb and things hang within the UST constructors and the main function of my application is never reached. > > -----Original Message----- > From: Burton, Michael [mailto:mburton at ciena.com] > Sent: Tuesday, June 12, 2012 11:45 AM > To: Mathieu Desnoyers; dgoulet at efficios.com > Cc: lttng-dev at lists.lttng.org > Subject: Re: [lttng-dev] UST Hanging at Tracepoint > > I put a fprintf at the beginning of main() and it never prints. > > -----Original Message----- > From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] > Sent: Tuesday, June 12, 2012 10:14 AM > To: Burton, Michael; dgoulet at efficios.com > Cc: lttng-dev at lists.lttng.org > Subject: Re: [lttng-dev] UST Hanging at Tracepoint > > * Mathieu Desnoyers (mathieu.desnoyers at efficios.com) wrote: > > Hi Michael, > > > > Please note that you should be able to run UST with either (or both) of: > > > > - root lttng-sessiond, for system-wide tracing > > - per-user lttng-sessiond, for per-user tracing. > > > > It is entirely normal that one of the two UST threads within the > > application can't find an active sessiond if, for instance, you have > > just a root (and no per-user) sessiond running. > > > > So although I'm glad that you got things working, I fail to understand > > your explanation: it should have worked fine with the root sessiond, > > even if no ~/.lttng exists. > > > > David, can you look into this ? > > hrm, further insight: is your program _really_ hang ? Try to make sure > you put a fprintf(stderr, ".....") at the beginning of your main(). > Looking once more at your detailed UST output indicates that the > per-user UST thread tells you that it cannot see any running sessiond. > So yes, this thread blocks, but the thread talking to the root sessiond > can very well be working fine, and your application might also be > running fine. This should be double-checked. > > Thanks, > > Mathieu > > > > > Thanks, > > > > Mathieu > > > > * Burton, Michael (mburton at ciena.com) wrote: > > > Mathieu, > > > > > > I figured out the problem. When we start sessiond on our system as root, the /.lttng/ folder is not created, along with its contents. UST timed out since there was no sock_path. I was able to get UST working once I got sessiond running. > > > > > > Thanks. > > > Michael > > > > > > ________________________________________ > > > From: Mathieu Desnoyers [mathieu.desnoyers at efficios.com] > > > Sent: 08 June 2012 21:38 > > > To: Burton, Michael > > > Cc: lttng-dev at lists.lttng.org > > > Subject: Re: [lttng-dev] UST Hanging at Tracepoint > > > > > > * Burton, Michael (mburton at ciena.com) wrote: > > > > For further insight, I've attached the strace from the hanging > > > > program. I also upgraded LTTng-UST to 2.0.3 and userspace-rcu to > > > > 0.7.3. > > > > > > Haven't had time to look at it yet, but could you also provide the > > > output of: > > > > > > LTTNG_UST_DEBUG=1 youprogname > > > > > > (starting your UST instrumented program with LTTNG_UST_DEBUG=1 env. var. > > > set) > > > > > > Thanks! > > > > > > Mathieu > > > > > > > > > > > Michael > > > > > > > > From: Burton, Michael [mailto:mburton at ciena.com] > > > > Sent: Thursday, June 07, 2012 2:16 PM > > > > To: lttng-dev at lists.lttng.org > > > > Subject: [lttng-dev] UST Hanging at Tracepoint > > > > > > > > I am working on getting LTTng 2.0 working in our 2.6.35 kernel. I have kernel tracing working but I'm having problems with UST. > > > > > > > > I have a program with a dynamic UST tracepoint. When I run the program with all UST events enabled (lttng enable-event -a -u) the program hangs on the tracepoint. The last line from the UST debug is: > > > > > > > > libust[6184/6185]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) > > > > > > > > Any insight into why this is happening? I've attached the UST and lttng-sessiond debug generated by the program. > > > > > > > > I am running the following: > > > > lttng-modules-2.6.32 (found through this mailing list) > > > > lttng-tools-2.0.1 > > > > lttng-ust-2.0.2 > > > > userspace-rcu-0.7.0 > > > > > > > > Thanks, > > > > Michael > > > > > > Content-Description: strace_debug.txt > > > > execve("/ciena/bin/idp", ["idp", "help"], [/* 11 vars */]) = 0 > > > > brk(0) = 0x804e000 > > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb783c000 > > > > access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) > > > > open("/etc/ld.so.cache", O_RDONLY) = 3 > > > > fstat64(3, {st_mode=S_IFREG|0644, st_size=17420, ...}) = 0 > > > > mmap2(NULL, 17420, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7837000 > > > > close(3) = 0 > > > > open("/usr/lib/liblttng-ust.so.0", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220A\0\0004\0\0\0$"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=209876, ...}) = 0 > > > > mmap2(NULL, 212940, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7803000 > > > > mmap2(0xb7832000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2e) = 0xb7832000 > > > > close(3) = 0 > > > > open("/usr/lib/liblttng-ust-ctl.so.0", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\200-\0\0004\0\0\0\274"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=140020, ...}) = 0 > > > > mmap2(NULL, 142828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77e0000 > > > > mmap2(0xb7802000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x21) = 0xb7802000 > > > > close(3) = 0 > > > > open("/usr/lib/liblttng-ust-fork.so.0", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\0\5\0\0004\0\0\0X"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=3864, ...}) = 0 > > > > mmap2(NULL, 6820, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77de000 > > > > mmap2(0xb77df000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77df000 > > > > close(3) = 0 > > > > open("/usr/lib/liblttng-ust-tracepoint.so.0", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220\v\0\0004\0\0\0008"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=16632, ...}) = 0 > > > > mmap2(NULL, 19944, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d9000 > > > > mmap2(0xb77dd000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3) = 0xb77dd000 > > > > close(3) = 0 > > > > open("/usr/lib/liblttng-ust-libc-wrapper.so.0", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\t\0\0004\0\0\0\364"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=9300, ...}) = 0 > > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb77d8000 > > > > mmap2(NULL, 12104, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d5000 > > > > mmap2(0xb77d7000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77d7000 > > > > close(3) = 0 > > > > open("/usr/lib/liburcu.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > > > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d0000 > > > > mmap2(0xb77d4000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77d4000 > > > > close(3) = 0 > > > > open("/usr/lib/liburcu-common.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340\5\0\0004\0\0\0\34"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=5084, ...}) = 0 > > > > mmap2(NULL, 8032, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77ce000 > > > > mmap2(0xb77cf000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77cf000 > > > > close(3) = 0 > > > > open("/usr/lib/liburcu-qsbr.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\22\0\0004\0\0\0\374"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > > > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77c9000 > > > > mmap2(0xb77cd000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77cd000 > > > > close(3) = 0 > > > > open("/usr/lib/liburcu-mb.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > > > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77c4000 > > > > mmap2(0xb77c8000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c8000 > > > > close(3) = 0 > > > > open("/usr/lib/liburcu-signal.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\23\0\0004\0\0\0\10"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18712, ...}) = 0 > > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb77c3000 > > > > mmap2(NULL, 21704, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77bd000 > > > > mmap2(0xb77c2000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c2000 > > > > close(3) = 0 > > > > open("/usr/lib/liburcu-cds.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\300\f\0\0004\0\0\0t"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=26204, ...}) = 0 > > > > mmap2(NULL, 25004, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77b6000 > > > > mmap2(0xb77bc000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb77bc000 > > > > close(3) = 0 > > > > open("/usr/lib/liburcu-bp.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320\24\0\0004\0\0\0\360"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=19968, ...}) = 0 > > > > mmap2(NULL, 23128, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77b0000 > > > > mmap2(0xb77b5000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77b5000 > > > > close(3) = 0 > > > > open("/lib/libdl.so.2", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0 \n\0\0004\0\0\0<"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=9668, ...}) = 0 > > > > mmap2(NULL, 12408, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77ac000 > > > > mmap2(0xb77ae000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77ae000 > > > > close(3) = 0 > > > > open("/lib/libuuid.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0@\r\0\0004\0\0\0\350"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=12872, ...}) = 0 > > > > mmap2(NULL, 15632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77a8000 > > > > mmap2(0xb77ab000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2) = 0xb77ab000 > > > > close(3) = 0 > > > > open("/lib/libc.so.6", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220n\1\0004\0\0\0L"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=1347780, ...}) = 0 > > > > mmap2(NULL, 1358120, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb765c000 > > > > mprotect(0xb77a1000, 4096, PROT_NONE) = 0 > > > > mmap2(0xb77a2000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x145) = 0xb77a2000 > > > > mmap2(0xb77a5000, 10536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb77a5000 > > > > close(3) = 0 > > > > open("/lib/librt.so.1", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0p\30\0\0004\0\0\0\230"..., 512) = 512 > > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb765b000 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=30616, ...}) = 0 > > > > mmap2(NULL, 33364, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7652000 > > > > mmap2(0xb7659000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb7659000 > > > > close(3) = 0 > > > > open("/lib/libpthread.so.0", O_RDONLY) = 3 > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360J\0\0004\0\0\0\354"..., 512) = 512 > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=88420, ...}) = 0 > > > > mmap2(NULL, 98828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7639000 > > > > mmap2(0xb764e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14) = 0xb764e000 > > > > mmap2(0xb7650000, 4620, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7650000 > > > > close(3) = 0 > > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7638000 > > > > mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7636000 > > > > set_thread_area({entry_number:-1 -> 6, base_addr:0xb7636d80, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 > > > > mprotect(0xb764e000, 4096, PROT_READ) = 0 > > > > mprotect(0xb7659000, 4096, PROT_READ) = 0 > > > > mprotect(0xb77a2000, 8192, PROT_READ) = 0 > > > > mprotect(0xb77ae000, 4096, PROT_READ) = 0 > > > > mprotect(0xb785a000, 4096, PROT_READ) = 0 > > > > munmap(0xb7837000, 17420) = 0 > > > > set_tid_address(0xb7636de8) = 3050 > > > > set_robust_list(0xb7636df0, 0xc) = 0 > > > > futex(0xbfaaf560, FUTEX_WAKE_PRIVATE, 1) = 0 > > > > futex(0xbfaaf560, 0x189 /* FUTEX_??? */, 1, NULL, bfaaf570) = -1 EAGAIN (Resource temporarily unavailable) > > > > rt_sigaction(SIGRTMIN, {0xb763d4f0, [], SA_SIGINFO}, NULL, 8) = 0 > > > > rt_sigaction(SIGRT_1, {0xb763d9c0, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0 > > > > rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0 > > > > getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0 > > > > uname({sys="Linux", node="5410", ...}) = 0 > > > > rt_sigaction(SIGUSR1, {0xb77bec0a, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0 > > > > futex(0xb77af06c, FUTEX_WAKE_PRIVATE, 2147483647) = 0 > > > > brk(0) = 0x804e000 > > > > brk(0x806f000) = 0x806f000 > > > > gettimeofday({1339187216, 880428}, NULL) = 0 > > > > open("/dev/urandom", O_RDONLY|O_LARGEFILE) = 3 > > > > fcntl64(3, F_GETFD) = 0 > > > > fcntl64(3, F_SETFD, FD_CLOEXEC) = 0 > > > > getuid32() = 0 > > > > gettimeofday({1339187216, 889603}, NULL) = 0 > > > > gettimeofday({1339187216, 894471}, NULL) = 0 > > > > read(3, "\275\0051F\215\373\371\202\377kfb/\362\303N"..., 16) = 16 > > > > clock_gettime(CLOCK_REALTIME, {1339187216, 902329405}) = 0 > > > > getuid32() = 0 > > > > geteuid32() = 0 > > > > rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0 > > > > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6e36000 > > > > mprotect(0xb6e36000, 4096, PROT_NONE) = 0 > > > > clone(child_stack=0xb7634d64, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0xb7635bd8, {entry_number:6, base_addr:0xb7635b70, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xb7635bd8) = 3051 > > > > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6636000 > > > > mprotect(0xb6636000, 4096, PROT_NONE) = 0 > > > > clone(child_stack=0xb6e34d64, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0xb6e35bd8, {entry_number:6, base_addr:0xb6e35b70, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xb6e35bd8) = 3052 > > > > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > > > > gettimeofday({1339187216, 974869}, NULL) = 0 > > > > futex(0xb7836e30, FUTEX_WAIT_PRIVATE, 0, {2, 927460405}) = -1 ETIMEDOUT (Connection timed out) > > > > gettid() = 3050 > > > > write(2, "libust[3050/3050]: Error: Timed o"..., 107libust[3050/3050]: Error: Timed out waiting for ltt-sessiond (in lttng_ust_init() at lttng-ust-comm.c:912) > > > > ) = 107 > > > > futex(0xb7836e80, FUTEX_WAIT_PRIVATE, 2, NULL > > > > _______________________________________________ > > > > lttng-dev mailing list > > > > lttng-dev at lists.lttng.org > > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > > > > > > > > -- > > > Mathieu Desnoyers > > > Operating System Efficiency R&D Consultant > > > EfficiOS Inc. > > > http://www.efficios.com > > > > -- > > Mathieu Desnoyers > > Operating System Efficiency R&D Consultant > > EfficiOS Inc. > > http://www.efficios.com > > > > _______________________________________________ > > lttng-dev mailing list > > lttng-dev at lists.lttng.org > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > -- > Mathieu Desnoyers > Operating System Efficiency R&D Consultant > EfficiOS Inc. > http://www.efficios.com > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Tue Jun 12 20:09:51 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 12 Jun 2012 20:09:51 -0400 Subject: [lttng-dev] UST Hanging at Tracepoint In-Reply-To: <3F56B905CF2083408E8482044F20428AEF7A0ED6@ONWVEXCHMB01.ciena.com> References: <3F56B905CF2083408E8482044F20428AEF604D22@ONWVEXCHMB01.ciena.com> <3F56B905CF2083408E8482044F20428AEF78BC79@ONWVEXCHMB01.ciena.com> <20120612140925.GB13917@Krystal> <20120612141344.GC13917@Krystal> <3F56B905CF2083408E8482044F20428AEF7A0AFE@ONWVEXCHMB01.ciena.com> <3F56B905CF2083408E8482044F20428AEF7A0B4C@ONWVEXCHMB01.ciena.com> <20120612162526.GB16098@Krystal> <3F56B905CF2083408E8482044F20428AEF7A0ED6@ONWVEXCHMB01.ciena.com> Message-ID: <20120613000950.GD21966@Krystal> * Burton, Michael (mburton at ciena.com) wrote: > Mathieu, > > I think there is a deadlock scenario in UST, which has been causing my problem. Good catch ! > > sessiond is started as root: > - creates global sockets ONLY > - DOES NOT CREATE shm in $HOME/lttng-ust-wait- > > application linked against ust is run as root: > - in lttng_ust_init constructor > - ust_listener_thread (local_apps) > - fails to connect to local_apps in $HOME/.lttng (as expected) > - prev_connect_failed=1 > - ust_unlock() > - restart > - wait_for_sessiond() > --> - ust_lock() > | - get_map_shm() > | - get_wait_shm() > DEADLOCK - shm_open() FAILS (not created by sessiond when run by root) > | - fork() (trying to create shared memory itself) > | - ust_before_fork() > ------------> - ust_lock() > > > You should be able to create this with an empty main, with no > tracepoints. As long as sessiond is started as root so > $HOME/lttng-ust-wait- is not created. You can also make the > lttng-ust constructor (lttng_ust_init) wait forever and then you'll be > able to see the deadlock in gdb without even leaving the > lttng_ust_init constructor. Ah, I see. This deadlock is caused by the interaction between liblttng-ust-libc-wrapper and liblttng-ust (the fork override is performed by liblttng-ust-libc-wrapper). I'll have to look into this issue further. More on that soon. > Also, a thing that is confusing in the lttng_ust_init constructor. > Why is the local ust listener thread always started, but the global > one is only started if local_apps.allowed=1? Shouldn't that be > reversed? The code would end up like this: > > ret = pthread_create(&global_apps.ust_listener, NULL, > ust_listener_thread, &global_apps); > > if (local_apps.allowed) { > ret = pthread_create(&local_apps.ust_listener, NULL, > ust_listener_thread, &local_apps); > } else { > handle_register_done(&local_apps); > } > > But this is unrelated to the deadlock scenario. Good catch too! This is a really stupid error on my behalf. Fixing immediately. Thanks! Mathieu > > Michael > > -----Original Message----- > From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] > Sent: Tuesday, June 12, 2012 12:25 PM > To: Burton, Michael > Cc: dgoulet at efficios.com; lttng-dev at lists.lttng.org > Subject: Re: [lttng-dev] UST Hanging at Tracepoint > > Please build you app and lttng-ust with: > > make CFLAGS=-g (so with -O0 optimisation, and debug info) > > then, under gdb, please provide the full backtrace of each thread when > stucked with: > > thread apply all bt full > > Thanks, > > Mathieu > > > * Burton, Michael (mburton at ciena.com) wrote: > > Also, I stepped through the code with gdb and things hang within the UST constructors and the main function of my application is never reached. > > > > -----Original Message----- > > From: Burton, Michael [mailto:mburton at ciena.com] > > Sent: Tuesday, June 12, 2012 11:45 AM > > To: Mathieu Desnoyers; dgoulet at efficios.com > > Cc: lttng-dev at lists.lttng.org > > Subject: Re: [lttng-dev] UST Hanging at Tracepoint > > > > I put a fprintf at the beginning of main() and it never prints. > > > > -----Original Message----- > > From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] > > Sent: Tuesday, June 12, 2012 10:14 AM > > To: Burton, Michael; dgoulet at efficios.com > > Cc: lttng-dev at lists.lttng.org > > Subject: Re: [lttng-dev] UST Hanging at Tracepoint > > > > * Mathieu Desnoyers (mathieu.desnoyers at efficios.com) wrote: > > > Hi Michael, > > > > > > Please note that you should be able to run UST with either (or both) of: > > > > > > - root lttng-sessiond, for system-wide tracing > > > - per-user lttng-sessiond, for per-user tracing. > > > > > > It is entirely normal that one of the two UST threads within the > > > application can't find an active sessiond if, for instance, you have > > > just a root (and no per-user) sessiond running. > > > > > > So although I'm glad that you got things working, I fail to understand > > > your explanation: it should have worked fine with the root sessiond, > > > even if no ~/.lttng exists. > > > > > > David, can you look into this ? > > > > hrm, further insight: is your program _really_ hang ? Try to make sure > > you put a fprintf(stderr, ".....") at the beginning of your main(). > > Looking once more at your detailed UST output indicates that the > > per-user UST thread tells you that it cannot see any running sessiond. > > So yes, this thread blocks, but the thread talking to the root sessiond > > can very well be working fine, and your application might also be > > running fine. This should be double-checked. > > > > Thanks, > > > > Mathieu > > > > > > > > Thanks, > > > > > > Mathieu > > > > > > * Burton, Michael (mburton at ciena.com) wrote: > > > > Mathieu, > > > > > > > > I figured out the problem. When we start sessiond on our system as root, the /.lttng/ folder is not created, along with its contents. UST timed out since there was no sock_path. I was able to get UST working once I got sessiond running. > > > > > > > > Thanks. > > > > Michael > > > > > > > > ________________________________________ > > > > From: Mathieu Desnoyers [mathieu.desnoyers at efficios.com] > > > > Sent: 08 June 2012 21:38 > > > > To: Burton, Michael > > > > Cc: lttng-dev at lists.lttng.org > > > > Subject: Re: [lttng-dev] UST Hanging at Tracepoint > > > > > > > > * Burton, Michael (mburton at ciena.com) wrote: > > > > > For further insight, I've attached the strace from the hanging > > > > > program. I also upgraded LTTng-UST to 2.0.3 and userspace-rcu to > > > > > 0.7.3. > > > > > > > > Haven't had time to look at it yet, but could you also provide the > > > > output of: > > > > > > > > LTTNG_UST_DEBUG=1 youprogname > > > > > > > > (starting your UST instrumented program with LTTNG_UST_DEBUG=1 env. var. > > > > set) > > > > > > > > Thanks! > > > > > > > > Mathieu > > > > > > > > > > > > > > Michael > > > > > > > > > > From: Burton, Michael [mailto:mburton at ciena.com] > > > > > Sent: Thursday, June 07, 2012 2:16 PM > > > > > To: lttng-dev at lists.lttng.org > > > > > Subject: [lttng-dev] UST Hanging at Tracepoint > > > > > > > > > > I am working on getting LTTng 2.0 working in our 2.6.35 kernel. I have kernel tracing working but I'm having problems with UST. > > > > > > > > > > I have a program with a dynamic UST tracepoint. When I run the program with all UST events enabled (lttng enable-event -a -u) the program hangs on the tracepoint. The last line from the UST debug is: > > > > > > > > > > libust[6184/6185]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) > > > > > > > > > > Any insight into why this is happening? I've attached the UST and lttng-sessiond debug generated by the program. > > > > > > > > > > I am running the following: > > > > > lttng-modules-2.6.32 (found through this mailing list) > > > > > lttng-tools-2.0.1 > > > > > lttng-ust-2.0.2 > > > > > userspace-rcu-0.7.0 > > > > > > > > > > Thanks, > > > > > Michael > > > > > > > > Content-Description: strace_debug.txt > > > > > execve("/ciena/bin/idp", ["idp", "help"], [/* 11 vars */]) = 0 > > > > > brk(0) = 0x804e000 > > > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb783c000 > > > > > access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) > > > > > open("/etc/ld.so.cache", O_RDONLY) = 3 > > > > > fstat64(3, {st_mode=S_IFREG|0644, st_size=17420, ...}) = 0 > > > > > mmap2(NULL, 17420, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7837000 > > > > > close(3) = 0 > > > > > open("/usr/lib/liblttng-ust.so.0", O_RDONLY) = 3 > > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220A\0\0004\0\0\0$"..., 512) = 512 > > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=209876, ...}) = 0 > > > > > mmap2(NULL, 212940, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7803000 > > > > > mmap2(0xb7832000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2e) = 0xb7832000 > > > > > close(3) = 0 > > > > > open("/usr/lib/liblttng-ust-ctl.so.0", O_RDONLY) = 3 > > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\200-\0\0004\0\0\0\274"..., 512) = 512 > > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=140020, ...}) = 0 > > > > > mmap2(NULL, 142828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77e0000 > > > > > mmap2(0xb7802000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x21) = 0xb7802000 > > > > > close(3) = 0 > > > > > open("/usr/lib/liblttng-ust-fork.so.0", O_RDONLY) = 3 > > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\0\5\0\0004\0\0\0X"..., 512) = 512 > > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=3864, ...}) = 0 > > > > > mmap2(NULL, 6820, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77de000 > > > > > mmap2(0xb77df000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77df000 > > > > > close(3) = 0 > > > > > open("/usr/lib/liblttng-ust-tracepoint.so.0", O_RDONLY) = 3 > > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220\v\0\0004\0\0\0008"..., 512) = 512 > > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=16632, ...}) = 0 > > > > > mmap2(NULL, 19944, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d9000 > > > > > mmap2(0xb77dd000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3) = 0xb77dd000 > > > > > close(3) = 0 > > > > > open("/usr/lib/liblttng-ust-libc-wrapper.so.0", O_RDONLY) = 3 > > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\t\0\0004\0\0\0\364"..., 512) = 512 > > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=9300, ...}) = 0 > > > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb77d8000 > > > > > mmap2(NULL, 12104, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d5000 > > > > > mmap2(0xb77d7000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77d7000 > > > > > close(3) = 0 > > > > > open("/usr/lib/liburcu.so.1", O_RDONLY) = 3 > > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"..., 512) = 512 > > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > > > > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77d0000 > > > > > mmap2(0xb77d4000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77d4000 > > > > > close(3) = 0 > > > > > open("/usr/lib/liburcu-common.so.1", O_RDONLY) = 3 > > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340\5\0\0004\0\0\0\34"..., 512) = 512 > > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=5084, ...}) = 0 > > > > > mmap2(NULL, 8032, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77ce000 > > > > > mmap2(0xb77cf000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77cf000 > > > > > close(3) = 0 > > > > > open("/usr/lib/liburcu-qsbr.so.1", O_RDONLY) = 3 > > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\22\0\0004\0\0\0\374"..., 512) = 512 > > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > > > > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77c9000 > > > > > mmap2(0xb77cd000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77cd000 > > > > > close(3) = 0 > > > > > open("/usr/lib/liburcu-mb.so.1", O_RDONLY) = 3 > > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"..., 512) = 512 > > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0 > > > > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77c4000 > > > > > mmap2(0xb77c8000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c8000 > > > > > close(3) = 0 > > > > > open("/usr/lib/liburcu-signal.so.1", O_RDONLY) = 3 > > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\23\0\0004\0\0\0\10"..., 512) = 512 > > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18712, ...}) = 0 > > > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb77c3000 > > > > > mmap2(NULL, 21704, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77bd000 > > > > > mmap2(0xb77c2000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c2000 > > > > > close(3) = 0 > > > > > open("/usr/lib/liburcu-cds.so.1", O_RDONLY) = 3 > > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\300\f\0\0004\0\0\0t"..., 512) = 512 > > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=26204, ...}) = 0 > > > > > mmap2(NULL, 25004, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77b6000 > > > > > mmap2(0xb77bc000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb77bc000 > > > > > close(3) = 0 > > > > > open("/usr/lib/liburcu-bp.so.1", O_RDONLY) = 3 > > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320\24\0\0004\0\0\0\360"..., 512) = 512 > > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=19968, ...}) = 0 > > > > > mmap2(NULL, 23128, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77b0000 > > > > > mmap2(0xb77b5000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77b5000 > > > > > close(3) = 0 > > > > > open("/lib/libdl.so.2", O_RDONLY) = 3 > > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0 \n\0\0004\0\0\0<"..., 512) = 512 > > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=9668, ...}) = 0 > > > > > mmap2(NULL, 12408, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77ac000 > > > > > mmap2(0xb77ae000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77ae000 > > > > > close(3) = 0 > > > > > open("/lib/libuuid.so.1", O_RDONLY) = 3 > > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0@\r\0\0004\0\0\0\350"..., 512) = 512 > > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=12872, ...}) = 0 > > > > > mmap2(NULL, 15632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb77a8000 > > > > > mmap2(0xb77ab000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2) = 0xb77ab000 > > > > > close(3) = 0 > > > > > open("/lib/libc.so.6", O_RDONLY) = 3 > > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220n\1\0004\0\0\0L"..., 512) = 512 > > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=1347780, ...}) = 0 > > > > > mmap2(NULL, 1358120, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb765c000 > > > > > mprotect(0xb77a1000, 4096, PROT_NONE) = 0 > > > > > mmap2(0xb77a2000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x145) = 0xb77a2000 > > > > > mmap2(0xb77a5000, 10536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb77a5000 > > > > > close(3) = 0 > > > > > open("/lib/librt.so.1", O_RDONLY) = 3 > > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0p\30\0\0004\0\0\0\230"..., 512) = 512 > > > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb765b000 > > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=30616, ...}) = 0 > > > > > mmap2(NULL, 33364, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7652000 > > > > > mmap2(0xb7659000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb7659000 > > > > > close(3) = 0 > > > > > open("/lib/libpthread.so.0", O_RDONLY) = 3 > > > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360J\0\0004\0\0\0\354"..., 512) = 512 > > > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=88420, ...}) = 0 > > > > > mmap2(NULL, 98828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7639000 > > > > > mmap2(0xb764e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14) = 0xb764e000 > > > > > mmap2(0xb7650000, 4620, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7650000 > > > > > close(3) = 0 > > > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7638000 > > > > > mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7636000 > > > > > set_thread_area({entry_number:-1 -> 6, base_addr:0xb7636d80, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 > > > > > mprotect(0xb764e000, 4096, PROT_READ) = 0 > > > > > mprotect(0xb7659000, 4096, PROT_READ) = 0 > > > > > mprotect(0xb77a2000, 8192, PROT_READ) = 0 > > > > > mprotect(0xb77ae000, 4096, PROT_READ) = 0 > > > > > mprotect(0xb785a000, 4096, PROT_READ) = 0 > > > > > munmap(0xb7837000, 17420) = 0 > > > > > set_tid_address(0xb7636de8) = 3050 > > > > > set_robust_list(0xb7636df0, 0xc) = 0 > > > > > futex(0xbfaaf560, FUTEX_WAKE_PRIVATE, 1) = 0 > > > > > futex(0xbfaaf560, 0x189 /* FUTEX_??? */, 1, NULL, bfaaf570) = -1 EAGAIN (Resource temporarily unavailable) > > > > > rt_sigaction(SIGRTMIN, {0xb763d4f0, [], SA_SIGINFO}, NULL, 8) = 0 > > > > > rt_sigaction(SIGRT_1, {0xb763d9c0, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0 > > > > > rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0 > > > > > getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0 > > > > > uname({sys="Linux", node="5410", ...}) = 0 > > > > > rt_sigaction(SIGUSR1, {0xb77bec0a, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0 > > > > > futex(0xb77af06c, FUTEX_WAKE_PRIVATE, 2147483647) = 0 > > > > > brk(0) = 0x804e000 > > > > > brk(0x806f000) = 0x806f000 > > > > > gettimeofday({1339187216, 880428}, NULL) = 0 > > > > > open("/dev/urandom", O_RDONLY|O_LARGEFILE) = 3 > > > > > fcntl64(3, F_GETFD) = 0 > > > > > fcntl64(3, F_SETFD, FD_CLOEXEC) = 0 > > > > > getuid32() = 0 > > > > > gettimeofday({1339187216, 889603}, NULL) = 0 > > > > > gettimeofday({1339187216, 894471}, NULL) = 0 > > > > > read(3, "\275\0051F\215\373\371\202\377kfb/\362\303N"..., 16) = 16 > > > > > clock_gettime(CLOCK_REALTIME, {1339187216, 902329405}) = 0 > > > > > getuid32() = 0 > > > > > geteuid32() = 0 > > > > > rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0 > > > > > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6e36000 > > > > > mprotect(0xb6e36000, 4096, PROT_NONE) = 0 > > > > > clone(child_stack=0xb7634d64, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0xb7635bd8, {entry_number:6, base_addr:0xb7635b70, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xb7635bd8) = 3051 > > > > > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6636000 > > > > > mprotect(0xb6636000, 4096, PROT_NONE) = 0 > > > > > clone(child_stack=0xb6e34d64, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED, parent_tidptr=0xb6e35bd8, {entry_number:6, base_addr:0xb6e35b70, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}, child_tidptr=0xb6e35bd8) = 3052 > > > > > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 > > > > > gettimeofday({1339187216, 974869}, NULL) = 0 > > > > > futex(0xb7836e30, FUTEX_WAIT_PRIVATE, 0, {2, 927460405}) = -1 ETIMEDOUT (Connection timed out) > > > > > gettid() = 3050 > > > > > write(2, "libust[3050/3050]: Error: Timed o"..., 107libust[3050/3050]: Error: Timed out waiting for ltt-sessiond (in lttng_ust_init() at lttng-ust-comm.c:912) > > > > > ) = 107 > > > > > futex(0xb7836e80, FUTEX_WAIT_PRIVATE, 2, NULL > > > > > _______________________________________________ > > > > > lttng-dev mailing list > > > > > lttng-dev at lists.lttng.org > > > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > > > > > > > > > > > -- > > > > Mathieu Desnoyers > > > > Operating System Efficiency R&D Consultant > > > > EfficiOS Inc. > > > > http://www.efficios.com > > > > > > -- > > > Mathieu Desnoyers > > > Operating System Efficiency R&D Consultant > > > EfficiOS Inc. > > > http://www.efficios.com > > > > > > _______________________________________________ > > > lttng-dev mailing list > > > lttng-dev at lists.lttng.org > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > > > -- > > Mathieu Desnoyers > > Operating System Efficiency R&D Consultant > > EfficiOS Inc. > > http://www.efficios.com > > > > _______________________________________________ > > lttng-dev mailing list > > lttng-dev at lists.lttng.org > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > -- > Mathieu Desnoyers > Operating System Efficiency R&D Consultant > EfficiOS Inc. > http://www.efficios.com -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Tue Jun 12 20:21:28 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 12 Jun 2012 20:21:28 -0400 Subject: [lttng-dev] UST Hanging at Tracepoint In-Reply-To: <20120613000950.GD21966@Krystal> References: <3F56B905CF2083408E8482044F20428AEF604D22@ONWVEXCHMB01.ciena.com> <3F56B905CF2083408E8482044F20428AEF78BC79@ONWVEXCHMB01.ciena.com> <20120612140925.GB13917@Krystal> <20120612141344.GC13917@Krystal> <3F56B905CF2083408E8482044F20428AEF7A0AFE@ONWVEXCHMB01.ciena.com> <3F56B905CF2083408E8482044F20428AEF7A0B4C@ONWVEXCHMB01.ciena.com> <20120612162526.GB16098@Krystal> <3F56B905CF2083408E8482044F20428AEF7A0ED6@ONWVEXCHMB01.ciena.com> <20120613000950.GD21966@Krystal> Message-ID: <20120613002128.GE21966@Krystal> * Mathieu Desnoyers (mathieu.desnoyers at efficios.com) wrote: > * Burton, Michael (mburton at ciena.com) wrote: > > Mathieu, > > > > I think there is a deadlock scenario in UST, which has been causing my problem. > > Good catch ! > > > > > sessiond is started as root: > > - creates global sockets ONLY > > - DOES NOT CREATE shm in $HOME/lttng-ust-wait- > > > > application linked against ust is run as root: > > - in lttng_ust_init constructor > > - ust_listener_thread (local_apps) > > - fails to connect to local_apps in $HOME/.lttng (as expected) > > - prev_connect_failed=1 > > - ust_unlock() > > - restart > > - wait_for_sessiond() > > --> - ust_lock() > > | - get_map_shm() > > | - get_wait_shm() > > DEADLOCK - shm_open() FAILS (not created by sessiond when run by root) > > | - fork() (trying to create shared memory itself) > > | - ust_before_fork() > > ------------> - ust_lock() > > > > > > You should be able to create this with an empty main, with no > > tracepoints. As long as sessiond is started as root so > > $HOME/lttng-ust-wait- is not created. You can also make the > > lttng-ust constructor (lttng_ust_init) wait forever and then you'll be > > able to see the deadlock in gdb without even leaving the > > lttng_ust_init constructor. > > Ah, I see. This deadlock is caused by the interaction between > liblttng-ust-libc-wrapper and liblttng-ust (the fork override is > performed by liblttng-ust-libc-wrapper). I'll have to look into this > issue further. More on that soon. Actually, I meant that it comes from interaction between liblttng-ust-fork and liblttng-ust (sorry about the confusion above). I'll see if I can provide a direct access to the fork() symbol to liblttng-ust, using a weak symbol so it is defined even when liblttng-ust-fork is not linked. More on that soon... Mathieu -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From toojays at toojays.net Wed Jun 13 03:04:42 2012 From: toojays at toojays.net (John Steele Scott) Date: Wed, 13 Jun 2012 16:34:42 +0930 Subject: [lttng-dev] urcu commit a767fd requires autoconf >= 2.64. Message-ID: http://lists.lttng.org/pipermail/lttng-dev/2012-May/017927.html I tried to build the latest urcu (git master e51500) on a Centos 6.2 box, and got: jscott at dxi0-62:~/src/userspace-rcu$ make -j4 CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /users/jscott/src/userspace-rcu/config/missing --run aclocal-1.11 -I config CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /users/jscott/src/userspace-rcu/config/missing --run autoconf cd . && /bin/sh /users/jscott/src/userspace-rcu/config/missing --run automake-1.11 --foreign configure:4010: error: possibly undefined macro: m4_ifnblank If this token and others are legitimate, please use m4_pattern_allow. See the Autoconf documentation. make: *** [configure] Error 1 make: *** Waiting for unfinished jobs.... Some digging showed that the macro m4_ifnblank requires autoconf 2.64. Centos 6.2 has autoconf 2.63. :( I just worked around it by reverting commit a767fd locally, then I can build fine. cheers, John From mathieu.desnoyers at efficios.com Wed Jun 13 04:10:01 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Wed, 13 Jun 2012 04:10:01 -0400 Subject: [lttng-dev] urcu commit a767fd requires autoconf >= 2.64. In-Reply-To: References: Message-ID: <20120613081001.GA30123@Krystal> * John Steele Scott (toojays at toojays.net) wrote: > http://lists.lttng.org/pipermail/lttng-dev/2012-May/017927.html > > I tried to build the latest urcu (git master e51500) on a Centos 6.2 box, and got: > > jscott at dxi0-62:~/src/userspace-rcu$ make -j4 > CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /users/jscott/src/userspace-rcu/config/missing --run aclocal-1.11 -I config > CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /users/jscott/src/userspace-rcu/config/missing --run autoconf > cd . && /bin/sh /users/jscott/src/userspace-rcu/config/missing --run automake-1.11 --foreign > configure:4010: error: possibly undefined macro: m4_ifnblank > If this token and others are legitimate, please use m4_pattern_allow. > See the Autoconf documentation. > make: *** [configure] Error 1 > make: *** Waiting for unfinished jobs.... > > Some digging showed that the macro m4_ifnblank requires autoconf 2.64. Centos 6.2 has autoconf 2.63. :( > > I just worked around it by reverting commit a767fd locally, then I can build fine. Thanks for pointing this out! Can you try the following patch and let me know if it fixes your issue ? diff --git a/config/ax_tls.m4 b/config/ax_tls.m4 index 033e3b1..5ab1a41 100644 --- a/config/ax_tls.m4 +++ b/config/ax_tls.m4 @@ -44,7 +44,23 @@ # modified version of the Autoconf Macro, you may extend this special # exception to the GPL to apply to your modified version as well. -#serial 10 +#serial 11 + +# Define m4_ifblank and m4_ifnblank macros from introduced in +# autotools 2.64 m4sugar.m4 if using an earlier autotools. + +ifdef([m4_ifblank], [], [ +m4_define([m4_ifblank], +[m4_if(m4_translit([[$1]], [ ][ ][ +]), [], [$2], [$3])]) +]) + + +ifdef([m4_ifnblank], [], [ +m4_define([m4_ifnblank], +[m4_if(m4_translit([[$1]], [ ][ ][ +]), [], [$3], [$2])]) +]) AC_DEFUN([AX_TLS], [ AC_MSG_CHECKING(for thread local storage (TLS) class) -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Wed Jun 13 04:29:05 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Wed, 13 Jun 2012 04:29:05 -0400 Subject: [lttng-dev] UST Hanging at Tracepoint In-Reply-To: <20120613002128.GE21966@Krystal> References: <3F56B905CF2083408E8482044F20428AEF604D22@ONWVEXCHMB01.ciena.com> <3F56B905CF2083408E8482044F20428AEF78BC79@ONWVEXCHMB01.ciena.com> <20120612140925.GB13917@Krystal> <20120612141344.GC13917@Krystal> <3F56B905CF2083408E8482044F20428AEF7A0AFE@ONWVEXCHMB01.ciena.com> <3F56B905CF2083408E8482044F20428AEF7A0B4C@ONWVEXCHMB01.ciena.com> <20120612162526.GB16098@Krystal> <3F56B905CF2083408E8482044F20428AEF7A0ED6@ONWVEXCHMB01.ciena.com> <20120613000950.GD21966@Krystal> <20120613002128.GE21966@Krystal> Message-ID: <20120613082905.GA30485@Krystal> * Mathieu Desnoyers (mathieu.desnoyers at efficios.com) wrote: > * Mathieu Desnoyers (mathieu.desnoyers at efficios.com) wrote: > > * Burton, Michael (mburton at ciena.com) wrote: > > > Mathieu, > > > > > > I think there is a deadlock scenario in UST, which has been causing my problem. > > > > Good catch ! > > > > > > > > sessiond is started as root: > > > - creates global sockets ONLY > > > - DOES NOT CREATE shm in $HOME/lttng-ust-wait- > > > > > > application linked against ust is run as root: > > > - in lttng_ust_init constructor > > > - ust_listener_thread (local_apps) > > > - fails to connect to local_apps in $HOME/.lttng (as expected) > > > - prev_connect_failed=1 > > > - ust_unlock() > > > - restart > > > - wait_for_sessiond() > > > --> - ust_lock() > > > | - get_map_shm() > > > | - get_wait_shm() > > > DEADLOCK - shm_open() FAILS (not created by sessiond when run by root) > > > | - fork() (trying to create shared memory itself) > > > | - ust_before_fork() > > > ------------> - ust_lock() > > > > > > > > > You should be able to create this with an empty main, with no > > > tracepoints. As long as sessiond is started as root so > > > $HOME/lttng-ust-wait- is not created. You can also make the > > > lttng-ust constructor (lttng_ust_init) wait forever and then you'll be > > > able to see the deadlock in gdb without even leaving the > > > lttng_ust_init constructor. > > > > Ah, I see. This deadlock is caused by the interaction between > > liblttng-ust-libc-wrapper and liblttng-ust (the fork override is > > performed by liblttng-ust-libc-wrapper). I'll have to look into this > > issue further. More on that soon. > > Actually, I meant that it comes from interaction between > liblttng-ust-fork and liblttng-ust (sorry about the confusion above). > I'll see if I can provide a direct access to the fork() symbol to > liblttng-ust, using a weak symbol so it is defined even when > liblttng-ust-fork is not linked. > > More on that soon... Fixed by master commit: commit e8508a495280a1396a0319a157409a7ea61ffc5a Author: Mathieu Desnoyers Date: Wed Jun 13 04:26:34 2012 -0400 Fix: liblttng-ust-fork deadlock and merged into stable-2.0. Thanks for pointing out this important bug. Mathieu -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From pavan.anumula at sasken.com Wed Jun 13 07:45:44 2012 From: pavan.anumula at sasken.com (Pavan Anumula) Date: Wed, 13 Jun 2012 17:15:44 +0530 Subject: [lttng-dev] Lttng start failed In-Reply-To: <4FD7618F.8090407@efficios.com> References: <47D610AD9C485E458630BA38C324D3B60113FB00F9EA@EXGMBX01.sasken.com> <20120608174226.GA23339@Krystal> <47D610AD9C485E458630BA38C324D3B60113FB00FA00@EXGMBX01.sasken.com> <20120609123251.GA14280@Krystal> <47D610AD9C485E458630BA38C324D3B60113FB0E22D1@EXGMBX01.sasken.com> <20120612140548.GA13917@Krystal> <4FD7618F.8090407@efficios.com> Message-ID: <47D610AD9C485E458630BA38C324D3B60113FB00FAF3@EXGMBX01.sasken.com> Hi David, I am still unable to see any traces when running the /tests/demo application in UST. Please find as much detail as I can give you below: We are using UST version 2.0.1 We are using userspace-rcu version 0.6.7 We are using modules version 2.0.2 We are using tools version 2.0.1 The Kernel version of 2.6.33.7 and we have applied the patches for 2.6.32 as supplied by Yannick Brosseau. The kernel is built with the following LTTNG configuration options: CONFIG_KPROBES is not set CONFIG_HAVE_KPROBES=y CONFIG_HAVE_KRETPROBES=y CONFIG_TRACEPOINTS=y CONFIG_HAVE_SYSCALL_TRACEPOINTS is not set CONFIG_PERF_EVENTS is not set I have booted and issued the following commands: root at arago:~# arm-none-linux-gnueabi-lttng-sessiond -d /* To start a deamon */ root at arago:~# arm-none-linux-gnueabi-lttng create xyz Session xyz created. Traces will be written in /home/root/lttng-traces/xyz-20110326-104303. root at arago:~# arm-none-linux-gnueabi-lttng enable-event -k -a All kernel events are enabled in channel channel0 root at arago:~# arm-none-linux-gnueabi-lttng start LTTng: Failure to write metadata to buffers (timeout) Error: Starting kernel trace failed ********* The output of lsmod After loading lttng modules before executing the lttng commands to trace*********** root at arago:~# lsmod lttng_ring_buffer_metadata_client 4146 0 lttng_ring_buffer_client_overwrite 9226 0 lttng_ring_buffer_client_mmap_overwrite 9234 0 lttng_ring_buffer_client_mmap_discard 7826 0 lttng_ring_buffer_client_discard 7822 0 lttng_probe_timer 6699 0 lttng_probe_signal 3332 0 lttng_probe_sched 6649 0 lttng_probe_lttng 1086 0 lttng_tracer 24543 9 lttng_ring_buffer_metadata_client,lttng_ring_buffer_client_overwrite,lttng_ring_buffer_client_mmap_overwrite,lttng_ring_buffer_client_mmap_discard,lttng_ring_buffer_client_discard,lttng_probe_timer,lttng_probe_signal,lttng_probe_sched,lttng_probe_lttng lttng_statedump 3587 1 lttng_tracer lttng_lib_ring_buffer 31358 6 lttng_ring_buffer_metadata_client,lttng_ring_buffer_client_overwrite,lttng_ring_buffer_client_mmap_overwrite,lttng_ring_buffer_client_mmap_discard,lttng_ring_buffer_client_discard,lttng_tracer ******************The new output of lsmod after executing trace commands********************** Module Size Used by lttng_probe_statedump 5848 7 lttng_probe_irq 2100 3 lttng_types 932 0 lttng_ring_buffer_metadata_mmap_client 4150 0 lttng_ring_buffer_metadata_client 4146 1 lttng_ring_buffer_client_overwrite 9226 0 lttng_ring_buffer_client_mmap_overwrite 9234 0 lttng_ring_buffer_client_mmap_discard 7826 0 lttng_ring_buffer_client_discard 7822 1 lttng_probe_timer 6699 12 lttng_probe_signal 3332 4 lttng_probe_sched 6649 12 lttng_probe_lttng 1086 1 lttng_tracer 24543 53 lttng_probe_statedump,lttng_probe_irq,lttng_ring_buffer_metadata_mmap_client,lttng_ring_buffer_metadata_client,lttng_ring_buffer_client_overwrite,lttng_ring_buffer_client_mmap_overwrite,lttng_ring_buffer_client_mmap_discard,lttng_ring_buffer_client_discard,lttng_probe_timer,lttng_probe_signal,lttng_probe_sched,lttng_probe_lttng lttng_statedump 3587 1 lttng_tracer lttng_lib_ring_buffer 31358 9 lttng_ring_buffer_metadata_mmap_client,lttng_ring_buffer_metadata_client,lttng_ring_buffer_client_overwrite,lttng_ring_buffer_client_mmap_overwrite,lttng_ring_buffer_client_mmap_discard,lttng_ring_buffer_client_discard,lttng_tracer ****************************************************************** The /lttng-traces/xyz-20110326-104303 is empty with no traces(only folder named kernel got created). In /dev, there is no lttng entry. In /proc, there is a file called lttng but when trying to print is says: root at arago:~/lttng-traces/xyz-20110326-104303/kernel# cat /proc/lttng cat: read error: Input/output error Thanks in Advance, Pavan -----Original Message----- From: David Goulet [mailto:dgoulet at efficios.com] Sent: Tuesday, June 12, 2012 9:05 PM To: Pavan Anumula Cc: lttng-dev at lists.lttng.org Subject: Re: [lttng-dev] Lttng start failed -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Pavan, The output below is normal and should be working fine. However, the problem is that you have _NO_ debug message after the "lttng start". That command should at least produce 10+ lines of messages after this last line: "DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1423]" Can you enlight me and tell me if by doing "lttng create newsessiom" and "lttng enable-event -a -k", you have output messages coming out of the session daemon in -vvv mode ? If NOT, you are talking to another daemon! A quick "ps faux | grep lttng-sessiond" could help clear that question. If only one daemon is running and stuck at the above debug message, the next step is to tell me on what the daemon is waiting on using either "strace -p" or "gdb" also. There is 6 threads so having the strace -p output for all of them would help me greatly. Thanks! David On 12/06/12 10:05 AM, Mathieu Desnoyers wrote: > * Pavan Anumula (pavan.anumula at sasken.com) wrote: >> Hi Mathieu, Still I am unable to start tracing, And I am facing the >> same start errors. Please kindly help me on this. This time I loaded >> the modules by modprobe( not with insmod as I did before), Then I >> run the command " arm-none-linux-gnueabi-lttng-sessiond -vvv " >> (before arm-none-linux-gnueabi-lttng-sessiond -d), > > When using "arm-none-linux-gnueabi-lttng-sessiond -vvv", you should > _not_ run "arm-none-linux-gnueabi-lttng-sessiond -d". > > >> The below is the Output, And It got hanged overthere. > > This is normal: the sessiond is waiting for applications to connect. > It does not hang, it just waits. > > David, can you follow up on this issue ? It seems to be sessiond-related. > > Thanks, > > Mathieu > > >> Later when I started lttng start I am acing the same erros below: >> root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng start LTTng: >> Failure to write metadata to buffers (timeout) Error: Starting kernel >> trace failed >> >> ********************************************************************* >> *************************** >> >> DEBUG3: Creating LTTng run directory: /var/run/lttng [in create_lttng_rundir() at main.c:4315] >> DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in >> main() at main.c:4543] DEBUG2: Kernel consumer cmd path: >> /var/run/lttng/kconsumerd/command [in main() at main.c:4545] DEBUG1: >> Client socket path /var/run/lttng/client-lttng-sessiond [in main() at >> main.c:4592] DEBUG1: Application socket path >> /var/run/lttng/apps-lttng-sessiond [in main() at main.c:4593] DEBUG1: >> LTTng run directory path: /var/run/lttng [in main() at main.c:4594] >> DEBUG2: UST consumer 32 bits err path: >> /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] DEBUG2: >> UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command >> [in >> main() at main.c:4605] DEBUG2: UST consumer 64 bits err path: >> /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] DEBUG2: >> UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command >> [in >> main() at main.c:4616] DEBUG3: Created hashtable size 4 at 0x4a080 of >> type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created >> hashtable size 4 at 0x4a168 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: >> Creating consumer directory: /var/run/lttng/kconsumerd [in >> set_consumer_sockets() at main.c:4357] DEBUG1: Modprobe successfully >> lttng-tracer [in modprobe_lttng_control() at modprobe.c:163] DEBUG2: >> Kernel tracer version validated (major version 2) [in >> kernel_validate_version() at kernel.c:675] DEBUG1: Modprobe >> successfully lttng-ftrace [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: >> Modprobe successfully lttng-kprobes [in modprobe_lttng_data() at >> modprobe.c:199] DEBUG1: Modprobe successfully lttng-kretprobes [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >> successfully lttng-lib-ring-buffer [in modprobe_lttng_data() at >> modprobe.c:199] >> DEBUG1: Modprobe successfully lttng-ring-buffer-client-discard [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >> successfully lttng-ring-buffer-client-overwrite [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >> successfully lttng-ring-buffer-metadata-client [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >> successfully lttng-ring-buffer-client-mmap-discard [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >> successfully lttng-ring-buffer-client-mmap-overwrite [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >> successfully lttng-ring-buffer-metadata-mmap-client [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >> successfully lttng-probe-lttng [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >> successfully lttng-types [in modprobe_lttng_data() at modprobe.c:199] >> DEBUG1: Modprobe successfully lttng-probe-block [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >> successfully lttng-probe-irq [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >> successfully lttng-probe-kvm [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: >> Modprobe successfully lttng-probe-sched [in modprobe_lttng_data() at >> modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-signal [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >> successfully lttng-probe-statedump [in modprobe_lttng_data() at >> modprobe.c:199] >> DEBUG1: Modprobe successfully lttng-probe-timer [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Kernel tracer fd 6 >> [in init_kernel_tracer() at main.c:1887] DEBUG2: Creating consumer directory: >> /var/run/lttng/ustconsumerd64 [in set_consumer_sockets() at >> main.c:4357] >> DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd32 >> [in >> set_consumer_sockets() at main.c:4357] DEBUG1: Signal handler set for >> SIGTERM, SIGPIPE and SIGINT [in set_signal_handler() at main.c:4449] >> DEBUG1: All permissions are set [in set_permissions() at main.c:4250] >> DEBUG1: poll set max size set to 65535 [in compat_poll_set_max_size() >> at compat-poll.c:196] DEBUG1: [thread] Manage application started [in >> thread_manage_apps() at main.c:1179] DEBUG1: fd 3 of 1 added to >> pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 13 of 2 >> added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: >> Apps thread polling on 2 fds [in thread_manage_apps() at main.c:1200] >> DEBUG1: Thread manage kernel started [in thread_manage_kernel() at main.c:876] DEBUG1: >> fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] >> DEBUG1: fd 11 of 2 added to pollfd [in compat_poll_add() at >> compat-poll.c:91] DEBUG1: Updating kernel poll set [in >> update_kernel_poll() at main.c:748] DEBUG1: Thread kernel polling on >> 2 fds [in thread_manage_kernel() at main.c:905] DEBUG1: [thread] >> Manage application registration started [in >> thread_registration_apps() at main.c:1392] DEBUG1: fd 3 of 1 added to >> pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 10 of 2 >> added to pollfd [in >> compat_poll_add() at compat-poll.c:91] DEBUG1: Notifying applications >> of session daemon state: 1 [in notify_ust_apps() at main.c:687] DEBUG1: >> [thread] Dispatch UST command started [in >> thread_dispatch_ust_registration() at main.c:1324] DEBUG1: Futex n to >> 1 prepare done [in futex_nto1_prepare() at futex.c:73] DEBUG1: Woken >> up but nothing in the UST command queue [in >> thread_dispatch_ust_registration() >> at main.c:1334] DEBUG1: [thread] Manage client started [in >> thread_manage_clients() at main.c:3794] DEBUG1: fd 3 of 1 added to >> pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 9 of 2 >> added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: >> Accepting client command ... [in thread_manage_clients() at main.c:3826] DEBUG1: >> Got the wait shm fd 15 [in get_wait_shm() at shm.c:117] DEBUG1: Futex >> wait update active 1 [in futex_wait_update() at futex.c:62] DEBUG1: >> Accepting application registration [in thread_registration_apps() at >> main.c:1423] >> ********************************************************************* >> *********************************** >> >> Am I missing anything?? >> >> Thanks in Advance, Pavan >> >> >> ________________________________________ From: Mathieu Desnoyers >> [mathieu.desnoyers at efficios.com] Sent: Monday, June 11, 2012 7:24 PM To: >> Pavan Anumula Cc: lttng-dev at lists.lttng.org Subject: Re: [lttng-dev] >> Lttng start failed >> >> * Pavan Anumula (pavan.anumula at sasken.com) wrote: >>> Hi Mathieu, >>> >>> Please find the info below as per your comments >>> >>> >>> ########## Output of command "arm-none-linux-gnueabi-lttng-sessiond >>> -vvv " , After loading the modules ####### >>> >>> DEBUG3: Creating LTTng run directory: /var/run/lttng [in >>> create_lttng_rundir() at main.c:4315] DEBUG2: Kernel consumer err path: >>> /var/run/lttng/kconsumerd/error [in main() at main.c:4543] DEBUG2: >>> Kernel consumer cmd path: /var/run/lttng/kconsumerd/command [in >>> main() at main.c:4545] DEBUG1: Client socket path >>> /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] DEBUG1: >>> Application socket path /var/run/lttng/apps-lttng-sessiond [in >>> main() at main.c:4593] DEBUG1: LTTng run directory path: >>> /var/run/lttng [in >>> main() at main.c:4594] DEBUG2: UST consumer 32 bits err path: >>> /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] DEBUG2: >>> UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command >>> [in main() at main.c:4605] DEBUG2: UST consumer 64 bits err path: >>> /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] DEBUG2: >>> UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command >>> [in main() at main.c:4616] DEBUG3: Created hashtable size 4 at >>> 0x4a080 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: >>> Created hashtable size 4 at 0x4a168 of type 1 [in lttng_ht_new() at >>> hashtable.c:96] DEBUG2: Creating consumer directory: >>> /var/run/lttng/kconsumerd [in set_consumer_sockets() at main.c:4357] >>> FATAL: Module lttng_tracer not found. Error: Unable to load module >>> lttng-tracer >> >> Hrm. You want to do kernel tracing, but modprobe cannot find the >> lttng kernel tracer modules. You might want to run depmod -a or >> something like that on your target, and ensure that modprobe works properly. >> >> Thanks, >> >> Mathieu >> >>> DEBUG2: Kernel tracer version validated (major version 2) [in >>> kernel_validate_version() at kernel.c:675] DEBUG1: Modprobe >>> successfully lttng-ftrace [in modprobe_lttng_data() at >>> modprobe.c:199] >>> DEBUG1: Modprobe successfully lttng-kprobes [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>> successfully lttng-kretprobes [in >>> modprobe_lttng_data() at modprobe.c:199] FATAL: Module >>> lttng_lib_ring_buffer not found. Error: Unable to load module >>> lttng-lib-ring-buffer FATAL: Module lttng_ring_buffer_client_discard >>> not found. Error: Unable to load module >>> lttng-ring-buffer-client-discard FATAL: Module >>> lttng_ring_buffer_client_overwrite not found. Error: Unable to load >>> module lttng-ring-buffer-client-overwrite FATAL: Module >>> lttng_ring_buffer_metadata_client not found. Error: Unable to load >>> module lttng-ring-buffer-metadata-client FATAL: Module >>> lttng_ring_buffer_client_mmap_discard not found. Error: Unable to >>> load module lttng-ring-buffer-client-mmap-discard FATAL: Module >>> lttng_ring_buffer_client_mmap_overwrite not found. Error: Unable to >>> load module lttng-ring-buffer-client-mmap-overwrite FATAL: Module >>> lttng_ring_buffer_metadata_mmap_client not found. Error: Unable to >>> load module lttng-ring-buffer-metadata-mmap-client FATAL: Module >>> lttng_probe_lttng not found. Error: Unable to load module >>> lttng-probe-lttng DEBUG1: Modprobe successfully lttng-types [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>> successfully lttng-probe-block [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: >>> Modprobe successfully lttng-probe-irq [in modprobe_lttng_data() at >>> modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-kvm [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>> successfully lttng-probe-sched [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: >>> Modprobe successfully lttng-probe-signal [in modprobe_lttng_data() >>> at modprobe.c:199] DEBUG1: Modprobe successfully >>> lttng-probe-statedump [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>> successfully lttng-probe-timer [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: >>> Kernel tracer fd 6 [in init_kernel_tracer() at main.c:1887] DEBUG2: >>> Creating consumer directory: /var/run/lttng/ustconsumerd64 [in >>> set_consumer_sockets() at main.c:4357] DEBUG2: Creating consumer >>> directory: /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() >>> at main.c:4357] DEBUG1: Signal handler set for SIGTERM, SIGPIPE and >>> SIGINT [in set_signal_handler() at main.c:4449] DEBUG1: All >>> permissions are set [in set_permissions() at main.c:4250] DEBUG1: >>> poll set max size set to 65535 [in compat_poll_set_max_size() at compat-poll.c:196] DEBUG1: >>> [thread] Manage client started [in thread_manage_clients() at >>> main.c:3794] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() >>> at compat-poll.c:91] DEBUG1: fd 9 of 2 added to pollfd [in >>> compat_poll_add() at compat-poll.c:91] DEBUG1: Accepting client >>> command ... [in thread_manage_clients() at main.c:3826] DEBUG1: >>> [thread] Dispatch UST command started [in >>> thread_dispatch_ust_registration() at main.c:1324] DEBUG1: Futex n >>> to 1 prepare done [in futex_nto1_prepare() at futex.c:73] DEBUG1: >>> Woken up but nothing in the UST command queue [in >>> thread_dispatch_ust_registration() at main.c:1334] DEBUG1: Thread manage kernel started [in thread_manage_kernel() at main.c:876] DEBUG1: >>> fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] >>> DEBUG1: fd 11 of 2 added to pollfd [in compat_poll_add() at >>> compat-poll.c:91] DEBUG1: Updating kernel poll set [in >>> update_kernel_poll() at main.c:748] DEBUG1: Thread kernel polling on >>> 2 fds [in thread_manage_kernel() at main.c:905] DEBUG1: [thread] >>> Manage application started [in thread_manage_apps() at main.c:1179] >>> DEBUG1: fd >>> 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] >>> DEBUG1: fd 13 of 2 added to pollfd [in compat_poll_add() at >>> compat-poll.c:91] DEBUG1: Apps thread polling on 2 fds [in >>> thread_manage_apps() at main.c:1200] DEBUG1: [thread] Manage >>> application registration started [in thread_registration_apps() at >>> main.c:1392] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() >>> at compat-poll.c:91] DEBUG1: fd 10 of 2 added to pollfd [in >>> compat_poll_add() at compat-poll.c:91] DEBUG1: Notifying >>> applications of session daemon state: 1 [in notify_ust_apps() at main.c:687] DEBUG1: >>> Got the wait shm fd 15 [in get_wait_shm() at shm.c:117] DEBUG1: >>> Futex wait update active 1 [in futex_wait_update() at futex.c:62] DEBUG1: >>> Accepting application registration [in thread_registration_apps() at >>> main.c:1423] >>> >>> >>> >>> #### Output of command "arm-none-linux-gnueabi-lttng-sessiond -vvv " >>> before loading the modules ############# >>> >>> DEBUG3: Creating LTTng run directory: /var/run/lttng [in >>> create_lttng_rundir() at main.c:4315] DEBUG2: Kernel consumer err path: >>> /var/run/lttng/kconsumerd/error [in main() at main.c:4543] DEBUG2: >>> Kernel consumer cmd path: /var/run/lttng/kconsumerd/command [in >>> main() at main.c:4545] DEBUG1: Client socket path >>> /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] DEBUG1: >>> Application socket path /var/run/lttng/apps-lttng-sessiond [in >>> main() at main.c:4593] DEBUG1: LTTng run directory path: >>> /var/run/lttng [in >>> main() at main.c:4594] DEBUG2: UST consumer 32 bits err path: >>> /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] DEBUG2: >>> UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command >>> [in main() at main.c:4605] DEBUG2: UST consumer 64 bits err path: >>> /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] DEBUG2: >>> UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command >>> [in main() at main.c:4616] DEBUG3: Created hashtable size 4 at >>> 0x4a080 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: >>> Created hashtable size 4 at 0x4a168 of type 1 [in lttng_ht_new() at >>> hashtable.c:96] DEBUG2: Creating consumer directory: >>> /var/run/lttng/kconsumerd [in set_consumer_sockets() at main.c:4357] >>> FATAL: Module lttng_tracer not found. Error: Unable to load module >>> lttng-tracer DEBUG1: Failed to open /proc/lttng [in >>> init_kernel_tracer() at main.c:1871] Error: Unable to remove module >>> lttng-tracer Warning: No kernel tracer available DEBUG2: Creating >>> consumer directory: /var/run/lttng/ustconsumerd64 [in >>> set_consumer_sockets() at main.c:4357] DEBUG2: Creating consumer >>> directory: /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() >>> at main.c:4357] DEBUG1: Signal handler set for SIGTERM, SIGPIPE and >>> SIGINT [in set_signal_handler() at main.c:4449] DEBUG1: All >>> permissions are set [in set_permissions() at main.c:4250] DEBUG1: >>> poll set max size set to 65535 [in compat_poll_set_max_size() at compat-poll.c:196] DEBUG1: >>> [thread] Manage application started [in thread_manage_apps() at >>> main.c:1179] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() >>> at compat-poll.c:91] DEBUG1: fd 12 of 2 added to pollfd [in >>> compat_poll_add() at compat-poll.c:91] DEBUG1: Apps thread polling >>> on 2 fds [in thread_manage_apps() at main.c:1200] DEBUG1: Thread >>> manage kernel started [in thread_manage_kernel() at main.c:876] >>> DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: >>> fd 10 of 2 added to pollfd [in compat_poll_add() at >>> compat-poll.c:91] >>> DEBUG1: Updating kernel poll set [in update_kernel_poll() at >>> main.c:748] DEBUG1: Thread kernel polling on 2 fds [in >>> thread_manage_kernel() at main.c:905] DEBUG1: [thread] Manage >>> application registration started [in thread_registration_apps() at >>> main.c:1392] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() >>> at compat-poll.c:91] DEBUG1: fd 9 of 2 added to pollfd [in >>> compat_poll_add() at compat-poll.c:91] DEBUG1: Notifying >>> applications of session daemon state: 1 [in notify_ust_apps() at main.c:687] DEBUG1: >>> [thread] Dispatch UST command started [in >>> thread_dispatch_ust_registration() at main.c:1324] DEBUG1: Futex n >>> to 1 prepare done [in futex_nto1_prepare() at futex.c:73] DEBUG1: >>> Woken up but nothing in the UST command queue [in >>> thread_dispatch_ust_registration() at main.c:1334] DEBUG1: [thread] >>> Manage client started [in thread_manage_clients() at main.c:3794] >>> DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at >>> compat-poll.c:91] DEBUG1: fd 8 of 2 added to pollfd [in >>> compat_poll_add() at compat-poll.c:91] DEBUG1: Accepting client >>> command ... [in thread_manage_clients() at main.c:3826] DEBUG1: Got >>> the wait shm fd 14 [in get_wait_shm() at shm.c:117] DEBUG1: Futex >>> wait update active 1 [in futex_wait_update() at futex.c:62] DEBUG1: >>> Accepting application registration [in thread_registration_apps() at >>> main.c:1423] >>> >>> >>> Regards, Pavan >>> >>> -----Original Message----- From: Mathieu Desnoyers >>> [mailto:mathieu.desnoyers at efficios.com] Sent: Saturday, June 09, >>> 2012 >>> 6:03 PM To: Pavan Anumula Cc: lttng-dev at lists.lttng.org Subject: Re: >>> [lttng-dev] Lttng start failed >>> >>> * Pavan Anumula (pavan.anumula at sasken.com) wrote: >>>> Hi Mathue, >>>> >>>> Thanks for the quick reply, >>>> >>>> After inserting lttng modules , I had given the command >>>> "arm-none-linux-gnueabi-lttng-sessiond -vvv" as you said, Below is >>>> the output where there are error messages. Please help me in >>>> resolving the issue, SO that I can catch kernel and user traces. >>>> >>>> >>>> root at arago:/usr/lttng/modules# >>>> arm-none-linux-gnueabi-lttng-sessiond >>>> -d root at arago:/usr/lttng/modules# >>>> arm-none-linux-gnueabi-lttng-sessiond -vvv DEBUG3: Creating LTTng >>>> run >>>> directory: /var/run/lttng [in create_lttng_rundir() at main.c:4315] >>>> DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error >>>> [in main() at main.c:4543] DEBUG2: Kernel consumer cmd path: >>>> /var/run/lttng/kconsumerd/command [in main() at main.c:4545] DEBUG1: >>>> Client socket path /var/run/lttng/client-lttng-sessiond [in main() >>>> at main.c:4592] DEBUG1: Application socket path >>>> /var/run/lttng/apps-lttng-sessiond [in main() at main.c:4593] DEBUG1: >>>> LTTng run directory path: /var/run/lttng [in main() at main.c:4594] >>>> DEBUG2: UST consumer 32 bits err path: >>>> /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] >>>> DEBUG2: UST consumer 32 bits cmd path: >>>> /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] >>>> DEBUG2: UST consumer 64 bits err path: >>>> /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] >>>> DEBUG2: UST consumer 64 bits cmd path: >>>> /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] >>>> Error: Already running daemon. >>> >>> Please kill the lttng-sessiond that is already started (the one with >>> -d). Instead of that one, run the sessiond with: >>> >>> lttng-sessiond -vvv >>> >>> (don't run lttng-sessiond -d before) >>> >>> Thanks, >>> >>> Mathieu >>> >>>> >>>> >>>> >>>> >>>> Thanks in advance, Pavan >>>> >>>> -----Original Message----- From: Mathieu Desnoyers >>>> [mailto:mathieu.desnoyers at efficios.com] Sent: Friday, June 08, 2012 >>>> 11:12 PM To: Pavan Anumula Cc: lttng-dev at lists.lttng.org Subject: Re: >>>> [lttng-dev] Lttng start failed >>>> >>>> * Pavan Anumula (pavan.anumula at sasken.com) wrote: >>>>> Hi , >>>>> >>>>> I am new to LTTng usage, I am trying to use LTTng 2.0.1 and >>>>> LTTng-modules-2.0.2 for ARM(omapL138), with Linux kernel 2.6.33 >>>>> wit RT-29 patch on it. >>>>> >>>>> After enabling all the kernel events I am facing the below errors. >>>>> I loaded all the modules manually by insmod. >>>>> >>>>> >>>>> >>>>> root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng create >>>>> newsessiom Session newsessiom created. Traces will be written in >>>>> /home/root/lttng-traces/newsessiom-20110325-154922 >>>>> >>>>> root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng >>>>> enable-event -a --kernel All kernel events are enabled in channel >>>>> channel0 >>>>> >>>>> root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng start >>>>> LTTng: Failure to write metadata to buffers (timeout) Error: >>>>> Starting kernel trace failed >>>>> >>>>> Please kindly help me on this. >>>> >>>> I think you should look into the lttng-sessiond --help : >>>> >>>> --consumerd32-path PATH Specify path for the 32-bit UST consumer >>>> daemon binary --consumerd32-libdir PATH Specify path for the 32-bit >>>> UST consumer daemon libraries --consumerd64-path PATH Specify >>>> path for the 64-bit UST consumer daemon binary --consumerd64-libdir >>>> PATH Specify path for the 64-bit UST consumer daemon libraries >>>> >>>> options. My guess is that lttng-sessiond is not able to find the >>>> consumerd binary files, maybe due to a rename or because they have >>>> been moved after install. >>>> >>>> One more thing that might help is to launch the lttng-sessiond with >>>> "-vvv" : it will provide verbose output and let us know where >>>> things fall apart. >>>> >>>> Thanks, >>>> >>>> Mathieu >>>> >>>> >>>>> >>>>> >>>>> Thanks in advance, Pavan >>>>> >>>>> ________________________________ SASKEN BUSINESS DISCLAIMER: This >>>>> message may contain confidential, proprietary or legally >>>>> privileged information. In case you are not the original intended >>>>> Recipient of the message, you must not, directly or indirectly, >>>>> use, disclose, distribute, print, or copy any part of this message >>>>> and you are requested to delete it and inform the sender. Any >>>>> views expressed in this message are those of the individual sender >>>>> unless otherwise stated. Nothing contained in this message shall >>>>> be construed as an offer or acceptance of any offer by Sasken >>>>> Communication Technologies Limited ("Sasken") unless sent with >>>>> that express intent and with due authority of Sasken. Sasken has >>>>> taken enough precautions to prevent the spread of viruses. However >>>>> the company accepts no liability for any damage caused by any >>>>> virus transmitted by this email. Read Disclaimer at >>>>> http://www.sasken.com/extras/mail_disclaimer.html >>>> >>>>> _______________________________________________ lttng-dev mailing >>>>> list lttng-dev at lists.lttng.org >>>>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev >>>> >>>> >>>> -- Mathieu Desnoyers Operating System Efficiency R&D Consultant >>>> EfficiOS Inc. http://www.efficios.com >>> >>> -- Mathieu Desnoyers Operating System Efficiency R&D Consultant >>> EfficiOS Inc. http://www.efficios.com >> >> -- Mathieu Desnoyers Operating System Efficiency R&D Consultant >> EfficiOS Inc. http://www.efficios.com > -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBAgAGBQJP12GMAAoJEELoaioR9I02+q8IAKw/ot4BmP6xibXM/VnNaRdP 60S1KEynFPxDmO9JS3u3x+r4UiDaGWrP4+rEfgAeUeMQcOj3ItHZLyNhp+F8A8+j Dj0q9YFNBLsANzHzAk9VsVlL/myYaeW7ysNGl1yxhL9Vc/Px3cmxaof79smxMGcQ dnMKf6Gk+eNL4IPrXQIxF0cvVK85PI2xaDA1SiiKQaULUiqUBebn2gtGr1l8oFTw FT1xryh4wRw4FUmqngw54SYPiYpZm1y/VUzTWB3Zs9hYPcvEum22wrhAaWNkoUkk ixBjE+HHIVRBzZatDymE35tI4+Ci6z9J3uRmJVC97x2qQz2F4nl5Yw3qRmm1PVM= =3VyT -----END PGP SIGNATURE----- From toojays at toojays.net Wed Jun 13 09:11:57 2012 From: toojays at toojays.net (John Steele Scott) Date: Wed, 13 Jun 2012 22:41:57 +0930 Subject: [lttng-dev] urcu commit a767fd requires autoconf >= 2.64. In-Reply-To: <20120613081001.GA30123@Krystal> References: <20120613081001.GA30123@Krystal> Message-ID: <4FD8919D.9060607@toojays.net> On 13/06/12 17:40, Mathieu Desnoyers wrote: > * John Steele Scott (toojays at toojays.net) wrote: >> http://lists.lttng.org/pipermail/lttng-dev/2012-May/017927.html >> >> I tried to build the latest urcu (git master e51500) on a Centos 6.2 box, and got: >> >> jscott at dxi0-62:~/src/userspace-rcu$ make -j4 >> CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /users/jscott/src/userspace-rcu/config/missing --run aclocal-1.11 -I config >> CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /users/jscott/src/userspace-rcu/config/missing --run autoconf >> cd . && /bin/sh /users/jscott/src/userspace-rcu/config/missing --run automake-1.11 --foreign >> configure:4010: error: possibly undefined macro: m4_ifnblank >> If this token and others are legitimate, please use m4_pattern_allow. >> See the Autoconf documentation. >> make: *** [configure] Error 1 >> make: *** Waiting for unfinished jobs.... >> >> Some digging showed that the macro m4_ifnblank requires autoconf 2.64. Centos 6.2 has autoconf 2.63. :( >> >> I just worked around it by reverting commit a767fd locally, then I can build fine. > Thanks for pointing this out! Can you try the following patch and let me > know if it fixes your issue ? > > Mathieu, Thanks for your quick response. Unfortunately, with that patch, ./configure fails like: checking for thread local storage (TLS) class... __thread ./configure: line 4029: syntax error near unexpected token `fi' ./configure: line 4029: `fi' The section of configure which it's complaining about looks like: if test "$ac_cv_tls" != "none"; then cat >>confdefs.h <<_ACEOF #define TLS $ac_cv_tls _ACEOF cat >>confdefs.h <<_ACEOF #define CONFIG_RCU_TLS $ac_cv_tls _ACEOF else fi It seems it doesn't like the empty else..fi clause. If I put a command in there ("true", "echo", whatever), configure completes and I can build successfully. cheers, John From mathieu.desnoyers at efficios.com Wed Jun 13 15:31:42 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Wed, 13 Jun 2012 15:31:42 -0400 Subject: [lttng-dev] urcu commit a767fd requires autoconf >= 2.64. In-Reply-To: <4FD8919D.9060607@toojays.net> References: <20120613081001.GA30123@Krystal> <4FD8919D.9060607@toojays.net> Message-ID: <20120613193142.GA20907@Krystal> * John Steele Scott (toojays at toojays.net) wrote: > On 13/06/12 17:40, Mathieu Desnoyers wrote: > > * John Steele Scott (toojays at toojays.net) wrote: > >> http://lists.lttng.org/pipermail/lttng-dev/2012-May/017927.html > >> > >> I tried to build the latest urcu (git master e51500) on a Centos 6.2 box, and got: > >> > >> jscott at dxi0-62:~/src/userspace-rcu$ make -j4 > >> CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /users/jscott/src/userspace-rcu/config/missing --run aclocal-1.11 -I config > >> CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /users/jscott/src/userspace-rcu/config/missing --run autoconf > >> cd . && /bin/sh /users/jscott/src/userspace-rcu/config/missing --run automake-1.11 --foreign > >> configure:4010: error: possibly undefined macro: m4_ifnblank > >> If this token and others are legitimate, please use m4_pattern_allow. > >> See the Autoconf documentation. > >> make: *** [configure] Error 1 > >> make: *** Waiting for unfinished jobs.... > >> > >> Some digging showed that the macro m4_ifnblank requires autoconf 2.64. Centos 6.2 has autoconf 2.63. :( > >> > >> I just worked around it by reverting commit a767fd locally, then I can build fine. > > Thanks for pointing this out! Can you try the following patch and let me > > know if it fixes your issue ? > > > > > > Mathieu, > > Thanks for your quick response. Unfortunately, with that patch, ./configure fails like: > [...] > It seems it doesn't like the empty else..fi clause. If I put a command in there ("true", "echo", whatever), configure completes and I can build successfully. Can you try with the following ? I just tested it with autoconf 2.63 here and it seems to work fine now. diff --git a/config/ax_tls.m4 b/config/ax_tls.m4 index 033e3b1..5ab1a41 100644 --- a/config/ax_tls.m4 +++ b/config/ax_tls.m4 @@ -44,7 +44,23 @@ # modified version of the Autoconf Macro, you may extend this special # exception to the GPL to apply to your modified version as well. -#serial 10 +#serial 11 + +# Define m4_ifblank and m4_ifnblank macros from introduced in +# autotools 2.64 m4sugar.m4 if using an earlier autotools. + +ifdef([m4_ifblank], [], [ +m4_define([m4_ifblank], +[m4_if(m4_translit([[$1]], [ ][ ][ +]), [], [$2], [$3])]) +]) + + +ifdef([m4_ifnblank], [], [ +m4_define([m4_ifnblank], +[m4_if(m4_translit([[$1]], [ ][ ][ +]), [], [$3], [$2])]) +]) AC_DEFUN([AX_TLS], [ AC_MSG_CHECKING(for thread local storage (TLS) class) diff --git a/configure.ac b/configure.ac index db34935..5f6bc40 100644 --- a/configure.ac +++ b/configure.ac @@ -28,7 +28,7 @@ AH_TEMPLATE([CONFIG_RCU_COMPAT_ARCH], [Compatibility mode for i386 which lacks c AH_TEMPLATE([CONFIG_RCU_ARM_HAVE_DMB], [Use the dmb instruction if available for use on ARM.]) AH_TEMPLATE([CONFIG_RCU_TLS], [TLS provided by the compiler.]) -AX_TLS([AC_DEFINE_UNQUOTED([CONFIG_RCU_TLS], $ac_cv_tls)], []) +AX_TLS(AC_DEFINE_UNQUOTED([CONFIG_RCU_TLS], $ac_cv_tls), [:]) # Checks for programs. AC_PROG_CC -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From toojays at toojays.net Wed Jun 13 19:17:37 2012 From: toojays at toojays.net (John Steele Scott) Date: Thu, 14 Jun 2012 08:47:37 +0930 Subject: [lttng-dev] urcu commit a767fd requires autoconf >= 2.64. In-Reply-To: <20120613193142.GA20907@Krystal> References: <20120613081001.GA30123@Krystal> <4FD8919D.9060607@toojays.net> <20120613193142.GA20907@Krystal> Message-ID: <4FD91F91.6080204@toojays.net> On 14/06/12 05:01, Mathieu Desnoyers wrote: > * John Steele Scott (toojays at toojays.net) wrote: >> On 13/06/12 17:40, Mathieu Desnoyers wrote: >>> * John Steele Scott (toojays at toojays.net) wrote: >>>> http://lists.lttng.org/pipermail/lttng-dev/2012-May/017927.html >>>> >>>> I tried to build the latest urcu (git master e51500) on a Centos 6.2 box, and got: >>>> >>>> jscott at dxi0-62:~/src/userspace-rcu$ make -j4 >>>> CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /users/jscott/src/userspace-rcu/config/missing --run aclocal-1.11 -I config >>>> CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /users/jscott/src/userspace-rcu/config/missing --run autoconf >>>> cd . && /bin/sh /users/jscott/src/userspace-rcu/config/missing --run automake-1.11 --foreign >>>> configure:4010: error: possibly undefined macro: m4_ifnblank >>>> If this token and others are legitimate, please use m4_pattern_allow. >>>> See the Autoconf documentation. >>>> make: *** [configure] Error 1 >>>> make: *** Waiting for unfinished jobs.... >>>> >>>> Some digging showed that the macro m4_ifnblank requires autoconf 2.64. Centos 6.2 has autoconf 2.63. :( >>>> >>>> I just worked around it by reverting commit a767fd locally, then I can build fine. >>> Thanks for pointing this out! Can you try the following patch and let me >>> know if it fixes your issue ? >>> >>> >> Mathieu, >> >> Thanks for your quick response. Unfortunately, with that patch, ./configure fails like: >> > [...] >> It seems it doesn't like the empty else..fi clause. If I put a command in there ("true", "echo", whatever), configure completes and I can build successfully. > Can you try with the following ? I just tested it with autoconf 2.63 > here and it seems to work fine now. > > > diff --git a/config/ax_tls.m4 b/config/ax_tls.m4 > index 033e3b1..5ab1a41 100644 > --- a/config/ax_tls.m4 > +++ b/config/ax_tls.m4 > @@ -44,7 +44,23 @@ > # modified version of the Autoconf Macro, you may extend this special > # exception to the GPL to apply to your modified version as well. > > -#serial 10 > +#serial 11 > + > +# Define m4_ifblank and m4_ifnblank macros from introduced in > +# autotools 2.64 m4sugar.m4 if using an earlier autotools. > + > +ifdef([m4_ifblank], [], [ > +m4_define([m4_ifblank], > +[m4_if(m4_translit([[$1]], [ ][ ][ > +]), [], [$2], [$3])]) > +]) > + > + > +ifdef([m4_ifnblank], [], [ > +m4_define([m4_ifnblank], > +[m4_if(m4_translit([[$1]], [ ][ ][ > +]), [], [$3], [$2])]) > +]) > > AC_DEFUN([AX_TLS], [ > AC_MSG_CHECKING(for thread local storage (TLS) class) > diff --git a/configure.ac b/configure.ac > index db34935..5f6bc40 100644 > --- a/configure.ac > +++ b/configure.ac > @@ -28,7 +28,7 @@ AH_TEMPLATE([CONFIG_RCU_COMPAT_ARCH], [Compatibility mode for i386 which lacks c > AH_TEMPLATE([CONFIG_RCU_ARM_HAVE_DMB], [Use the dmb instruction if available for use on ARM.]) > AH_TEMPLATE([CONFIG_RCU_TLS], [TLS provided by the compiler.]) > > -AX_TLS([AC_DEFINE_UNQUOTED([CONFIG_RCU_TLS], $ac_cv_tls)], []) > +AX_TLS(AC_DEFINE_UNQUOTED([CONFIG_RCU_TLS], $ac_cv_tls), [:]) > > # Checks for programs. > AC_PROG_CC > Mathieu, This fixed it for me too. Thanks, John From mathieu.desnoyers at efficios.com Thu Jun 14 00:55:33 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Thu, 14 Jun 2012 00:55:33 -0400 Subject: [lttng-dev] urcu commit a767fd requires autoconf >= 2.64. In-Reply-To: <4FD91F91.6080204@toojays.net> References: <20120613081001.GA30123@Krystal> <4FD8919D.9060607@toojays.net> <20120613193142.GA20907@Krystal> <4FD91F91.6080204@toojays.net> Message-ID: <20120614045533.GA26983@Krystal> * John Steele Scott (toojays at toojays.net) wrote: > On 14/06/12 05:01, Mathieu Desnoyers wrote: > > * John Steele Scott (toojays at toojays.net) wrote: > >> On 13/06/12 17:40, Mathieu Desnoyers wrote: > >>> * John Steele Scott (toojays at toojays.net) wrote: > >>>> http://lists.lttng.org/pipermail/lttng-dev/2012-May/017927.html > >>>> > >>>> I tried to build the latest urcu (git master e51500) on a Centos 6.2 box, and got: > >>>> > >>>> jscott at dxi0-62:~/src/userspace-rcu$ make -j4 > >>>> CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /users/jscott/src/userspace-rcu/config/missing --run aclocal-1.11 -I config > >>>> CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh /users/jscott/src/userspace-rcu/config/missing --run autoconf > >>>> cd . && /bin/sh /users/jscott/src/userspace-rcu/config/missing --run automake-1.11 --foreign > >>>> configure:4010: error: possibly undefined macro: m4_ifnblank > >>>> If this token and others are legitimate, please use m4_pattern_allow. > >>>> See the Autoconf documentation. > >>>> make: *** [configure] Error 1 > >>>> make: *** Waiting for unfinished jobs.... > >>>> > >>>> Some digging showed that the macro m4_ifnblank requires autoconf 2.64. Centos 6.2 has autoconf 2.63. :( > >>>> > >>>> I just worked around it by reverting commit a767fd locally, then I can build fine. > >>> Thanks for pointing this out! Can you try the following patch and let me > >>> know if it fixes your issue ? > >>> > >>> > >> Mathieu, > >> > >> Thanks for your quick response. Unfortunately, with that patch, ./configure fails like: > >> > > [...] > >> It seems it doesn't like the empty else..fi clause. If I put a command in there ("true", "echo", whatever), configure completes and I can build successfully. > > Can you try with the following ? I just tested it with autoconf 2.63 > > here and it seems to work fine now. > > > > > > diff --git a/config/ax_tls.m4 b/config/ax_tls.m4 > > index 033e3b1..5ab1a41 100644 > > --- a/config/ax_tls.m4 > > +++ b/config/ax_tls.m4 > > @@ -44,7 +44,23 @@ > > # modified version of the Autoconf Macro, you may extend this special > > # exception to the GPL to apply to your modified version as well. > > > > -#serial 10 > > +#serial 11 > > + > > +# Define m4_ifblank and m4_ifnblank macros from introduced in > > +# autotools 2.64 m4sugar.m4 if using an earlier autotools. > > + > > +ifdef([m4_ifblank], [], [ > > +m4_define([m4_ifblank], > > +[m4_if(m4_translit([[$1]], [ ][ ][ > > +]), [], [$2], [$3])]) > > +]) > > + > > + > > +ifdef([m4_ifnblank], [], [ > > +m4_define([m4_ifnblank], > > +[m4_if(m4_translit([[$1]], [ ][ ][ > > +]), [], [$3], [$2])]) > > +]) > > > > AC_DEFUN([AX_TLS], [ > > AC_MSG_CHECKING(for thread local storage (TLS) class) > > diff --git a/configure.ac b/configure.ac > > index db34935..5f6bc40 100644 > > --- a/configure.ac > > +++ b/configure.ac > > @@ -28,7 +28,7 @@ AH_TEMPLATE([CONFIG_RCU_COMPAT_ARCH], [Compatibility mode for i386 which lacks c > > AH_TEMPLATE([CONFIG_RCU_ARM_HAVE_DMB], [Use the dmb instruction if available for use on ARM.]) > > AH_TEMPLATE([CONFIG_RCU_TLS], [TLS provided by the compiler.]) > > > > -AX_TLS([AC_DEFINE_UNQUOTED([CONFIG_RCU_TLS], $ac_cv_tls)], []) > > +AX_TLS(AC_DEFINE_UNQUOTED([CONFIG_RCU_TLS], $ac_cv_tls), [:]) > > > > # Checks for programs. > > AC_PROG_CC > > > > Mathieu, > > This fixed it for me too. Thanks for confirming. Pushed as commit: commit 450b97095e27646fcd1e4b83c99477d7253b987b Author: Mathieu Desnoyers Date: Thu Jun 14 00:56:40 2012 -0400 Fix: re-enable compatibility with autoconf < 2.64 > > Thanks, > > John > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Thu Jun 14 11:24:06 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Thu, 14 Jun 2012 11:24:06 -0400 Subject: [lttng-dev] [RELEASE] LTTng-UST 2.0.4 Message-ID: <20120614152406.GA2306@Krystal> LTTng-UST, the Linux Trace Toolkit Next Generation Userspace Tracer, is port of the low-overhead tracing capabilities of the LTTng kernel tracer to user-space. The library "liblttng-ust" enables tracing of applications and libraries. Note: the "liblttng-ust-fork deadlock" fix is an important fix. Anyone using the liblttng-ust-fork library is encouraged to upgrade. Changelog: 2012-06-14 lttng-ust 2.0.4 * Fix: perform TLS fixup of nest count outside of UST mutex * Fix: liblttng-ust-fork deadlock * Fix: handle pthread errors * Fix: local apps allowed should disable local (not global) tracing * Fix strict ISO-C compatibility for ust-tracepoint-event.h public header * Fix: support -std=c99 in tracepoint macros * Fix: tracepoint.h should not generate old-style definitions * Fix: don't define variables in headers * test "hello": add boolean test * Fix: perform macro expansion on tracepoint signatures * UST check pointer/de-reference order Project website: http://lttng.org Download link: http://lttng.org/download (please refer to the README file for installation instructions) -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From dgoulet at efficios.com Thu Jun 14 14:08:51 2012 From: dgoulet at efficios.com (David Goulet) Date: Thu, 14 Jun 2012 14:08:51 -0400 Subject: [lttng-dev] [RELEASE] LTTng-tools 2.0.2 In-Reply-To: <4F902D0D.9020902@efficios.com> References: <4F902D0D.9020902@efficios.com> Message-ID: <4FDA28B3.90302@efficios.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The lttng-tools project provides a session daemon (lttng-sessiond) that acts as a tracing registry, the "lttng" command line for tracing control and the liblttng-ctl library also for tracing control. 2012-06-14 lttng-tools 2.0.2 * Fix: enable event with different loglevel error * Fix: move memset in channel_set_attr after NULL check * Fix: close all file descriptors when executed as daemon * Fix: clang llvm warnings * Add CodingStyle to tarball * Add coding style document Project website: http://lttng.org/lttng2.0 Download link: http://lttng.org/files/lttng-tools/lttng-tools-2.0.2.tar.bz2 Cheers! David -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBAgAGBQJP2iizAAoJEELoaioR9I021F4IAKkugbuwgmdpMNkAuTDNTgQz TjHO2bVhGN4jlFelutytRWW92T+pY5qfivSPF+pIp4q4DD8I7srNmIzejCtMxCOH QvFBQmHrD/2iuNv8qBesZxQaJ1rbb23dBRSLN5bfmjyOUU6ryHOAL/Y3oAHeehCf A/qudeJmNyRXzYwXXli2kw6t5OvHjO0XFUEM8qoe8FZo+f56czcxeGFAMYnT+X5l gFU19ane3Dz8HE4prhwndXK/Jwjm4fD4WBbzPq/I1LqMiivMSOVWN7tg8H27WqoI gwTPZf4fjIO5Z6hER1mXAYdBu28UMCiZKAXRGBMhbqAVqx7ul1MbDd2Q2i4FDSs= =j/A2 -----END PGP SIGNATURE----- From bapi_mvit2004 at yahoo.com Fri Jun 15 03:41:28 2012 From: bapi_mvit2004 at yahoo.com (somanath sahoo) Date: Fri, 15 Jun 2012 00:41:28 -0700 (PDT) Subject: [lttng-dev] lttng - ubuntu 11.10 - problem Message-ID: <1339746088.90723.YahooMailNeo@web161201.mail.bf1.yahoo.com> Hi, As i am new to lttng usage, I have followed the instructions from in order to install the lttng tool set in my ubuntu 11.10. After successful installation, i followed the getting started link on lttng.org in "root mode". 1. when i am executing the first command - ??? # lttng list -k ??? i am getting the following error ??? " Error: Unable to list kernel events" 2. # lttng create mysession ??? Session mysession created. 3. # lttng enable-event sched_switch,sched_process_fork -k ??? Error: Event sched_switch: Kernel tracer not available (channel channel0, session mysession) ??? Error: Event sched_process_fork: Kernel tracer not available (channel channel0, session mysession) ??? Warning: Some command(s) went wrong? 4. # lttng start ??? Tracing started for session mysession 5. # lttng stop ??? Tracing stopped for session mysession 6.? # lttng destroy ??? Session mysession destroyed at /root ------------ I have checked that lttng-sessiond is running by below command # ps -ef|grep lttn root?????? 535???? 1? 0 Jun14 ???????? 00:00:00 lttng-sessiond somanath? 5359???? 1? 0 Jun14 pts/1??? 00:00:00 lttng-sessiond --sig-parent --quiet root???? 19514 19495? 0 12:44 pts/2??? 00:00:00 grep --color=auto lttn -------------- Could anyone please help me to understand or clarify why kernel tracer is not available when above point (3) / (1) is executed or whether i need to do anything more , please let me know so that i can start using lttng in ubuntu 11.10 successfully. I have googled about this problem, I got to know about this similar problem from one link - http://www.mail-archive.com/lttng-dev at lists.lttng.org/msg00763.html However i executed all this commands under root and explicit lttng-seesiond start, But still the problem is persistant. Any help will be appreciated. Thanks & Regards, Somanath -------------- next part -------------- An HTML attachment was scrubbed... URL: From francis.giraldeau at gmail.com Fri Jun 15 04:16:28 2012 From: francis.giraldeau at gmail.com (Francis Giraldeau) Date: Fri, 15 Jun 2012 10:16:28 +0200 Subject: [lttng-dev] lttng - ubuntu 11.10 - problem In-Reply-To: <1339746088.90723.YahooMailNeo@web161201.mail.bf1.yahoo.com> References: <1339746088.90723.YahooMailNeo@web161201.mail.bf1.yahoo.com> Message-ID: <4FDAEF5C.9030102@gmail.com> Le 2012-06-15 09:41, somanath sahoo a ?crit : > Hi, > > As i am new to lttng usage, I have followed the instructions from > in order to install the > lttng tool set in my ubuntu 11.10. > After successful installation, i followed the getting started link on > lttng.org in "root mode". You may verify if kernel modules are loaded correctly. If not, then there may be a problem while the installation. lsmod | grep lttng If not, you may try to load them and see if any error message occurs: sudo modprobe lttng_tracer Francis -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4476 bytes Desc: Signature cryptographique S/MIME URL: From Fredrik_Oestman at mentor.com Fri Jun 15 04:23:51 2012 From: Fredrik_Oestman at mentor.com (Oestman, Fredrik) Date: Fri, 15 Jun 2012 08:23:51 +0000 Subject: [lttng-dev] Prerequsites for adding perf context In-Reply-To: <4FDAEF5C.9030102@gmail.com> References: <1339746088.90723.YahooMailNeo@web161201.mail.bf1.yahoo.com> <4FDAEF5C.9030102@gmail.com> Message-ID: <524C960C5DFC794E82BE548D825F05CF283461BF@EU-MBX-01.mgc.mentorg.com> Hi, What exactly is needed on a system for # lttng add-context -k -t perf: to work? Does perf have to be installed? I have hw perfevents, but not perf, and get hw perfevents: unable to reserve pmu cheers, Fredrik ?stman From david.goulet at polymtl.ca Fri Jun 15 09:12:40 2012 From: david.goulet at polymtl.ca (David Goulet) Date: Fri, 15 Jun 2012 09:12:40 -0400 Subject: [lttng-dev] lttng - ubuntu 11.10 - problem In-Reply-To: <4FDAEF5C.9030102@gmail.com> References: <1339746088.90723.YahooMailNeo@web161201.mail.bf1.yahoo.com> <4FDAEF5C.9030102@gmail.com> Message-ID: <4FDB34C8.7000505@polymtl.ca> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, In order to trace the kernel, you need the LTTng kernel modules available in the package "lttng-modules-dkms". So make sure you have these three packages installed: # sudo apt-get install lttng-tools lttng-modules-dkms babeltrace The error message you got indicates that there is simply no LTTng modules loaded. Please make sure they are loaded and if the problem persist, let us know! Thanks! David On 15/06/12 04:16 AM, Francis Giraldeau wrote: > Le 2012-06-15 09:41, somanath sahoo a ?crit : >> Hi, >> >> As i am new to lttng usage, I have followed the instructions from >> in order to install the lttng >> tool set in my ubuntu 11.10. After successful installation, i followed >> the getting started link on lttng.org in >> "root mode". > > You may verify if kernel modules are loaded correctly. If not, then there > may be a problem while the installation. > > lsmod | grep lttng > > If not, you may try to load them and see if any error message occurs: > > sudo modprobe lttng_tracer > > Francis > > > > _______________________________________________ lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBAgAGBQJP2zTIAAoJEELoaioR9I02CVEH/1ripsIVxDQ2JRPNXs0TVWLD 5B/qf3oKTohuKCoKE7Q7ZLyo1A9WN0z9JFouVecEPD7MisdJqYufEXkUNbxHg2+y 9nVT8xF7gToLRBi/qC/hH0JN6AADJmqTVxYAvygv8qWNkLR3PSqguDhE5nM9vrtF LJSEYlYQh0dktkBM9ESm0lrnbZO/AtXxQB4IbCQSwP1tsNmftEdZ24H8DN0/QWl9 niRT6XpBzogNJfQQOPFtJTFxpwKHiMQXdE78UtzzMBPZ9P57Go9jMgP+2FELFZl4 wcu+XMIF2Gx81AajfTkGAjPOczagqKzrgnMCduQaNgCI1hRoWOn7Yu1jTmA6Yxc= =oy1n -----END PGP SIGNATURE----- From matthew.khouzam at ericsson.com Fri Jun 15 09:51:29 2012 From: matthew.khouzam at ericsson.com (Matthew Khouzam) Date: Fri, 15 Jun 2012 09:51:29 -0400 Subject: [lttng-dev] lttng - ubuntu 11.10 - problem In-Reply-To: <4FDB34C8.7000505@polymtl.ca> References: <1339746088.90723.YahooMailNeo@web161201.mail.bf1.yahoo.com> <4FDAEF5C.9030102@gmail.com> <4FDB34C8.7000505@polymtl.ca> Message-ID: <4FDB3DE1.70409@ericsson.com> After, check that your session daemon is loaded too. the following commands should return +- the following: lsmod |grep lttng lttng_probe_timer 15523 0 lttng_probe_statedump 15004 0 lttng_probe_signal 13567 0 lttng_probe_sched 16688 0 lttng_probe_kvm 16020 0 lttng_probe_irq 13728 0 lttng_probe_block 16630 0 lttng_types 12508 0 lttng_probe_lttng 12828 0 lttng_ring_buffer_metadata_mmap_client 13369 0 lttng_ring_buffer_client_mmap_overwrite 17622 0 lttng_ring_buffer_client_mmap_discard 17622 0 lttng_ring_buffer_metadata_client 13369 0 lttng_ring_buffer_client_overwrite 17622 0 lttng_ring_buffer_client_discard 17622 0 lttng_tracer 240715 14 lttng_probe_timer,lttng_probe_statedump,lttng_probe_signal,lttng_probe_sched,lttng_probe_kvm,lttng_probe_irq,lttng_probe_block,lttng_probe_lttng,lttng_ring_buffer_metadata_mmap_client,lttng_ring_buffer_client_mmap_overwrite,lttng_ring_buffer_client_mmap_discard,lttng_ring_buffer_metadata_client,lttng_ring_buffer_client_overwrite,lttng_ring_buffer_client_discard lttng_kretprobes 13278 1 lttng_tracer lttng_kprobes 12971 1 lttng_tracer lttng_ftrace 13162 1 lttng_tracer lttng_statedump 13903 1 lttng_tracer lttng_lib_ring_buffer 46389 7 lttng_ring_buffer_metadata_mmap_client,lttng_ring_buffer_client_mmap_overwrite,lttng_ring_buffer_client_mmap_discard,lttng_ring_buffer_metadata_client,lttng_ring_buffer_client_overwrite,lttng_ring_buffer_client_discard,lttng_tracer ps -aux |grep "ltt" root 1123 0.0 0.0 47008 728 ? Ssl Jun14 0:00 lttng-sessiond Hope this helps. Matthew On 12-06-15 09:12 AM, David Goulet wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi, > > In order to trace the kernel, you need the LTTng kernel modules available in > the package "lttng-modules-dkms". So make sure you have these three packages > installed: > > # sudo apt-get install lttng-tools lttng-modules-dkms babeltrace > > The error message you got indicates that there is simply no LTTng modules > loaded. Please make sure they are loaded and if the problem persist, let us know! > > Thanks! > David > > On 15/06/12 04:16 AM, Francis Giraldeau wrote: >> Le 2012-06-15 09:41, somanath sahoo a ?crit : >>> Hi, >>> >>> As i am new to lttng usage, I have followed the instructions from >>> in order to install the lttng >>> tool set in my ubuntu 11.10. After successful installation, i followed >>> the getting started link on lttng.org in >>> "root mode". >> You may verify if kernel modules are loaded correctly. If not, then there >> may be a problem while the installation. >> >> lsmod | grep lttng >> >> If not, you may try to load them and see if any error message occurs: >> >> sudo modprobe lttng_tracer >> >> Francis >> >> >> >> _______________________________________________ lttng-dev mailing list >> lttng-dev at lists.lttng.org >> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.12 (GNU/Linux) > > iQEcBAEBAgAGBQJP2zTIAAoJEELoaioR9I02CVEH/1ripsIVxDQ2JRPNXs0TVWLD > 5B/qf3oKTohuKCoKE7Q7ZLyo1A9WN0z9JFouVecEPD7MisdJqYufEXkUNbxHg2+y > 9nVT8xF7gToLRBi/qC/hH0JN6AADJmqTVxYAvygv8qWNkLR3PSqguDhE5nM9vrtF > LJSEYlYQh0dktkBM9ESm0lrnbZO/AtXxQB4IbCQSwP1tsNmftEdZ24H8DN0/QWl9 > niRT6XpBzogNJfQQOPFtJTFxpwKHiMQXdE78UtzzMBPZ9P57Go9jMgP+2FELFZl4 > wcu+XMIF2Gx81AajfTkGAjPOczagqKzrgnMCduQaNgCI1hRoWOn7Yu1jTmA6Yxc= > =oy1n > -----END PGP SIGNATURE----- > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev From pavan.anumula at sasken.com Fri Jun 15 10:50:55 2012 From: pavan.anumula at sasken.com (Pavan Anumula) Date: Fri, 15 Jun 2012 20:20:55 +0530 Subject: [lttng-dev] Empty traces for UST Message-ID: <47D610AD9C485E458630BA38C324D3B60113FB00FBB0@EXGMBX01.sasken.com> Hi David, Finally I was able to get kernel traces by referring to one of your other posts , Actually lttng-consumerd binary was actually not present at the needed path, Thanks for your help. And sorry for bugging you with mails :) . But this time I am facing issues at UST tracing . Please kindly help me on this. When I try to trace for User space it is creating an empty folder with no traces on ARM target, I tried with all the sample trace programs in LTTng-UST/tests folder. I am using NFS to load binaries and for tracing will that causes problem. Please also find the debug logs attached, Please also let me know if you want some more information. ************************************** fcntl64(12, F_SETFD, FD_CLOEXEC) = 0 fcntl64(13, F_SETFD, FD_CLOEXEC) = 0 pipe([14, 15]) = 0 fcntl64(14, F_SETFD, FD_CLOEXEC) = 0 fcntl64(15, F_SETFD, FD_CLOEXEC) = 0 getrlimit(RLIMIT_NOFILE, {rlim_cur=65535, rlim_max=65535}) = 0 write(2, "DEBUG1: poll set max size set to"..., 92DEBUG1: poll set max size set to 65535 [in compat_poll_set_max_size() at compat-poll.c:196] ) = 92 mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4024b000 mprotect(0x4024b000, 4096, PROT_NONE) = 0 clone(child_stack=0x40a49ef8, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x40a4a3e8, tls=0x40a4a840, child_tidptr=0x40a4a3e8) = 921 mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40a4b000 mprotect(0x40a4b000, 4096, PROT_NONE) = 0 clone(child_stack=0x41249ef8, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x4124a3e8, tls=0x4124a840, child_tidptr=0x4124a3e8) = 922 mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4124b000 mprotect(0x4124b000, 4096, PROT_NONE) = 0 clone(child_stack=0x41a49ef8, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x41a4a3e8, tls=0x41a4a840, child_tidptr=0x41a4a3e8) = 923 mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x41a4b000 mprotect(0x41a4b000, 4096, PROT_NONE) = 0 clone(DEBUG1: [thread] Dispatch UST command started [in thread_dispatch_ust_registration() at main.c:1324] DEBUG1: Futex n to 1 prepare done [in futex_nto1_prepare() at futex.c:73] DEBUG1: Woken up but nothing in the UST command queue [in thread_dispatch_ust_registration() at main.c:1334] DEBUG1: [thread] Manage application registration started [in thread_registration_apps() at main.c:1392] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 11 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: Notifying applications of session daemon state: 1 [in notify_ust_apps() at main.c:687] DEBUG1: Got the wait shm fd 16 [in get_wait_shm() at shm.c:117] DEBUG1: Futex wait update active 1 [in futex_wait_update() at futex.c:62] DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1423] DEBUG1: [thread] Manage application started [in thread_manage_apps() at main.c:1179] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 14 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: Apps thread polling on 2 fds [in thread_manage_apps() at main.c:1200] DEBUG1: [thread] Manage client started [in thread_manage_clients() at main.c:3794] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 10 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3826] child_stack=0x42249ef8, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x4224a3e8, tls=0x4224a840, child_tidptr=0x4224a3e8) = 924 mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4224b000 mprotect(0x4224b000, 4096, PROT_NONE) = 0 clone(child_stack=0x42a49ef8, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x42a4a3e8, tls=0x42a4a840, child_tidptr=0x42a4a3e8) = 925 futex(0x42a4a3e8, FUTEX_WAIT, 925, NULLDEBUG1: Thread manage kernel started [in thread_manage_kernel() at main.c:876] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 12 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: Updating kernel poll set [in update_kernel_poll() at main.c:748] DEBUG1: Thread kernel polling on 2 fds [in thread_manage_kernel() at main.c:905] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3863] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3898] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3902] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:534] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3826] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3863] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3898] DEBUG1: Processing client command 8 [in process_client_msg() at main.c:3309] DEBUG2: Trying to find session by name sesCCC [in session_find_by_name() at session.c:126] DEBUG3: mkdir() recursive /home/root/lttng-traces/sesCCC-20110327-045731 with mode 504 for uid 0 and gid 0 [in run_as_mkdir_recursive() at runas.c:338] DEBUG1: Using run_as_clone [in run_as() at runas.c:322] DEBUG1: Tracing session sesCCC created in /home/root/lttng-traces/sesCCC-20110327-045731 with ID 1 by UID 0 GID 0 [in session_create() at session.c:234] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3936] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:534] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3826] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3863] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3898] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3902] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:534] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3826] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3863] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3898] DEBUG1: Processing client command 6 [in process_client_msg() at main.c:3309] DEBUG1: Getting session sesCCC by name [in process_client_msg() at main.c:3364] DEBUG2: Trying to find session by name sesCCC [in session_find_by_name() at session.c:126] DEBUG1: Creating UST session [in create_ust_session() at main.c:1965] DEBUG3: Created hashtable size 4 at 0x4fa08 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x4fb28 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x4fc48 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG2: UST trace session create successful [in trace_ust_create_session() at trace-ust.c:119] DEBUG3: mkdir() recursive /home/root/lttng-traces/sesCCC-20110327-045731/ust with mode 504 for uid 0 and gid 0 [in run_as_mkdir_recursive() at runas.c:338] DEBUG1: Using run_as_clone [in run_as() at runas.c:322] DEBUG1: Spawning consumerd [in spawn_consumerd() at main.c:1659] DEBUG1: Using 32-bit UST consumer at: /root/armfilesystem/lib/lttng/libexec/lttng-consumerd [in spawn_consumerd() at main.c:1777] DEBUG2: Consumer pid 934 [in start_consumerd() at main.c:1830] DEBUG2: Spawning consumer control thread [in start_consumerd() at main.c:1833] DEBUG1: [thread] Manage consumer started [in thread_manage_consumer() at main.c:982] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 9 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG2: Receiving code from consumer err_sock [in thread_manage_consumer() at main.c:1043] DEBUG1: consumer command socket ready [in thread_manage_consumer() at main.c:1062] DEBUG1: fd 17 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG2: Trace UST channel channel0 not found by name [in trace_ust_find_channel_by_name() at trace-ust.c:52] DEBUG3: Created hashtable size 4 at 0x51310 of type 0 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x51430 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Trace UST channel channel0 created [in trace_ust_create_channel() at trace-ust.c:181] DEBUG2: Channel channel0 being created in UST global domain [in channel_ust_create() at channel.c:267] DEBUG2: UST app adding channel channel0 to global domain for session id 1 [in ust_app_create_channel_glb() at ust-app.c:1792] DEBUG2: Channel channel0 created successfully [in channel_ust_create() at channel.c:292] DEBUG2: Trace UST channel channel0 found by name [in trace_ust_find_channel_by_name() at trace-ust.c:47] DEBUG2: Trace UST event NOT found by name * [in trace_ust_find_event_by_name() at trace-ust.c:79] DEBUG3: Created hashtable size 4 at 0x51550 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: Trace UST event *, loglevel (0,-1) created [in trace_ust_create_event() at trace-ust.c:260] DEBUG1: UST app creating event * for all apps for session id 1 [in ust_app_create_event_glb() at ust-app.c:1916] DEBUG1: Event UST * created in channel channel0 [in event_ust_enable_tracepoint() at event.c:446] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3936] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:534] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3826] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3863] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3898] DEBUG1: Nothing recv() from client... continuing [in thread_manage_clients() at main.c:3902] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:534] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3826] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3863] DEBUG1: Receiving data from client ... [in thread_manage_clients() at main.c:3898] DEBUG1: Processing client command 16 [in process_client_msg() at main.c:3309] DEBUG1: Getting session sesCCC by name [in process_client_msg() at main.c:3364] DEBUG2: Trying to find session by name sesCCC [in session_find_by_name() at session.c:126] DEBUG1: Starting all UST traces [in ust_app_start_trace_all() at ust-app.c:2207] DEBUG1: Sending response (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3936] DEBUG1: Clean command context structure [in clean_command_ctx() at main.c:534] DEBUG1: Accepting client command ... [in thread_manage_clients() at main.c:3826] ************************************************ Thanks in Advance, Pavan -----Original Message----- From: David Goulet [mailto:dgoulet at efficios.com] Sent: Tuesday, June 12, 2012 9:05 PM To: Pavan Anumula Cc: lttng-dev at lists.lttng.org Subject: Re: [lttng-dev] Lttng start failed -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Pavan, The output below is normal and should be working fine. However, the problem is that you have _NO_ debug message after the "lttng start". That command should at least produce 10+ lines of messages after this last line: "DEBUG1: Accepting application registration [in thread_registration_apps() at main.c:1423]" Can you enlight me and tell me if by doing "lttng create newsessiom" and "lttng enable-event -a -k", you have output messages coming out of the session daemon in -vvv mode ? If NOT, you are talking to another daemon! A quick "ps faux | grep lttng-sessiond" could help clear that question. If only one daemon is running and stuck at the above debug message, the next step is to tell me on what the daemon is waiting on using either "strace -p" or "gdb" also. There is 6 threads so having the strace -p output for all of them would help me greatly. Thanks! David On 12/06/12 10:05 AM, Mathieu Desnoyers wrote: > * Pavan Anumula (pavan.anumula at sasken.com) wrote: >> Hi Mathieu, Still I am unable to start tracing, And I am facing the >> same start errors. Please kindly help me on this. This time I loaded >> the modules by modprobe( not with insmod as I did before), Then I >> run the command " arm-none-linux-gnueabi-lttng-sessiond -vvv " >> (before arm-none-linux-gnueabi-lttng-sessiond -d), > > When using "arm-none-linux-gnueabi-lttng-sessiond -vvv", you should > _not_ run "arm-none-linux-gnueabi-lttng-sessiond -d". > > >> The below is the Output, And It got hanged overthere. > > This is normal: the sessiond is waiting for applications to connect. > It does not hang, it just waits. > > David, can you follow up on this issue ? It seems to be sessiond-related. > > Thanks, > > Mathieu > > >> Later when I started lttng start I am acing the same erros below: >> root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng start LTTng: >> Failure to write metadata to buffers (timeout) Error: Starting kernel >> trace failed >> >> ********************************************************************* >> *************************** >> >> DEBUG3: Creating LTTng run directory: /var/run/lttng [in create_lttng_rundir() at main.c:4315] >> DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in >> main() at main.c:4543] DEBUG2: Kernel consumer cmd path: >> /var/run/lttng/kconsumerd/command [in main() at main.c:4545] DEBUG1: >> Client socket path /var/run/lttng/client-lttng-sessiond [in main() at >> main.c:4592] DEBUG1: Application socket path >> /var/run/lttng/apps-lttng-sessiond [in main() at main.c:4593] DEBUG1: >> LTTng run directory path: /var/run/lttng [in main() at main.c:4594] >> DEBUG2: UST consumer 32 bits err path: >> /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] DEBUG2: >> UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command >> [in >> main() at main.c:4605] DEBUG2: UST consumer 64 bits err path: >> /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] DEBUG2: >> UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command >> [in >> main() at main.c:4616] DEBUG3: Created hashtable size 4 at 0x4a080 of >> type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created >> hashtable size 4 at 0x4a168 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: >> Creating consumer directory: /var/run/lttng/kconsumerd [in >> set_consumer_sockets() at main.c:4357] DEBUG1: Modprobe successfully >> lttng-tracer [in modprobe_lttng_control() at modprobe.c:163] DEBUG2: >> Kernel tracer version validated (major version 2) [in >> kernel_validate_version() at kernel.c:675] DEBUG1: Modprobe >> successfully lttng-ftrace [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: >> Modprobe successfully lttng-kprobes [in modprobe_lttng_data() at >> modprobe.c:199] DEBUG1: Modprobe successfully lttng-kretprobes [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >> successfully lttng-lib-ring-buffer [in modprobe_lttng_data() at >> modprobe.c:199] >> DEBUG1: Modprobe successfully lttng-ring-buffer-client-discard [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >> successfully lttng-ring-buffer-client-overwrite [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >> successfully lttng-ring-buffer-metadata-client [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >> successfully lttng-ring-buffer-client-mmap-discard [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >> successfully lttng-ring-buffer-client-mmap-overwrite [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >> successfully lttng-ring-buffer-metadata-mmap-client [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >> successfully lttng-probe-lttng [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >> successfully lttng-types [in modprobe_lttng_data() at modprobe.c:199] >> DEBUG1: Modprobe successfully lttng-probe-block [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >> successfully lttng-probe-irq [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >> successfully lttng-probe-kvm [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: >> Modprobe successfully lttng-probe-sched [in modprobe_lttng_data() at >> modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-signal [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >> successfully lttng-probe-statedump [in modprobe_lttng_data() at >> modprobe.c:199] >> DEBUG1: Modprobe successfully lttng-probe-timer [in >> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Kernel tracer fd 6 >> [in init_kernel_tracer() at main.c:1887] DEBUG2: Creating consumer directory: >> /var/run/lttng/ustconsumerd64 [in set_consumer_sockets() at >> main.c:4357] >> DEBUG2: Creating consumer directory: /var/run/lttng/ustconsumerd32 >> [in >> set_consumer_sockets() at main.c:4357] DEBUG1: Signal handler set for >> SIGTERM, SIGPIPE and SIGINT [in set_signal_handler() at main.c:4449] >> DEBUG1: All permissions are set [in set_permissions() at main.c:4250] >> DEBUG1: poll set max size set to 65535 [in compat_poll_set_max_size() >> at compat-poll.c:196] DEBUG1: [thread] Manage application started [in >> thread_manage_apps() at main.c:1179] DEBUG1: fd 3 of 1 added to >> pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 13 of 2 >> added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: >> Apps thread polling on 2 fds [in thread_manage_apps() at main.c:1200] >> DEBUG1: Thread manage kernel started [in thread_manage_kernel() at main.c:876] DEBUG1: >> fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] >> DEBUG1: fd 11 of 2 added to pollfd [in compat_poll_add() at >> compat-poll.c:91] DEBUG1: Updating kernel poll set [in >> update_kernel_poll() at main.c:748] DEBUG1: Thread kernel polling on >> 2 fds [in thread_manage_kernel() at main.c:905] DEBUG1: [thread] >> Manage application registration started [in >> thread_registration_apps() at main.c:1392] DEBUG1: fd 3 of 1 added to >> pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 10 of 2 >> added to pollfd [in >> compat_poll_add() at compat-poll.c:91] DEBUG1: Notifying applications >> of session daemon state: 1 [in notify_ust_apps() at main.c:687] DEBUG1: >> [thread] Dispatch UST command started [in >> thread_dispatch_ust_registration() at main.c:1324] DEBUG1: Futex n to >> 1 prepare done [in futex_nto1_prepare() at futex.c:73] DEBUG1: Woken >> up but nothing in the UST command queue [in >> thread_dispatch_ust_registration() >> at main.c:1334] DEBUG1: [thread] Manage client started [in >> thread_manage_clients() at main.c:3794] DEBUG1: fd 3 of 1 added to >> pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 9 of 2 >> added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: >> Accepting client command ... [in thread_manage_clients() at main.c:3826] DEBUG1: >> Got the wait shm fd 15 [in get_wait_shm() at shm.c:117] DEBUG1: Futex >> wait update active 1 [in futex_wait_update() at futex.c:62] DEBUG1: >> Accepting application registration [in thread_registration_apps() at >> main.c:1423] >> ********************************************************************* >> *********************************** >> >> Am I missing anything?? >> >> Thanks in Advance, Pavan >> >> >> ________________________________________ From: Mathieu Desnoyers >> [mathieu.desnoyers at efficios.com] Sent: Monday, June 11, 2012 7:24 PM To: >> Pavan Anumula Cc: lttng-dev at lists.lttng.org Subject: Re: [lttng-dev] >> Lttng start failed >> >> * Pavan Anumula (pavan.anumula at sasken.com) wrote: >>> Hi Mathieu, >>> >>> Please find the info below as per your comments >>> >>> >>> ########## Output of command "arm-none-linux-gnueabi-lttng-sessiond >>> -vvv " , After loading the modules ####### >>> >>> DEBUG3: Creating LTTng run directory: /var/run/lttng [in >>> create_lttng_rundir() at main.c:4315] DEBUG2: Kernel consumer err path: >>> /var/run/lttng/kconsumerd/error [in main() at main.c:4543] DEBUG2: >>> Kernel consumer cmd path: /var/run/lttng/kconsumerd/command [in >>> main() at main.c:4545] DEBUG1: Client socket path >>> /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] DEBUG1: >>> Application socket path /var/run/lttng/apps-lttng-sessiond [in >>> main() at main.c:4593] DEBUG1: LTTng run directory path: >>> /var/run/lttng [in >>> main() at main.c:4594] DEBUG2: UST consumer 32 bits err path: >>> /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] DEBUG2: >>> UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command >>> [in main() at main.c:4605] DEBUG2: UST consumer 64 bits err path: >>> /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] DEBUG2: >>> UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command >>> [in main() at main.c:4616] DEBUG3: Created hashtable size 4 at >>> 0x4a080 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: >>> Created hashtable size 4 at 0x4a168 of type 1 [in lttng_ht_new() at >>> hashtable.c:96] DEBUG2: Creating consumer directory: >>> /var/run/lttng/kconsumerd [in set_consumer_sockets() at main.c:4357] >>> FATAL: Module lttng_tracer not found. Error: Unable to load module >>> lttng-tracer >> >> Hrm. You want to do kernel tracing, but modprobe cannot find the >> lttng kernel tracer modules. You might want to run depmod -a or >> something like that on your target, and ensure that modprobe works properly. >> >> Thanks, >> >> Mathieu >> >>> DEBUG2: Kernel tracer version validated (major version 2) [in >>> kernel_validate_version() at kernel.c:675] DEBUG1: Modprobe >>> successfully lttng-ftrace [in modprobe_lttng_data() at >>> modprobe.c:199] >>> DEBUG1: Modprobe successfully lttng-kprobes [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>> successfully lttng-kretprobes [in >>> modprobe_lttng_data() at modprobe.c:199] FATAL: Module >>> lttng_lib_ring_buffer not found. Error: Unable to load module >>> lttng-lib-ring-buffer FATAL: Module lttng_ring_buffer_client_discard >>> not found. Error: Unable to load module >>> lttng-ring-buffer-client-discard FATAL: Module >>> lttng_ring_buffer_client_overwrite not found. Error: Unable to load >>> module lttng-ring-buffer-client-overwrite FATAL: Module >>> lttng_ring_buffer_metadata_client not found. Error: Unable to load >>> module lttng-ring-buffer-metadata-client FATAL: Module >>> lttng_ring_buffer_client_mmap_discard not found. Error: Unable to >>> load module lttng-ring-buffer-client-mmap-discard FATAL: Module >>> lttng_ring_buffer_client_mmap_overwrite not found. Error: Unable to >>> load module lttng-ring-buffer-client-mmap-overwrite FATAL: Module >>> lttng_ring_buffer_metadata_mmap_client not found. Error: Unable to >>> load module lttng-ring-buffer-metadata-mmap-client FATAL: Module >>> lttng_probe_lttng not found. Error: Unable to load module >>> lttng-probe-lttng DEBUG1: Modprobe successfully lttng-types [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>> successfully lttng-probe-block [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: >>> Modprobe successfully lttng-probe-irq [in modprobe_lttng_data() at >>> modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-kvm [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>> successfully lttng-probe-sched [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: >>> Modprobe successfully lttng-probe-signal [in modprobe_lttng_data() >>> at modprobe.c:199] DEBUG1: Modprobe successfully >>> lttng-probe-statedump [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>> successfully lttng-probe-timer [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: >>> Kernel tracer fd 6 [in init_kernel_tracer() at main.c:1887] DEBUG2: >>> Creating consumer directory: /var/run/lttng/ustconsumerd64 [in >>> set_consumer_sockets() at main.c:4357] DEBUG2: Creating consumer >>> directory: /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() >>> at main.c:4357] DEBUG1: Signal handler set for SIGTERM, SIGPIPE and >>> SIGINT [in set_signal_handler() at main.c:4449] DEBUG1: All >>> permissions are set [in set_permissions() at main.c:4250] DEBUG1: >>> poll set max size set to 65535 [in compat_poll_set_max_size() at compat-poll.c:196] DEBUG1: >>> [thread] Manage client started [in thread_manage_clients() at >>> main.c:3794] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() >>> at compat-poll.c:91] DEBUG1: fd 9 of 2 added to pollfd [in >>> compat_poll_add() at compat-poll.c:91] DEBUG1: Accepting client >>> command ... [in thread_manage_clients() at main.c:3826] DEBUG1: >>> [thread] Dispatch UST command started [in >>> thread_dispatch_ust_registration() at main.c:1324] DEBUG1: Futex n >>> to 1 prepare done [in futex_nto1_prepare() at futex.c:73] DEBUG1: >>> Woken up but nothing in the UST command queue [in >>> thread_dispatch_ust_registration() at main.c:1334] DEBUG1: Thread manage kernel started [in thread_manage_kernel() at main.c:876] DEBUG1: >>> fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] >>> DEBUG1: fd 11 of 2 added to pollfd [in compat_poll_add() at >>> compat-poll.c:91] DEBUG1: Updating kernel poll set [in >>> update_kernel_poll() at main.c:748] DEBUG1: Thread kernel polling on >>> 2 fds [in thread_manage_kernel() at main.c:905] DEBUG1: [thread] >>> Manage application started [in thread_manage_apps() at main.c:1179] >>> DEBUG1: fd >>> 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] >>> DEBUG1: fd 13 of 2 added to pollfd [in compat_poll_add() at >>> compat-poll.c:91] DEBUG1: Apps thread polling on 2 fds [in >>> thread_manage_apps() at main.c:1200] DEBUG1: [thread] Manage >>> application registration started [in thread_registration_apps() at >>> main.c:1392] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() >>> at compat-poll.c:91] DEBUG1: fd 10 of 2 added to pollfd [in >>> compat_poll_add() at compat-poll.c:91] DEBUG1: Notifying >>> applications of session daemon state: 1 [in notify_ust_apps() at main.c:687] DEBUG1: >>> Got the wait shm fd 15 [in get_wait_shm() at shm.c:117] DEBUG1: >>> Futex wait update active 1 [in futex_wait_update() at futex.c:62] DEBUG1: >>> Accepting application registration [in thread_registration_apps() at >>> main.c:1423] >>> >>> >>> >>> #### Output of command "arm-none-linux-gnueabi-lttng-sessiond -vvv " >>> before loading the modules ############# >>> >>> DEBUG3: Creating LTTng run directory: /var/run/lttng [in >>> create_lttng_rundir() at main.c:4315] DEBUG2: Kernel consumer err path: >>> /var/run/lttng/kconsumerd/error [in main() at main.c:4543] DEBUG2: >>> Kernel consumer cmd path: /var/run/lttng/kconsumerd/command [in >>> main() at main.c:4545] DEBUG1: Client socket path >>> /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] DEBUG1: >>> Application socket path /var/run/lttng/apps-lttng-sessiond [in >>> main() at main.c:4593] DEBUG1: LTTng run directory path: >>> /var/run/lttng [in >>> main() at main.c:4594] DEBUG2: UST consumer 32 bits err path: >>> /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] DEBUG2: >>> UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command >>> [in main() at main.c:4605] DEBUG2: UST consumer 64 bits err path: >>> /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] DEBUG2: >>> UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command >>> [in main() at main.c:4616] DEBUG3: Created hashtable size 4 at >>> 0x4a080 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: >>> Created hashtable size 4 at 0x4a168 of type 1 [in lttng_ht_new() at >>> hashtable.c:96] DEBUG2: Creating consumer directory: >>> /var/run/lttng/kconsumerd [in set_consumer_sockets() at main.c:4357] >>> FATAL: Module lttng_tracer not found. Error: Unable to load module >>> lttng-tracer DEBUG1: Failed to open /proc/lttng [in >>> init_kernel_tracer() at main.c:1871] Error: Unable to remove module >>> lttng-tracer Warning: No kernel tracer available DEBUG2: Creating >>> consumer directory: /var/run/lttng/ustconsumerd64 [in >>> set_consumer_sockets() at main.c:4357] DEBUG2: Creating consumer >>> directory: /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() >>> at main.c:4357] DEBUG1: Signal handler set for SIGTERM, SIGPIPE and >>> SIGINT [in set_signal_handler() at main.c:4449] DEBUG1: All >>> permissions are set [in set_permissions() at main.c:4250] DEBUG1: >>> poll set max size set to 65535 [in compat_poll_set_max_size() at compat-poll.c:196] DEBUG1: >>> [thread] Manage application started [in thread_manage_apps() at >>> main.c:1179] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() >>> at compat-poll.c:91] DEBUG1: fd 12 of 2 added to pollfd [in >>> compat_poll_add() at compat-poll.c:91] DEBUG1: Apps thread polling >>> on 2 fds [in thread_manage_apps() at main.c:1200] DEBUG1: Thread >>> manage kernel started [in thread_manage_kernel() at main.c:876] >>> DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: >>> fd 10 of 2 added to pollfd [in compat_poll_add() at >>> compat-poll.c:91] >>> DEBUG1: Updating kernel poll set [in update_kernel_poll() at >>> main.c:748] DEBUG1: Thread kernel polling on 2 fds [in >>> thread_manage_kernel() at main.c:905] DEBUG1: [thread] Manage >>> application registration started [in thread_registration_apps() at >>> main.c:1392] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() >>> at compat-poll.c:91] DEBUG1: fd 9 of 2 added to pollfd [in >>> compat_poll_add() at compat-poll.c:91] DEBUG1: Notifying >>> applications of session daemon state: 1 [in notify_ust_apps() at main.c:687] DEBUG1: >>> [thread] Dispatch UST command started [in >>> thread_dispatch_ust_registration() at main.c:1324] DEBUG1: Futex n >>> to 1 prepare done [in futex_nto1_prepare() at futex.c:73] DEBUG1: >>> Woken up but nothing in the UST command queue [in >>> thread_dispatch_ust_registration() at main.c:1334] DEBUG1: [thread] >>> Manage client started [in thread_manage_clients() at main.c:3794] >>> DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at >>> compat-poll.c:91] DEBUG1: fd 8 of 2 added to pollfd [in >>> compat_poll_add() at compat-poll.c:91] DEBUG1: Accepting client >>> command ... [in thread_manage_clients() at main.c:3826] DEBUG1: Got >>> the wait shm fd 14 [in get_wait_shm() at shm.c:117] DEBUG1: Futex >>> wait update active 1 [in futex_wait_update() at futex.c:62] DEBUG1: >>> Accepting application registration [in thread_registration_apps() at >>> main.c:1423] >>> >>> >>> Regards, Pavan >>> >>> -----Original Message----- From: Mathieu Desnoyers >>> [mailto:mathieu.desnoyers at efficios.com] Sent: Saturday, June 09, >>> 2012 >>> 6:03 PM To: Pavan Anumula Cc: lttng-dev at lists.lttng.org Subject: Re: >>> [lttng-dev] Lttng start failed >>> >>> * Pavan Anumula (pavan.anumula at sasken.com) wrote: >>>> Hi Mathue, >>>> >>>> Thanks for the quick reply, >>>> >>>> After inserting lttng modules , I had given the command >>>> "arm-none-linux-gnueabi-lttng-sessiond -vvv" as you said, Below is >>>> the output where there are error messages. Please help me in >>>> resolving the issue, SO that I can catch kernel and user traces. >>>> >>>> >>>> root at arago:/usr/lttng/modules# >>>> arm-none-linux-gnueabi-lttng-sessiond >>>> -d root at arago:/usr/lttng/modules# >>>> arm-none-linux-gnueabi-lttng-sessiond -vvv DEBUG3: Creating LTTng >>>> run >>>> directory: /var/run/lttng [in create_lttng_rundir() at main.c:4315] >>>> DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error >>>> [in main() at main.c:4543] DEBUG2: Kernel consumer cmd path: >>>> /var/run/lttng/kconsumerd/command [in main() at main.c:4545] DEBUG1: >>>> Client socket path /var/run/lttng/client-lttng-sessiond [in main() >>>> at main.c:4592] DEBUG1: Application socket path >>>> /var/run/lttng/apps-lttng-sessiond [in main() at main.c:4593] DEBUG1: >>>> LTTng run directory path: /var/run/lttng [in main() at main.c:4594] >>>> DEBUG2: UST consumer 32 bits err path: >>>> /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] >>>> DEBUG2: UST consumer 32 bits cmd path: >>>> /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] >>>> DEBUG2: UST consumer 64 bits err path: >>>> /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] >>>> DEBUG2: UST consumer 64 bits cmd path: >>>> /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] >>>> Error: Already running daemon. >>> >>> Please kill the lttng-sessiond that is already started (the one with >>> -d). Instead of that one, run the sessiond with: >>> >>> lttng-sessiond -vvv >>> >>> (don't run lttng-sessiond -d before) >>> >>> Thanks, >>> >>> Mathieu >>> >>>> >>>> >>>> >>>> >>>> Thanks in advance, Pavan >>>> >>>> -----Original Message----- From: Mathieu Desnoyers >>>> [mailto:mathieu.desnoyers at efficios.com] Sent: Friday, June 08, 2012 >>>> 11:12 PM To: Pavan Anumula Cc: lttng-dev at lists.lttng.org Subject: Re: >>>> [lttng-dev] Lttng start failed >>>> >>>> * Pavan Anumula (pavan.anumula at sasken.com) wrote: >>>>> Hi , >>>>> >>>>> I am new to LTTng usage, I am trying to use LTTng 2.0.1 and >>>>> LTTng-modules-2.0.2 for ARM(omapL138), with Linux kernel 2.6.33 >>>>> wit RT-29 patch on it. >>>>> >>>>> After enabling all the kernel events I am facing the below errors. >>>>> I loaded all the modules manually by insmod. >>>>> >>>>> >>>>> >>>>> root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng create >>>>> newsessiom Session newsessiom created. Traces will be written in >>>>> /home/root/lttng-traces/newsessiom-20110325-154922 >>>>> >>>>> root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng >>>>> enable-event -a --kernel All kernel events are enabled in channel >>>>> channel0 >>>>> >>>>> root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng start >>>>> LTTng: Failure to write metadata to buffers (timeout) Error: >>>>> Starting kernel trace failed >>>>> >>>>> Please kindly help me on this. >>>> >>>> I think you should look into the lttng-sessiond --help : >>>> >>>> --consumerd32-path PATH Specify path for the 32-bit UST consumer >>>> daemon binary --consumerd32-libdir PATH Specify path for the 32-bit >>>> UST consumer daemon libraries --consumerd64-path PATH Specify >>>> path for the 64-bit UST consumer daemon binary --consumerd64-libdir >>>> PATH Specify path for the 64-bit UST consumer daemon libraries >>>> >>>> options. My guess is that lttng-sessiond is not able to find the >>>> consumerd binary files, maybe due to a rename or because they have >>>> been moved after install. >>>> >>>> One more thing that might help is to launch the lttng-sessiond with >>>> "-vvv" : it will provide verbose output and let us know where >>>> things fall apart. >>>> >>>> Thanks, >>>> >>>> Mathieu >>>> >>>> >>>>> >>>>> >>>>> Thanks in advance, Pavan >>>>> >>>>> ________________________________ SASKEN BUSINESS DISCLAIMER: This >>>>> message may contain confidential, proprietary or legally >>>>> privileged information. In case you are not the original intended >>>>> Recipient of the message, you must not, directly or indirectly, >>>>> use, disclose, distribute, print, or copy any part of this message >>>>> and you are requested to delete it and inform the sender. Any >>>>> views expressed in this message are those of the individual sender >>>>> unless otherwise stated. Nothing contained in this message shall >>>>> be construed as an offer or acceptance of any offer by Sasken >>>>> Communication Technologies Limited ("Sasken") unless sent with >>>>> that express intent and with due authority of Sasken. Sasken has >>>>> taken enough precautions to prevent the spread of viruses. However >>>>> the company accepts no liability for any damage caused by any >>>>> virus transmitted by this email. Read Disclaimer at >>>>> http://www.sasken.com/extras/mail_disclaimer.html >>>> >>>>> _______________________________________________ lttng-dev mailing >>>>> list lttng-dev at lists.lttng.org >>>>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev >>>> >>>> >>>> -- Mathieu Desnoyers Operating System Efficiency R&D Consultant >>>> EfficiOS Inc. http://www.efficios.com >>> >>> -- Mathieu Desnoyers Operating System Efficiency R&D Consultant >>> EfficiOS Inc. http://www.efficios.com >> >> -- Mathieu Desnoyers Operating System Efficiency R&D Consultant >> EfficiOS Inc. http://www.efficios.com > -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBAgAGBQJP12GMAAoJEELoaioR9I02+q8IAKw/ot4BmP6xibXM/VnNaRdP 60S1KEynFPxDmO9JS3u3x+r4UiDaGWrP4+rEfgAeUeMQcOj3ItHZLyNhp+F8A8+j Dj0q9YFNBLsANzHzAk9VsVlL/myYaeW7ysNGl1yxhL9Vc/Px3cmxaof79smxMGcQ dnMKf6Gk+eNL4IPrXQIxF0cvVK85PI2xaDA1SiiKQaULUiqUBebn2gtGr1l8oFTw FT1xryh4wRw4FUmqngw54SYPiYpZm1y/VUzTWB3Zs9hYPcvEum22wrhAaWNkoUkk ixBjE+HHIVRBzZatDymE35tI4+Ci6z9J3uRmJVC97x2qQz2F4nl5Yw3qRmm1PVM= =3VyT -----END PGP SIGNATURE----- From dgoulet at efficios.com Fri Jun 15 10:57:31 2012 From: dgoulet at efficios.com (David Goulet) Date: Fri, 15 Jun 2012 10:57:31 -0400 Subject: [lttng-dev] Empty traces for UST In-Reply-To: <47D610AD9C485E458630BA38C324D3B60113FB00FBB0@EXGMBX01.sasken.com> References: <47D610AD9C485E458630BA38C324D3B60113FB00FBB0@EXGMBX01.sasken.com> Message-ID: <4FDB4D5B.6050701@efficios.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Pavan, Glad for the kernel! :) For UST, I'll recommend you start a session daemon like so "lttng-sessiond - -vvv". After that, use the lttng-ust/tests/hello program and execute it. You should see on the debug output of the session daemon that the application "hello" has registered and is ready for tracing. Your output below does NOT show those debug messages so it would be a good start to see if there is a problem at that point. Also, make sure that the liblttng-ust is installed upon compiling lttng-tools so that the user space tracer support is enabled in the tools. Thanks! David On 15/06/12 10:50 AM, Pavan Anumula wrote: > Hi David, > > Finally I was able to get kernel traces by referring to one of your other > posts , Actually lttng-consumerd binary was actually not present at the > needed path, Thanks for your help. > > And sorry for bugging you with mails :) . > > > But this time I am facing issues at UST tracing . Please kindly help me on > this. > > When I try to trace for User space it is creating an empty folder with no > traces on ARM target, I tried with all the sample trace programs in > LTTng-UST/tests folder. > > I am using NFS to load binaries and for tracing will that causes problem. > Please also find the debug logs attached, > > Please also let me know if you want some more information. > > > ************************************** fcntl64(12, F_SETFD, FD_CLOEXEC) > = 0 fcntl64(13, F_SETFD, FD_CLOEXEC) = 0 pipe([14, 15]) > = 0 fcntl64(14, F_SETFD, FD_CLOEXEC) = 0 fcntl64(15, F_SETFD, > FD_CLOEXEC) = 0 getrlimit(RLIMIT_NOFILE, {rlim_cur=65535, > rlim_max=65535}) = 0 write(2, "DEBUG1: poll set max size set to"..., > 92DEBUG1: poll set max size set to 65535 [in compat_poll_set_max_size() at > compat-poll.c:196] ) = 92 mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4024b000 mprotect(0x4024b000, 4096, > PROT_NONE) = 0 clone(child_stack=0x40a49ef8, > flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, > parent_tidptr=0x40a4a3e8, tls=0x40a4a840, child_tidptr=0x40a4a3e8) = 921 > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) = 0x40a4b000 mprotect(0x40a4b000, 4096, PROT_NONE) = 0 > clone(child_stack=0x41249ef8, > flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, > parent_tidptr=0x4124a3e8, tls=0x4124a840, child_tidptr=0x4124a3e8) = 922 > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) = 0x4124b000 mprotect(0x4124b000, 4096, PROT_NONE) = 0 > clone(child_stack=0x41a49ef8, > flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, > parent_tidptr=0x41a4a3e8, tls=0x41a4a840, child_tidptr=0x41a4a3e8) = 923 > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) = 0x41a4b000 mprotect(0x41a4b000, 4096, PROT_NONE) = 0 clone(DEBUG1: > [thread] Dispatch UST command started [in > thread_dispatch_ust_registration() at main.c:1324] DEBUG1: Futex n to 1 > prepare done [in futex_nto1_prepare() at futex.c:73] DEBUG1: Woken up but > nothing in the UST command queue [in thread_dispatch_ust_registration() at > main.c:1334] DEBUG1: [thread] Manage application registration started [in > thread_registration_apps() at main.c:1392] DEBUG1: fd 3 of 1 added to > pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 11 of 2 added > to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: Notifying > applications of session daemon state: 1 [in notify_ust_apps() at > main.c:687] DEBUG1: Got the wait shm fd 16 [in get_wait_shm() at > shm.c:117] DEBUG1: Futex wait update active 1 [in futex_wait_update() at > futex.c:62] DEBUG1: Accepting application registration [in > thread_registration_apps() at main.c:1423] DEBUG1: [thread] Manage > application started [in thread_manage_apps() at main.c:1179] DEBUG1: fd 3 > of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd > 14 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: > Apps thread polling on 2 fds [in thread_manage_apps() at main.c:1200] > DEBUG1: [thread] Manage client started [in thread_manage_clients() at > main.c:3794] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at > compat-poll.c:91] DEBUG1: fd 10 of 2 added to pollfd [in compat_poll_add() > at compat-poll.c:91] DEBUG1: Accepting client command ... [in > thread_manage_clients() at main.c:3826] child_stack=0x42249ef8, > flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, > parent_tidptr=0x4224a3e8, tls=0x4224a840, child_tidptr=0x4224a3e8) = 924 > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) = 0x4224b000 mprotect(0x4224b000, 4096, PROT_NONE) = 0 > clone(child_stack=0x42a49ef8, > flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, > parent_tidptr=0x42a4a3e8, tls=0x42a4a840, child_tidptr=0x42a4a3e8) = 925 > futex(0x42a4a3e8, FUTEX_WAIT, 925, NULLDEBUG1: Thread manage kernel started > [in thread_manage_kernel() at main.c:876] DEBUG1: fd 3 of 1 added to pollfd > [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 12 of 2 added to > pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: Updating kernel > poll set [in update_kernel_poll() at main.c:748] DEBUG1: Thread kernel > polling on 2 fds [in thread_manage_kernel() at main.c:905] DEBUG1: Wait for > client response [in thread_manage_clients() at main.c:3863] DEBUG1: > Receiving data from client ... [in thread_manage_clients() at main.c:3898] > DEBUG1: Nothing recv() from client... continuing [in > thread_manage_clients() at main.c:3902] DEBUG1: Clean command context > structure [in clean_command_ctx() at main.c:534] DEBUG1: Accepting client > command ... [in thread_manage_clients() at main.c:3826] DEBUG1: Wait for > client response [in thread_manage_clients() at main.c:3863] DEBUG1: > Receiving data from client ... [in thread_manage_clients() at main.c:3898] > DEBUG1: Processing client command 8 [in process_client_msg() at > main.c:3309] DEBUG2: Trying to find session by name sesCCC [in > session_find_by_name() at session.c:126] DEBUG3: mkdir() recursive > /home/root/lttng-traces/sesCCC-20110327-045731 with mode 504 for uid 0 and > gid 0 [in run_as_mkdir_recursive() at runas.c:338] DEBUG1: Using > run_as_clone [in run_as() at runas.c:322] DEBUG1: Tracing session sesCCC > created in /home/root/lttng-traces/sesCCC-20110327-045731 with ID 1 by UID > 0 GID 0 [in session_create() at session.c:234] DEBUG1: Sending response > (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3936] > DEBUG1: Clean command context structure [in clean_command_ctx() at > main.c:534] DEBUG1: Accepting client command ... [in > thread_manage_clients() at main.c:3826] DEBUG1: Wait for client response > [in thread_manage_clients() at main.c:3863] DEBUG1: Receiving data from > client ... [in thread_manage_clients() at main.c:3898] DEBUG1: Nothing > recv() from client... continuing [in thread_manage_clients() at > main.c:3902] DEBUG1: Clean command context structure [in > clean_command_ctx() at main.c:534] DEBUG1: Accepting client command ... [in > thread_manage_clients() at main.c:3826] DEBUG1: Wait for client response > [in thread_manage_clients() at main.c:3863] DEBUG1: Receiving data from > client ... [in thread_manage_clients() at main.c:3898] DEBUG1: Processing > client command 6 [in process_client_msg() at main.c:3309] DEBUG1: Getting > session sesCCC by name [in process_client_msg() at main.c:3364] DEBUG2: > Trying to find session by name sesCCC [in session_find_by_name() at > session.c:126] DEBUG1: Creating UST session [in create_ust_session() at > main.c:1965] DEBUG3: Created hashtable size 4 at 0x4fa08 of type 1 [in > lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at > 0x4fb28 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created > hashtable size 4 at 0x4fc48 of type 0 [in lttng_ht_new() at > hashtable.c:96] DEBUG2: UST trace session create successful [in > trace_ust_create_session() at trace-ust.c:119] DEBUG3: mkdir() recursive > /home/root/lttng-traces/sesCCC-20110327-045731/ust with mode 504 for uid 0 > and gid 0 [in run_as_mkdir_recursive() at runas.c:338] DEBUG1: Using > run_as_clone [in run_as() at runas.c:322] DEBUG1: Spawning consumerd [in > spawn_consumerd() at main.c:1659] DEBUG1: Using 32-bit UST consumer at: > /root/armfilesystem/lib/lttng/libexec/lttng-consumerd [in spawn_consumerd() > at main.c:1777] DEBUG2: Consumer pid 934 [in start_consumerd() at > main.c:1830] DEBUG2: Spawning consumer control thread [in start_consumerd() > at main.c:1833] DEBUG1: [thread] Manage consumer started [in > thread_manage_consumer() at main.c:982] DEBUG1: fd 3 of 1 added to pollfd > [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 9 of 2 added to > pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG2: Receiving code > from consumer err_sock [in thread_manage_consumer() at main.c:1043] DEBUG1: > consumer command socket ready [in thread_manage_consumer() at main.c:1062] > DEBUG1: fd 17 of 2 added to pollfd [in compat_poll_add() at > compat-poll.c:91] DEBUG2: Trace UST channel channel0 not found by name [in > trace_ust_find_channel_by_name() at trace-ust.c:52] DEBUG3: Created > hashtable size 4 at 0x51310 of type 0 [in lttng_ht_new() at > hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x51430 of type 1 [in > lttng_ht_new() at hashtable.c:96] DEBUG2: Trace UST channel channel0 > created [in trace_ust_create_channel() at trace-ust.c:181] DEBUG2: Channel > channel0 being created in UST global domain [in channel_ust_create() at > channel.c:267] DEBUG2: UST app adding channel channel0 to global domain for > session id 1 [in ust_app_create_channel_glb() at ust-app.c:1792] DEBUG2: > Channel channel0 created successfully [in channel_ust_create() at > channel.c:292] DEBUG2: Trace UST channel channel0 found by name [in > trace_ust_find_channel_by_name() at trace-ust.c:47] DEBUG2: Trace UST event > NOT found by name * [in trace_ust_find_event_by_name() at trace-ust.c:79] > DEBUG3: Created hashtable size 4 at 0x51550 of type 1 [in lttng_ht_new() at > hashtable.c:96] DEBUG2: Trace UST event *, loglevel (0,-1) created [in > trace_ust_create_event() at trace-ust.c:260] DEBUG1: UST app creating event > * for all apps for session id 1 [in ust_app_create_event_glb() at > ust-app.c:1916] DEBUG1: Event UST * created in channel channel0 [in > event_ust_enable_tracepoint() at event.c:446] DEBUG1: Sending response > (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3936] > DEBUG1: Clean command context structure [in clean_command_ctx() at > main.c:534] DEBUG1: Accepting client command ... [in > thread_manage_clients() at main.c:3826] DEBUG1: Wait for client response > [in thread_manage_clients() at main.c:3863] DEBUG1: Receiving data from > client ... [in thread_manage_clients() at main.c:3898] DEBUG1: Nothing > recv() from client... continuing [in thread_manage_clients() at > main.c:3902] DEBUG1: Clean command context structure [in > clean_command_ctx() at main.c:534] DEBUG1: Accepting client command ... [in > thread_manage_clients() at main.c:3826] DEBUG1: Wait for client response > [in thread_manage_clients() at main.c:3863] DEBUG1: Receiving data from > client ... [in thread_manage_clients() at main.c:3898] DEBUG1: Processing > client command 16 [in process_client_msg() at main.c:3309] DEBUG1: Getting > session sesCCC by name [in process_client_msg() at main.c:3364] DEBUG2: > Trying to find session by name sesCCC [in session_find_by_name() at > session.c:126] DEBUG1: Starting all UST traces [in > ust_app_start_trace_all() at ust-app.c:2207] DEBUG1: Sending response > (size: 16, retcode: Success) [in thread_manage_clients() at main.c:3936] > DEBUG1: Clean command context structure [in clean_command_ctx() at > main.c:534] DEBUG1: Accepting client command ... [in > thread_manage_clients() at main.c:3826] > ************************************************ > > > > > > Thanks in Advance, Pavan > > > -----Original Message----- From: David Goulet > [mailto:dgoulet at efficios.com] Sent: Tuesday, June 12, 2012 9:05 PM To: > Pavan Anumula Cc: lttng-dev at lists.lttng.org Subject: Re: [lttng-dev] Lttng > start failed > > Hi Pavan, > > The output below is normal and should be working fine. However, the problem > is that you have _NO_ debug message after the "lttng start". That command > should at least produce 10+ lines of messages after this last line: > > "DEBUG1: Accepting application registration [in thread_registration_apps() > at main.c:1423]" > > Can you enlight me and tell me if by doing "lttng create newsessiom" and > "lttng enable-event -a -k", you have output messages coming out of the > session daemon in -vvv mode ? > > If NOT, you are talking to another daemon! A quick "ps faux | grep > lttng-sessiond" could help clear that question. > > If only one daemon is running and stuck at the above debug message, the > next step is to tell me on what the daemon is waiting on using either > "strace -p" or "gdb" also. There is 6 threads so having the strace -p > output for all of them would help me greatly. > > Thanks! David > > On 12/06/12 10:05 AM, Mathieu Desnoyers wrote: >> * Pavan Anumula (pavan.anumula at sasken.com) wrote: >>> Hi Mathieu, Still I am unable to start tracing, And I am facing the >>> same start errors. Please kindly help me on this. This time I loaded >>> the modules by modprobe( not with insmod as I did before), Then I run >>> the command " arm-none-linux-gnueabi-lttng-sessiond -vvv " (before >>> arm-none-linux-gnueabi-lttng-sessiond -d), > >> When using "arm-none-linux-gnueabi-lttng-sessiond -vvv", you should >> _not_ run "arm-none-linux-gnueabi-lttng-sessiond -d". > > >>> The below is the Output, And It got hanged overthere. > >> This is normal: the sessiond is waiting for applications to connect. It >> does not hang, it just waits. > >> David, can you follow up on this issue ? It seems to be >> sessiond-related. > >> Thanks, > >> Mathieu > > >>> Later when I started lttng start I am acing the same erros below: >>> root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng start LTTng: >>> Failure to write metadata to buffers (timeout) Error: Starting kernel >>> trace failed >>> >>> ********************************************************************* >>> *************************** >>> >>> > DEBUG3: Creating LTTng run directory: /var/run/lttng [in > create_lttng_rundir() at main.c:4315] >>> DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error [in >>> main() at main.c:4543] DEBUG2: Kernel consumer cmd path: >>> /var/run/lttng/kconsumerd/command [in main() at main.c:4545] DEBUG1: >>> Client socket path /var/run/lttng/client-lttng-sessiond [in main() at >>> main.c:4592] DEBUG1: Application socket path >>> /var/run/lttng/apps-lttng-sessiond [in main() at main.c:4593] DEBUG1: >>> LTTng run directory path: /var/run/lttng [in main() at main.c:4594] >>> DEBUG2: UST consumer 32 bits err path: >>> /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] DEBUG2: >>> UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command >>> [in main() at main.c:4605] DEBUG2: UST consumer 64 bits err path: >>> /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] DEBUG2: >>> UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command >>> [in main() at main.c:4616] DEBUG3: Created hashtable size 4 at 0x4a080 >>> of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: Created >>> hashtable size 4 at 0x4a168 of type 1 [in lttng_ht_new() at >>> hashtable.c:96] DEBUG2: Creating consumer directory: >>> /var/run/lttng/kconsumerd [in set_consumer_sockets() at main.c:4357] >>> DEBUG1: Modprobe successfully lttng-tracer [in modprobe_lttng_control() >>> at modprobe.c:163] DEBUG2: Kernel tracer version validated (major >>> version 2) [in kernel_validate_version() at kernel.c:675] DEBUG1: >>> Modprobe successfully lttng-ftrace [in modprobe_lttng_data() at >>> modprobe.c:199] DEBUG1: Modprobe successfully lttng-kprobes [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully >>> lttng-kretprobes [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: >>> Modprobe successfully lttng-lib-ring-buffer [in modprobe_lttng_data() >>> at modprobe.c:199] DEBUG1: Modprobe successfully >>> lttng-ring-buffer-client-discard [in modprobe_lttng_data() at >>> modprobe.c:199] DEBUG1: Modprobe successfully >>> lttng-ring-buffer-client-overwrite [in modprobe_lttng_data() at >>> modprobe.c:199] DEBUG1: Modprobe successfully >>> lttng-ring-buffer-metadata-client [in modprobe_lttng_data() at >>> modprobe.c:199] DEBUG1: Modprobe successfully >>> lttng-ring-buffer-client-mmap-discard [in modprobe_lttng_data() at >>> modprobe.c:199] DEBUG1: Modprobe successfully >>> lttng-ring-buffer-client-mmap-overwrite [in modprobe_lttng_data() at >>> modprobe.c:199] DEBUG1: Modprobe successfully >>> lttng-ring-buffer-metadata-mmap-client [in modprobe_lttng_data() at >>> modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-lttng [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully >>> lttng-types [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: >>> Modprobe successfully lttng-probe-block [in modprobe_lttng_data() at >>> modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-irq [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully >>> lttng-probe-kvm [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: >>> Modprobe successfully lttng-probe-sched [in modprobe_lttng_data() at >>> modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-signal [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe successfully >>> lttng-probe-statedump [in modprobe_lttng_data() at modprobe.c:199] >>> DEBUG1: Modprobe successfully lttng-probe-timer [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Kernel tracer fd 6 [in >>> init_kernel_tracer() at main.c:1887] DEBUG2: Creating consumer >>> directory: /var/run/lttng/ustconsumerd64 [in set_consumer_sockets() at >>> main.c:4357] DEBUG2: Creating consumer directory: >>> /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() at >>> main.c:4357] DEBUG1: Signal handler set for SIGTERM, SIGPIPE and SIGINT >>> [in set_signal_handler() at main.c:4449] DEBUG1: All permissions are >>> set [in set_permissions() at main.c:4250] DEBUG1: poll set max size set >>> to 65535 [in compat_poll_set_max_size() at compat-poll.c:196] DEBUG1: >>> [thread] Manage application started [in thread_manage_apps() at >>> main.c:1179] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at >>> compat-poll.c:91] DEBUG1: fd 13 of 2 added to pollfd [in >>> compat_poll_add() at compat-poll.c:91] DEBUG1: Apps thread polling on 2 >>> fds [in thread_manage_apps() at main.c:1200] DEBUG1: Thread manage >>> kernel started [in thread_manage_kernel() at main.c:876] DEBUG1: fd 3 >>> of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: >>> fd 11 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] >>> DEBUG1: Updating kernel poll set [in update_kernel_poll() at >>> main.c:748] DEBUG1: Thread kernel polling on 2 fds [in >>> thread_manage_kernel() at main.c:905] DEBUG1: [thread] Manage >>> application registration started [in thread_registration_apps() at >>> main.c:1392] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at >>> compat-poll.c:91] DEBUG1: fd 10 of 2 added to pollfd [in >>> compat_poll_add() at compat-poll.c:91] DEBUG1: Notifying applications >>> of session daemon state: 1 [in notify_ust_apps() at main.c:687] >>> DEBUG1: [thread] Dispatch UST command started [in >>> thread_dispatch_ust_registration() at main.c:1324] DEBUG1: Futex n to 1 >>> prepare done [in futex_nto1_prepare() at futex.c:73] DEBUG1: Woken up >>> but nothing in the UST command queue [in >>> thread_dispatch_ust_registration() at main.c:1334] DEBUG1: [thread] >>> Manage client started [in thread_manage_clients() at main.c:3794] >>> DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at >>> compat-poll.c:91] DEBUG1: fd 9 of 2 added to pollfd [in >>> compat_poll_add() at compat-poll.c:91] DEBUG1: Accepting client command >>> ... [in thread_manage_clients() at main.c:3826] DEBUG1: Got the wait >>> shm fd 15 [in get_wait_shm() at shm.c:117] DEBUG1: Futex wait update >>> active 1 [in futex_wait_update() at futex.c:62] DEBUG1: Accepting >>> application registration [in thread_registration_apps() at >>> main.c:1423] >>> ********************************************************************* >>> *********************************** >>> >>> > Am I missing anything?? >>> >>> Thanks in Advance, Pavan >>> >>> >>> ________________________________________ From: Mathieu Desnoyers >>> [mathieu.desnoyers at efficios.com] Sent: Monday, June 11, 2012 7:24 PM >>> To: Pavan Anumula Cc: lttng-dev at lists.lttng.org Subject: Re: >>> [lttng-dev] Lttng start failed >>> >>> * Pavan Anumula (pavan.anumula at sasken.com) wrote: >>>> Hi Mathieu, >>>> >>>> Please find the info below as per your comments >>>> >>>> >>>> ########## Output of command "arm-none-linux-gnueabi-lttng-sessiond >>>> -vvv " , After loading the modules ####### >>>> >>>> DEBUG3: Creating LTTng run directory: /var/run/lttng [in >>>> create_lttng_rundir() at main.c:4315] DEBUG2: Kernel consumer err >>>> path: /var/run/lttng/kconsumerd/error [in main() at main.c:4543] >>>> DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command >>>> [in main() at main.c:4545] DEBUG1: Client socket path >>>> /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] >>>> DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond >>>> [in main() at main.c:4593] DEBUG1: LTTng run directory path: >>>> /var/run/lttng [in main() at main.c:4594] DEBUG2: UST consumer 32 >>>> bits err path: /var/run/lttng/ustconsumerd32/error [in main() at >>>> main.c:4603] DEBUG2: UST consumer 32 bits cmd path: >>>> /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] >>>> DEBUG2: UST consumer 64 bits err path: >>>> /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] >>>> DEBUG2: UST consumer 64 bits cmd path: >>>> /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] >>>> DEBUG3: Created hashtable size 4 at 0x4a080 of type 1 [in >>>> lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at >>>> 0x4a168 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: >>>> Creating consumer directory: /var/run/lttng/kconsumerd [in >>>> set_consumer_sockets() at main.c:4357] FATAL: Module lttng_tracer not >>>> found. Error: Unable to load module lttng-tracer >>> >>> Hrm. You want to do kernel tracing, but modprobe cannot find the lttng >>> kernel tracer modules. You might want to run depmod -a or something >>> like that on your target, and ensure that modprobe works properly. >>> >>> Thanks, >>> >>> Mathieu >>> >>>> DEBUG2: Kernel tracer version validated (major version 2) [in >>>> kernel_validate_version() at kernel.c:675] DEBUG1: Modprobe >>>> successfully lttng-ftrace [in modprobe_lttng_data() at >>>> modprobe.c:199] DEBUG1: Modprobe successfully lttng-kprobes [in >>>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>>> successfully lttng-kretprobes [in modprobe_lttng_data() at >>>> modprobe.c:199] FATAL: Module lttng_lib_ring_buffer not found. Error: >>>> Unable to load module lttng-lib-ring-buffer FATAL: Module >>>> lttng_ring_buffer_client_discard not found. Error: Unable to load >>>> module lttng-ring-buffer-client-discard FATAL: Module >>>> lttng_ring_buffer_client_overwrite not found. Error: Unable to load >>>> module lttng-ring-buffer-client-overwrite FATAL: Module >>>> lttng_ring_buffer_metadata_client not found. Error: Unable to load >>>> module lttng-ring-buffer-metadata-client FATAL: Module >>>> lttng_ring_buffer_client_mmap_discard not found. Error: Unable to >>>> load module lttng-ring-buffer-client-mmap-discard FATAL: Module >>>> lttng_ring_buffer_client_mmap_overwrite not found. Error: Unable to >>>> load module lttng-ring-buffer-client-mmap-overwrite FATAL: Module >>>> lttng_ring_buffer_metadata_mmap_client not found. Error: Unable to >>>> load module lttng-ring-buffer-metadata-mmap-client FATAL: Module >>>> lttng_probe_lttng not found. Error: Unable to load module >>>> lttng-probe-lttng DEBUG1: Modprobe successfully lttng-types [in >>>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>>> successfully lttng-probe-block [in modprobe_lttng_data() at >>>> modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-irq [in >>>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>>> successfully lttng-probe-kvm [in modprobe_lttng_data() at >>>> modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-sched [in >>>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>>> successfully lttng-probe-signal [in modprobe_lttng_data() at >>>> modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-statedump >>>> [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>>> successfully lttng-probe-timer [in modprobe_lttng_data() at >>>> modprobe.c:199] DEBUG1: Kernel tracer fd 6 [in init_kernel_tracer() >>>> at main.c:1887] DEBUG2: Creating consumer directory: >>>> /var/run/lttng/ustconsumerd64 [in set_consumer_sockets() at >>>> main.c:4357] DEBUG2: Creating consumer directory: >>>> /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() at >>>> main.c:4357] DEBUG1: Signal handler set for SIGTERM, SIGPIPE and >>>> SIGINT [in set_signal_handler() at main.c:4449] DEBUG1: All >>>> permissions are set [in set_permissions() at main.c:4250] DEBUG1: >>>> poll set max size set to 65535 [in compat_poll_set_max_size() at >>>> compat-poll.c:196] DEBUG1: [thread] Manage client started [in >>>> thread_manage_clients() at main.c:3794] DEBUG1: fd 3 of 1 added to >>>> pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 9 of 2 >>>> added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: >>>> Accepting client command ... [in thread_manage_clients() at >>>> main.c:3826] DEBUG1: [thread] Dispatch UST command started [in >>>> thread_dispatch_ust_registration() at main.c:1324] DEBUG1: Futex n to >>>> 1 prepare done [in futex_nto1_prepare() at futex.c:73] DEBUG1: Woken >>>> up but nothing in the UST command queue [in >>>> thread_dispatch_ust_registration() at main.c:1334] DEBUG1: Thread >>>> manage kernel started [in thread_manage_kernel() at main.c:876] >>>> DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at >>>> compat-poll.c:91] DEBUG1: fd 11 of 2 added to pollfd [in >>>> compat_poll_add() at compat-poll.c:91] DEBUG1: Updating kernel poll >>>> set [in update_kernel_poll() at main.c:748] DEBUG1: Thread kernel >>>> polling on 2 fds [in thread_manage_kernel() at main.c:905] DEBUG1: >>>> [thread] Manage application started [in thread_manage_apps() at >>>> main.c:1179] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() >>>> at compat-poll.c:91] DEBUG1: fd 13 of 2 added to pollfd [in >>>> compat_poll_add() at compat-poll.c:91] DEBUG1: Apps thread polling on >>>> 2 fds [in thread_manage_apps() at main.c:1200] DEBUG1: [thread] >>>> Manage application registration started [in >>>> thread_registration_apps() at main.c:1392] DEBUG1: fd 3 of 1 added to >>>> pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 10 of 2 >>>> added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: >>>> Notifying applications of session daemon state: 1 [in >>>> notify_ust_apps() at main.c:687] DEBUG1: Got the wait shm fd 15 [in >>>> get_wait_shm() at shm.c:117] DEBUG1: Futex wait update active 1 [in >>>> futex_wait_update() at futex.c:62] DEBUG1: Accepting application >>>> registration [in thread_registration_apps() at main.c:1423] >>>> >>>> >>>> >>>> #### Output of command "arm-none-linux-gnueabi-lttng-sessiond -vvv >>>> " before loading the modules ############# >>>> >>>> DEBUG3: Creating LTTng run directory: /var/run/lttng [in >>>> create_lttng_rundir() at main.c:4315] DEBUG2: Kernel consumer err >>>> path: /var/run/lttng/kconsumerd/error [in main() at main.c:4543] >>>> DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command >>>> [in main() at main.c:4545] DEBUG1: Client socket path >>>> /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] >>>> DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond >>>> [in main() at main.c:4593] DEBUG1: LTTng run directory path: >>>> /var/run/lttng [in main() at main.c:4594] DEBUG2: UST consumer 32 >>>> bits err path: /var/run/lttng/ustconsumerd32/error [in main() at >>>> main.c:4603] DEBUG2: UST consumer 32 bits cmd path: >>>> /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] >>>> DEBUG2: UST consumer 64 bits err path: >>>> /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] >>>> DEBUG2: UST consumer 64 bits cmd path: >>>> /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] >>>> DEBUG3: Created hashtable size 4 at 0x4a080 of type 1 [in >>>> lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at >>>> 0x4a168 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: >>>> Creating consumer directory: /var/run/lttng/kconsumerd [in >>>> set_consumer_sockets() at main.c:4357] FATAL: Module lttng_tracer not >>>> found. Error: Unable to load module lttng-tracer DEBUG1: Failed to >>>> open /proc/lttng [in init_kernel_tracer() at main.c:1871] Error: >>>> Unable to remove module lttng-tracer Warning: No kernel tracer >>>> available DEBUG2: Creating consumer directory: >>>> /var/run/lttng/ustconsumerd64 [in set_consumer_sockets() at >>>> main.c:4357] DEBUG2: Creating consumer directory: >>>> /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() at >>>> main.c:4357] DEBUG1: Signal handler set for SIGTERM, SIGPIPE and >>>> SIGINT [in set_signal_handler() at main.c:4449] DEBUG1: All >>>> permissions are set [in set_permissions() at main.c:4250] DEBUG1: >>>> poll set max size set to 65535 [in compat_poll_set_max_size() at >>>> compat-poll.c:196] DEBUG1: [thread] Manage application started [in >>>> thread_manage_apps() at main.c:1179] DEBUG1: fd 3 of 1 added to >>>> pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 12 of 2 >>>> added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: >>>> Apps thread polling on 2 fds [in thread_manage_apps() at main.c:1200] >>>> DEBUG1: Thread manage kernel started [in thread_manage_kernel() at >>>> main.c:876] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() >>>> at compat-poll.c:91] DEBUG1: fd 10 of 2 added to pollfd [in >>>> compat_poll_add() at compat-poll.c:91] DEBUG1: Updating kernel poll >>>> set [in update_kernel_poll() at main.c:748] DEBUG1: Thread kernel >>>> polling on 2 fds [in thread_manage_kernel() at main.c:905] DEBUG1: >>>> [thread] Manage application registration started [in >>>> thread_registration_apps() at main.c:1392] DEBUG1: fd 3 of 1 added to >>>> pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 9 of 2 >>>> added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: >>>> Notifying applications of session daemon state: 1 [in >>>> notify_ust_apps() at main.c:687] DEBUG1: [thread] Dispatch UST >>>> command started [in thread_dispatch_ust_registration() at >>>> main.c:1324] DEBUG1: Futex n to 1 prepare done [in >>>> futex_nto1_prepare() at futex.c:73] DEBUG1: Woken up but nothing in >>>> the UST command queue [in thread_dispatch_ust_registration() at >>>> main.c:1334] DEBUG1: [thread] Manage client started [in >>>> thread_manage_clients() at main.c:3794] DEBUG1: fd 3 of 1 added to >>>> pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 8 of 2 >>>> added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: >>>> Accepting client command ... [in thread_manage_clients() at >>>> main.c:3826] DEBUG1: Got the wait shm fd 14 [in get_wait_shm() at >>>> shm.c:117] DEBUG1: Futex wait update active 1 [in futex_wait_update() >>>> at futex.c:62] DEBUG1: Accepting application registration [in >>>> thread_registration_apps() at main.c:1423] >>>> >>>> >>>> Regards, Pavan >>>> >>>> -----Original Message----- From: Mathieu Desnoyers >>>> [mailto:mathieu.desnoyers at efficios.com] Sent: Saturday, June 09, >>>> 2012 6:03 PM To: Pavan Anumula Cc: lttng-dev at lists.lttng.org Subject: >>>> Re: [lttng-dev] Lttng start failed >>>> >>>> * Pavan Anumula (pavan.anumula at sasken.com) wrote: >>>>> Hi Mathue, >>>>> >>>>> Thanks for the quick reply, >>>>> >>>>> After inserting lttng modules , I had given the command >>>>> "arm-none-linux-gnueabi-lttng-sessiond -vvv" as you said, Below >>>>> is the output where there are error messages. Please help me in >>>>> resolving the issue, SO that I can catch kernel and user traces. >>>>> >>>>> >>>>> root at arago:/usr/lttng/modules# >>>>> arm-none-linux-gnueabi-lttng-sessiond -d >>>>> root at arago:/usr/lttng/modules# >>>>> arm-none-linux-gnueabi-lttng-sessiond -vvv DEBUG3: Creating LTTng >>>>> run directory: /var/run/lttng [in create_lttng_rundir() at >>>>> main.c:4315] DEBUG2: Kernel consumer err path: >>>>> /var/run/lttng/kconsumerd/error [in main() at main.c:4543] DEBUG2: >>>>> Kernel consumer cmd path: /var/run/lttng/kconsumerd/command [in >>>>> main() at main.c:4545] DEBUG1: Client socket path >>>>> /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] >>>>> DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond >>>>> [in main() at main.c:4593] DEBUG1: LTTng run directory path: >>>>> /var/run/lttng [in main() at main.c:4594] DEBUG2: UST consumer 32 >>>>> bits err path: /var/run/lttng/ustconsumerd32/error [in main() at >>>>> main.c:4603] DEBUG2: UST consumer 32 bits cmd path: >>>>> /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] >>>>> DEBUG2: UST consumer 64 bits err path: >>>>> /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] >>>>> DEBUG2: UST consumer 64 bits cmd path: >>>>> /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] >>>>> Error: Already running daemon. >>>> >>>> Please kill the lttng-sessiond that is already started (the one with >>>> -d). Instead of that one, run the sessiond with: >>>> >>>> lttng-sessiond -vvv >>>> >>>> (don't run lttng-sessiond -d before) >>>> >>>> Thanks, >>>> >>>> Mathieu >>>> >>>>> >>>>> >>>>> >>>>> >>>>> Thanks in advance, Pavan >>>>> >>>>> -----Original Message----- From: Mathieu Desnoyers >>>>> [mailto:mathieu.desnoyers at efficios.com] Sent: Friday, June 08, >>>>> 2012 11:12 PM To: Pavan Anumula Cc: lttng-dev at lists.lttng.org >>>>> Subject: Re: [lttng-dev] Lttng start failed >>>>> >>>>> * Pavan Anumula (pavan.anumula at sasken.com) wrote: >>>>>> Hi , >>>>>> >>>>>> I am new to LTTng usage, I am trying to use LTTng 2.0.1 and >>>>>> LTTng-modules-2.0.2 for ARM(omapL138), with Linux kernel 2.6.33 >>>>>> wit RT-29 patch on it. >>>>>> >>>>>> After enabling all the kernel events I am facing the below >>>>>> errors. I loaded all the modules manually by insmod. >>>>>> >>>>>> >>>>>> >>>>>> root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng create >>>>>> newsessiom Session newsessiom created. Traces will be written in >>>>>> /home/root/lttng-traces/newsessiom-20110325-154922 >>>>>> >>>>>> root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng >>>>>> enable-event -a --kernel All kernel events are enabled in >>>>>> channel channel0 >>>>>> >>>>>> root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng start >>>>>> LTTng: Failure to write metadata to buffers (timeout) Error: >>>>>> Starting kernel trace failed >>>>>> >>>>>> Please kindly help me on this. >>>>> >>>>> I think you should look into the lttng-sessiond --help : >>>>> >>>>> --consumerd32-path PATH Specify path for the 32-bit UST >>>>> consumer daemon binary --consumerd32-libdir PATH Specify path for >>>>> the 32-bit UST consumer daemon libraries --consumerd64-path PATH >>>>> Specify path for the 64-bit UST consumer daemon binary >>>>> --consumerd64-libdir PATH Specify path for the 64-bit UST >>>>> consumer daemon libraries >>>>> >>>>> options. My guess is that lttng-sessiond is not able to find the >>>>> consumerd binary files, maybe due to a rename or because they have >>>>> been moved after install. >>>>> >>>>> One more thing that might help is to launch the lttng-sessiond >>>>> with "-vvv" : it will provide verbose output and let us know where >>>>> things fall apart. >>>>> >>>>> Thanks, >>>>> >>>>> Mathieu >>>>> >>>>> >>>>>> >>>>>> >>>>>> Thanks in advance, Pavan >>>>>> >>>>>> ________________________________ SASKEN BUSINESS DISCLAIMER: >>>>>> This message may contain confidential, proprietary or legally >>>>>> privileged information. In case you are not the original >>>>>> intended Recipient of the message, you must not, directly or >>>>>> indirectly, use, disclose, distribute, print, or copy any part of >>>>>> this message and you are requested to delete it and inform the >>>>>> sender. Any views expressed in this message are those of the >>>>>> individual sender unless otherwise stated. Nothing contained in >>>>>> this message shall be construed as an offer or acceptance of any >>>>>> offer by Sasken Communication Technologies Limited ("Sasken") >>>>>> unless sent with that express intent and with due authority of >>>>>> Sasken. Sasken has taken enough precautions to prevent the spread >>>>>> of viruses. However the company accepts no liability for any >>>>>> damage caused by any virus transmitted by this email. Read >>>>>> Disclaimer at http://www.sasken.com/extras/mail_disclaimer.html >>>>> >>>>>> _______________________________________________ lttng-dev >>>>>> mailing list lttng-dev at lists.lttng.org >>>>>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev >>>>> >>>>> >>>>> -- Mathieu Desnoyers Operating System Efficiency R&D Consultant >>>>> EfficiOS Inc. http://www.efficios.com >>>> >>>> -- Mathieu Desnoyers Operating System Efficiency R&D Consultant >>>> EfficiOS Inc. http://www.efficios.com >>> >>> -- Mathieu Desnoyers Operating System Efficiency R&D Consultant >>> EfficiOS Inc. http://www.efficios.com > > > _______________________________________________ lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBAgAGBQJP201YAAoJEELoaioR9I02a7AH/3sTHsbxRJ3z+//6+wPQu7GC 6KIZxfBhhq4WSSZRJGHvD9O6QCHPgUmrYjR4leONB9//FfoWdzzRZb0IDErpJ3+L B8I49nd7X2UHp1jf7CpbJUzrs1AQZa3K32D/nRlF3LceOGnvIpkKstur5msXvN5B msGap8UQ8NsxkyCRW8cmPNt/Qm7lMsrg9zeP5VGEncj7dywIqaPfjz7MGFwNp4Vz uxxBusAjnI50Kmx+ETYZmTiqEIPCV9rnpVK4T2LtZgX/+rh06gDkkvW6xy0wO7pf ymZ0MytfvuKaHXmQnXyGvWhrodmn+D9E32DwyrcuhZa6K0ZPE5eDRbxUT5WGbuQ= =VYvD -----END PGP SIGNATURE----- From mathieu.desnoyers at efficios.com Fri Jun 15 11:12:34 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Fri, 15 Jun 2012 11:12:34 -0400 Subject: [lttng-dev] lttng - ubuntu 11.10 - problem In-Reply-To: <4FDB34C8.7000505@polymtl.ca> References: <1339746088.90723.YahooMailNeo@web161201.mail.bf1.yahoo.com> <4FDAEF5C.9030102@gmail.com> <4FDB34C8.7000505@polymtl.ca> Message-ID: <20120615151234.GA28295@Krystal> * David Goulet (david.goulet at polymtl.ca) wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi, > > In order to trace the kernel, you need the LTTng kernel modules available in > the package "lttng-modules-dkms". So make sure you have these three packages > installed: > > # sudo apt-get install lttng-tools lttng-modules-dkms babeltrace hopefully lttng-modules-dkms issues "depmod -a" after the make modules_install ? Mathieu > > The error message you got indicates that there is simply no LTTng modules > loaded. Please make sure they are loaded and if the problem persist, let us know! > > Thanks! > David > > On 15/06/12 04:16 AM, Francis Giraldeau wrote: > > Le 2012-06-15 09:41, somanath sahoo a ?crit : > >> Hi, > >> > >> As i am new to lttng usage, I have followed the instructions from > >> in order to install the lttng > >> tool set in my ubuntu 11.10. After successful installation, i followed > >> the getting started link on lttng.org in > >> "root mode". > > > > You may verify if kernel modules are loaded correctly. If not, then there > > may be a problem while the installation. > > > > lsmod | grep lttng > > > > If not, you may try to load them and see if any error message occurs: > > > > sudo modprobe lttng_tracer > > > > Francis > > > > > > > > _______________________________________________ lttng-dev mailing list > > lttng-dev at lists.lttng.org > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.12 (GNU/Linux) > > iQEcBAEBAgAGBQJP2zTIAAoJEELoaioR9I02CVEH/1ripsIVxDQ2JRPNXs0TVWLD > 5B/qf3oKTohuKCoKE7Q7ZLyo1A9WN0z9JFouVecEPD7MisdJqYufEXkUNbxHg2+y > 9nVT8xF7gToLRBi/qC/hH0JN6AADJmqTVxYAvygv8qWNkLR3PSqguDhE5nM9vrtF > LJSEYlYQh0dktkBM9ESm0lrnbZO/AtXxQB4IbCQSwP1tsNmftEdZ24H8DN0/QWl9 > niRT6XpBzogNJfQQOPFtJTFxpwKHiMQXdE78UtzzMBPZ9P57Go9jMgP+2FELFZl4 > wcu+XMIF2Gx81AajfTkGAjPOczagqKzrgnMCduQaNgCI1hRoWOn7Yu1jTmA6Yxc= > =oy1n > -----END PGP SIGNATURE----- > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Fri Jun 15 11:13:42 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Fri, 15 Jun 2012 11:13:42 -0400 Subject: [lttng-dev] Prerequsites for adding perf context In-Reply-To: <524C960C5DFC794E82BE548D825F05CF283461BF@EU-MBX-01.mgc.mentorg.com> References: <1339746088.90723.YahooMailNeo@web161201.mail.bf1.yahoo.com> <4FDAEF5C.9030102@gmail.com> <524C960C5DFC794E82BE548D825F05CF283461BF@EU-MBX-01.mgc.mentorg.com> Message-ID: <20120615151342.GB28295@Krystal> * Oestman, Fredrik (Fredrik_Oestman at mentor.com) wrote: > Hi, > > > What exactly is needed on a system for > > # lttng add-context -k -t perf: > > to work? Does perf have to be installed? > > I have hw perfevents, but not perf, and get > > hw perfevents: unable to reserve pmu CONFIG_PERF_EVENTS. And what version of kernel do you use ? > > > cheers, > > Fredrik ?stman > > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From alexandre.montplaisir at polymtl.ca Fri Jun 15 11:32:02 2012 From: alexandre.montplaisir at polymtl.ca (Alexandre Montplaisir) Date: Fri, 15 Jun 2012 11:32:02 -0400 Subject: [lttng-dev] lttng - ubuntu 11.10 - problem In-Reply-To: <20120615151234.GA28295@Krystal> References: <1339746088.90723.YahooMailNeo@web161201.mail.bf1.yahoo.com> <4FDAEF5C.9030102@gmail.com> <4FDB34C8.7000505@polymtl.ca> <20120615151234.GA28295@Krystal> Message-ID: <4FDB5572.7000209@polymtl.ca> On 12-06-15 11:12 AM, Mathieu Desnoyers wrote: > * David Goulet (david.goulet at polymtl.ca) wrote: > Hi, > > In order to trace the kernel, you need the LTTng kernel modules > available in > the package "lttng-modules-dkms". So make sure you have these three > packages > installed: > > # sudo apt-get install lttng-tools lttng-modules-dkms babeltrace > > > hopefully lttng-modules-dkms issues "depmod -a" after the make > > modules_install ? Yeah it does, it runs it once at the end after all modules have been compiled. In Somanath's original email, there are two sessiond running, one as root and one as the user. I'm curious as to how the user one got started, perhaps the commands weren't executed as root somehow? Cheers, -- Alexandre Montplaisir DORSAL lab, ?cole Polytechnique de Montr?al From mathieu.desnoyers at efficios.com Sat Jun 16 17:14:55 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Sat, 16 Jun 2012 17:14:55 -0400 Subject: [lttng-dev] LPC2012 Tracing Summit, August 30th, 2012 (was: [CFP] Tracing Micro-conference at LPC2012 (August 28-30, San Diego)) In-Reply-To: <20120515234826.GA10967@Krystal> References: <20120515234826.GA10967@Krystal> Message-ID: <20120616211455.GA10257@Krystal> Hi, Due to the high interest shown in the event, the Linux Plumbers Tracing Micro-Conference will be held as a full day Tracing Summit on August 30th. Please refer to the http://tracingsummit.org/wiki/TracingSummit2012 page for details. There has been some back and forth recently on the exact date (28th or 30th), and between co-location with either LinuxCon or Plumbers (for full day room availability). We have been trying to avoid major schedule conflicts with co-located events. Sorry about that. So just to clear things up: the Tracing Summit will be held within the Linux Plumbers Conference 2012. Registration to Linux Plumbers Conference is required for attendees and speakers. It will be possible to cancel LinuxCon registration if some of you only registered to LinuxCon for the Tracing Summit add-on. If you wish to do so, please contact the Linux Foundation. Thanks! Mathieu * Mathieu Desnoyers (mathieu.desnoyers at efficios.com) wrote: > Hi, > > We are organizing a tracing micro-conference taking place in San Diego, > within Linux Plumbers Conference 2012. The micro-conference is planned > to be held somewhere between August 28 and 30, in San Diego, CA. > > If you are interested to present, please don't hesitate to get in touch > with us quickly (ideally within this week, or next week), since > deadlines are approaching. > > Our intent is to gather people involved in development and users of > tracing tools and trace analysis tools to allow discussing the latest > developments, allow users to express their needs, and generally ensure > that developers and end users understand each other. > > We are welcoming presentations from both end users and developers, on > topics covering, but not limited to: > > - Trace collection and extraction, > - Trace filtering, > - Trace aggregation, > - Trace formats, > - Tracing multi-core systems, > - Trace abstraction, > - Trace modeling, > - Automated trace analysis (e.g. dependency analysis), > - Tracing large clusters and distributed systems, > - Hardware-level tracing (e.g. DSP, GPU, bare-metal), > - Trace visualisation, > - Interaction between debugging and tracing, > - Tracing remote control, > - Analysis of large trace datasets. > > It can cover both recently available technologies and ongoing work. > > If you are interested to present, please get in touch with Dominique > Toupin or myself. The micro-conference wiki can be accessed at: > http://wiki.linuxplumbersconf.org/2012:tracing > > Feedback is welcome! > > Thank you, > > Dominique Toupin & Mathieu Desnoyers > > -- > Mathieu Desnoyers > Operating System Efficiency R&D Consultant > EfficiOS Inc. > http://www.efficios.com -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From lars at grandepotatis.se Sun Jun 17 11:08:23 2012 From: lars at grandepotatis.se (Lars) Date: Sun, 17 Jun 2012 17:08:23 +0200 Subject: [lttng-dev] LTTng-UST alternative to LD_PRELOAD Message-ID: <1339945703.2599.21.camel@DellE5500> Hello, I have recently started to use LTTng-UST and am using LD_PRELOAD to make it possilbe to execute instrumented code on systems without LTTng-UST installed. To define LD_PRELOAD however affects too much. For instance all forked processes also get the library loaded. This can be fixed by removing the library from the LD_PRELOAD variable early in the main function. This looks odd and required some explaining in comments. Also, in my case, the instrumented program is started with a "wrapper" program which I can not alter. This wrapper also gets the library loaded. As an alternative to using LD_PRELOAD I would like to use an explicit function, like; int traceload(char const* library_path); that is called early in main (instead of the LD_PRELOAD environment manipulation) and loads the library with dlopen if it exists. I have already tried this (of course) and it almost works already. The tracepoints becomes visible but I get no trace data written. So I think a really small modification is needed. Is anything like this planned? Or can anybody give me a hint on what is missing? I checked the mailing list a couple of months back but did not find anything. Best Regards, Lars Ekman From bapi_mvit2004 at yahoo.com Mon Jun 18 02:59:41 2012 From: bapi_mvit2004 at yahoo.com (somanath sahoo) Date: Sun, 17 Jun 2012 23:59:41 -0700 (PDT) Subject: [lttng-dev] lttng - ubuntu 11.10 - problem Message-ID: <1340002781.5676.YahooMailNeo@web161206.mail.bf1.yahoo.com> Hi, I have tried to reinstall lttng-tools,? lttng-modules-dkms? and babeltrace. the reinstallation of lttng-tools and babeltrace has been successful. But reinstallation of lttng-modules-dkms has been failing. the log is provided below -------------------------start of log-------------- # apt-get install --reinstall lttng-modules-dkms Reading package lists... Done Building dependency tree?????? Reading state information... Done 0 upgraded, 0 newly installed, 1 reinstalled, 0 to remove and 0 not upgraded. Need to get 0 B/198 kB of archives. After this operation, 0 B of additional disk space will be used. (Reading database ... 199394 files and directories currently installed.) Preparing to replace lttng-modules-dkms 2.0.0-0ubuntu2~oneiric1 (using .../lttng-modules-dkms_2.0.0-0ubuntu2~oneiric1_all.deb) ... ------------------------------ Deleting module version: 2.0.0 completely from the DKMS tree. ------------------------------ Done. Unpacking replacement lttng-modules-dkms ... Setting up lttng-modules-dkms (2.0.0-0ubuntu2~oneiric1) ... Loading new lttng-modules-2.0.0 DKMS files... Building only for 3.0.0-21-generic Building initial module for 3.0.0-21-generic Error! Bad return status for module build on kernel: 3.0.0-21-generic (i686) Consult /var/lib/dkms/lttng-modules/2.0.0/build/make.log for more information. ------------------------------ END OF LOG ------------------------- The content of make.log is provided below : DKMS make.log for lttng-modules-2.0.0 for kernel 3.0.0-21-generic (i686) Mon Jun 18 12:05:51 IST 2012 make: Entering directory `/usr/src/linux-headers-3.0.0-21-generic' ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-ring-buffer-client-discard.o ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-ring-buffer-client-overwrite.o ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-ring-buffer-metadata-client.o ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-ring-buffer-client-mmap-discard.o ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-ring-buffer-client-mmap-overwrite.o ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-ring-buffer-metadata-mmap-client.o ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-statedump-impl.o ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/wrapper/irqdesc.o ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-events.o ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-abi.o ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-probes.o ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context.o ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-pid.o ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-procname.o ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-prio.o ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-nice.o ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-vpid.o ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-tid.o ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-vtid.o ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-ppid.o ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-vppid.o ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-calibrate.o ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/wrapper/random.o ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-syscalls.o /var/lib/dkms/lttng-modules/2.0.0/build/lttng-syscalls.c:32:38: error: macro "is_compat_task" passed 1 arguments, but takes just 0 /var/lib/dkms/lttng-modules/2.0.0/build/lttng-syscalls.c:33:1: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?{? token make[1]: *** [/var/lib/dkms/lttng-modules/2.0.0/build/lttng-syscalls.o] Error 1 make: *** [_module_/var/lib/dkms/lttng-modules/2.0.0/build] Error 2 make: Leaving directory `/usr/src/linux-headers-3.0.0-21-generic' ------------------------- End of make.log ----------------- As per the compilation errors in the file lttng-syscalls.c, i dont think these errors are valids. It may be due to somethings else. Kindly guide how to come over this reinstallation problem of "lttng-modules-dkms" and please let me know if i need to do any other alternatives. Thanks & Regards, Somanath ------------------------------ Message: 3 Date: Fri, 15 Jun 2012 09:12:40 -0400 From: David Goulet To: Francis Giraldeau Cc: "lttng-dev at lists.lttng.org" Subject: Re: [lttng-dev] lttng - ubuntu 11.10 - problem Message-ID: <4FDB34C8.7000505 at polymtl.ca> Content-Type: text/plain; charset=ISO-8859-1 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, In order to trace the kernel, you need the LTTng kernel modules available in the package "lttng-modules-dkms". So make sure you have these three packages installed: # sudo apt-get install lttng-tools lttng-modules-dkms babeltrace The error message you got indicates that there is simply no LTTng modules loaded. Please make sure they are loaded and if the problem persist, let us know! Thanks! David On 15/06/12 04:16 AM, Francis Giraldeau wrote: > Le 2012-06-15 09:41, somanath sahoo a ?crit : >> Hi, >> >> As i am new to lttng usage, I have followed the instructions from >> in order to install the lttng >> tool set in my ubuntu 11.10. After successful installation, i followed >> the getting started link on lttng.org in >> "root mode". > > You may verify if kernel modules are loaded correctly. If not, then there > may be a problem while the installation. > > lsmod | grep lttng > > If not, you may try to load them and see if any error message occurs: > > sudo modprobe lttng_tracer > > Francis > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From toojays at toojays.net Mon Jun 18 08:10:42 2012 From: toojays at toojays.net (John Steele Scott) Date: Mon, 18 Jun 2012 21:40:42 +0930 Subject: [lttng-dev] lttng-ust with --std=c99 -pedantic In-Reply-To: <20120612155539.GA15859@Krystal> References: <20120612153257.GB14189@Krystal> <20120612155539.GA15859@Krystal> Message-ID: <4FDF1AC2.3070009@toojays.net> On 13/06/12 01:25, Mathieu Desnoyers wrote: > * Mathieu Desnoyers (mathieu.desnoyers at efficios.com) wrote: >> Hi John, >> >> * John Steele Scott (toojays at toojays.net) wrote: >>> I want to add lttng-ust tracepoints to a program which builds with "--std=c99 -pedantic". Right now this does not work. >>> >>> Using the demo program as an example, if you enable --std=c99, the first issue looks like: >>> >>> jscott at saaz:~/src/lttng-ust/tests/demo$ ccache gcc -std=c99 -DHAVE_CONFIG_H -I. -I../.. -I../../include/lttng -I../../include -Wall -g -O2 -MT demo.o -MD -MP -MF .deps/demo.Tpo -c -o demo.o demo.c >>> In file included from demo.c:34:0: >>> ust_tests_demo.h: In function ?__tracepoint_cb_ust_tests_demo___starting?: >>> ust_tests_demo.h:27:23: warning: implicit declaration of function ?typeof? [-Wimplicit-function-declaration] >>> ust_tests_demo.h:27:218: error: expected ?;? before ?_________p1? >>> ust_tests_demo.h:27:395: error: ?_________p1? undeclared (first use in this function) >>> ust_tests_demo.h:27:395: note: each undeclared identifier is reported only once for each function it appears in >>> In file included from demo.c:34:0: >>> ust_tests_demo.h: In function ?__tracepoint_cb_ust_tests_demo___done?: >>> ust_tests_demo.h:35:214: error: expected ?;? before ?_________p1? >>> ust_tests_demo.h:35:383: error: ?_________p1? undeclared (first use in this function) >>> In file included from demo.c:35:0: >>> ust_tests_demo2.h: In function ?__tracepoint_cb_ust_tests_demo2___loop?: >>> ust_tests_demo2.h:27:299: error: expected ?;? before ?_________p1? >>> ust_tests_demo2.h:27:470: error: ?_________p1? undeclared (first use in this function) >>> In file included from demo.c:36:0: >>> ust_tests_demo3.h: In function ?__tracepoint_cb_ust_tests_demo3___done?: >>> ust_tests_demo3.h:27:215: error: expected ?;? before ?_________p1? >>> ust_tests_demo3.h:27:386: error: ?_________p1? undeclared (first use in this function) >>> >>> This can be easily resolved by using __typeof__() instead of typeof(). Then I can build with --std=c99. But adding -pedantic still fails: >> I pushed a fix for this: >> >> commit 6423c3134bf07d4a7db56f69f2c79b540a79c4f1 >> Author: Mathieu Desnoyers >> Date: Mon Jun 11 10:15:25 2012 -0400 >> >> Fix c99 compatibility: use __typeof__ instead of typeof in public headers >> >> Reported-by: John Steele Scott >> Signed-off-by: Mathieu Desnoyers >> >> I also had issues with (void) arg parameters with tracepoints, so >> pushed: >> >> commit 4495dd39c05739d0fb2bc463b7c093d2459ce2b6 >> Author: Mathieu Desnoyers >> Date: Tue Jun 12 11:22:46 2012 -0400 >> >> Fix: support -std=c99 in tracepoint macros >> >> Signed-off-by: Mathieu Desnoyers >> >> I had also to fix userspace RCU library, with: >> >> >> commit e51500edbd9919cee53bc85cbb4b22cd4786fc42 >> Author: Mathieu Desnoyers >> Date: Tue Jun 12 11:24:31 2012 -0400 >> >> Fix c99 compatibility: use __asm__ and __volatile__ in public headers >> >> Signed-off-by: Mathieu Desnoyers >> >> commit bdffa73aa208ad5f1e5b3a3cb6cbf86ac6996559 >> Author: Mathieu Desnoyers >> Date: Mon Jun 11 10:16:35 2012 -0400 >> >> Fix c99 compatibility: use __typeof__ instead of typeof in public headers >> >> Reported-by: John Steele Scott >> Signed-off-by: Mathieu Desnoyers >> >> >>> jscott at saaz:~/src/lttng-ust/tests/demo$ ccache gcc -std=c99 -Dtypeof=__typeof__ -pedantic -DHAVE_CONFIG_H -I. -I../.. -I../../include/lttng -I../../include -Wall -g -O2 -MT demo.o -MD -MP -MF .deps/demo.Tpo -c -o demo.o demo.c >>> In file included from ust_tests_demo.h:25:0, >>> from demo.c:34: >>> ../../include/lttng/tracepoint.h: In function ?__tracepoints__init?: >>> ../../include/lttng/tracepoint.h:251:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] >>> ../../include/lttng/tracepoint.h:255:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] >>> ../../include/lttng/tracepoint.h:260:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] >>> ../../include/lttng/tracepoint.h:264:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] >>> ../../include/lttng/tracepoint.h:268:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] >>> In file included from demo.c:34:0: >>> ust_tests_demo.h: In function ?__tracepoint_cb_ust_tests_demo___starting?: >>> ust_tests_demo.h:27:161: warning: ISO C forbids braced-groups within expressions [-pedantic] >>> ust_tests_demo.h:27:549: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] >>> In file included from demo.c:34:0: >>> ust_tests_demo.h: In function ?__tracepoint_cb_ust_tests_demo___done?: >>> ust_tests_demo.h:35:161: warning: ISO C forbids braced-groups within expressions [-pedantic] >>> ust_tests_demo.h:35:537: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] >>> In file included from demo.c:35:0: >>> ust_tests_demo2.h: In function ?__tracepoint_cb_ust_tests_demo2___loop?: >>> ust_tests_demo2.h:27:245: warning: ISO C forbids braced-groups within expressions [-pedantic] >>> ust_tests_demo2.h:27:624: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] >>> In file included from demo.c:36:0: >>> ust_tests_demo3.h: In function ?__tracepoint_cb_ust_tests_demo3___done?: >>> ust_tests_demo3.h:27:161: warning: ISO C forbids braced-groups within expressions [-pedantic] >>> ust_tests_demo3.h:27:540: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] >> I see these warnings, the only thing that currently fails is due to >> BYTE_ORDER and BIG_ENDIAN not being defined. By adding: >> >> -DBYTE_ORDER=__BYTE_ORDER -DBIG_ENDIAN=__BIG_ENDIAN >> >> my tests/hello program, with a Makefile.am modified to do: >> >> hello_CFLAGS = -Werror=old-style-definition --std=c99 -pedantic >> >> prints many pedantic warnings, and fails with: >> >> ././ust_tests_hello.h:55:1: error: zero or negative size array ?__event_fields___ust_tests_hello___tptest_sighandler? >> >> which seems to be caused by my event with 0 fields, which try to create >> an array of length 0. >> >> I'll look into this one. > Please try again with lttng-ust master HEAD, userspace-rcu master HEAD, > and let me know if you experience problems compiling your instrumented > application with --std=c99 -pedantic. Please note that the probe object > (e.g. tp.c in the tests/hello program) needs to be compiled with gnu > extensions, so you should not use --std=c99 for this specific object. Mathieu, Thanks for your reply, sorry it has taken me so long to respond. I can now build this application with userspace tracing. I was actually able to compile the probe object with --std=c99 as well (this time on Centos 6.2, didn't try yet on Ubuntu). Would you expect any problems from this? I only have a single tracepoint in this app, but it does seem to work. The thing I forgot to mention is: okay, I can build now with -pedantic, but I can't build with "-pedantic -Werror". This application builds with "--std=c99 -pedantic -Werror" (among many other flags). I can of course remove -Werror for my tests, but in the longer term I would like to see lttng-ust enabled in our regular build, and disabling -Werror for that is not something we want to do. If I only had to disable it for the trace provider, I could negotiate that, but disabling it for any module which uses tracepoints is undesirable. I haven't yet looked at how difficult it would be do eliminate these pedantic warnings. Do you have a feel for what is involved? Preprocessor tricks warp my mind. :( cheers, John From mathieu.desnoyers at efficios.com Mon Jun 18 09:42:51 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Mon, 18 Jun 2012 09:42:51 -0400 Subject: [lttng-dev] lttng - ubuntu 11.10 - problem In-Reply-To: <1340002781.5676.YahooMailNeo@web161206.mail.bf1.yahoo.com> References: <1340002781.5676.YahooMailNeo@web161206.mail.bf1.yahoo.com> Message-ID: <20120618134251.GA6865@Krystal> * somanath sahoo (bapi_mvit2004 at yahoo.com) wrote: > Hi, > > I have tried to reinstall lttng-tools,? lttng-modules-dkms? and babeltrace. > > > the reinstallation of lttng-tools and babeltrace has been successful. But reinstallation of lttng-modules-dkms has been failing. the log is provided below > [...] > The content of make.log is provided below : > > > DKMS make.log for lttng-modules-2.0.0 for kernel 3.0.0-21-generic (i686) > Mon Jun 18 12:05:51 IST 2012 > make: Entering directory `/usr/src/linux-headers-3.0.0-21-generic' > ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-ring-buffer-client-discard.o > ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-ring-buffer-client-overwrite.o > ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-ring-buffer-metadata-client.o > ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-ring-buffer-client-mmap-discard.o > ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-ring-buffer-client-mmap-overwrite.o > ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-ring-buffer-metadata-mmap-client.o > ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-statedump-impl.o > ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/wrapper/irqdesc.o > ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-events.o > ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-abi.o > ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-probes.o > ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context.o > ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-pid.o > ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-procname.o > ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-prio.o > ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-nice.o > ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-vpid.o > ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-tid.o > ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-vtid.o > ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-ppid.o > ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-vppid.o > ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-calibrate.o > ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/wrapper/random.o > ? CC [M]? /var/lib/dkms/lttng-modules/2.0.0/build/lttng-syscalls.o > /var/lib/dkms/lttng-modules/2.0.0/build/lttng-syscalls.c:32:38: error: macro "is_compat_task" passed 1 arguments, but takes just 0 > /var/lib/dkms/lttng-modules/2.0.0/build/lttng-syscalls.c:33:1: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?{? token > make[1]: *** [/var/lib/dkms/lttng-modules/2.0.0/build/lttng-syscalls.o] Error 1 > make: *** [_module_/var/lib/dkms/lttng-modules/2.0.0/build] Error 2 > make: Leaving directory `/usr/src/linux-headers-3.0.0-21-generic' It looks like the lttng-modules DKMS package is out of date (2.0.0). Alex, can you update it ? Thanks, Mathieu > > ------------------------- End of make.log ----------------- > > As per the compilation errors in the file lttng-syscalls.c, i dont think these errors are valids. It may be due to somethings else. > > > Kindly guide how to come over this reinstallation problem of "lttng-modules-dkms" and please let me know if i need to do any other alternatives. > > > > > Thanks & Regards, > Somanath > > > ------------------------------ > > Message: 3 > Date: Fri, 15 Jun 2012 09:12:40 -0400 > From: David Goulet > To: Francis Giraldeau > Cc: "lttng-dev at lists.lttng.org" > Subject: Re: [lttng-dev] lttng - ubuntu 11.10 - problem > Message-ID: <4FDB34C8.7000505 at polymtl.ca> > Content-Type: text/plain; charset=ISO-8859-1 > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi, > > In order to trace the kernel, you need the LTTng kernel modules available in > the package "lttng-modules-dkms". So make sure you have these three packages > installed: > > # sudo apt-get install lttng-tools lttng-modules-dkms babeltrace > > The error message you got indicates that there is simply no LTTng modules > loaded. Please make sure they are loaded and if the problem persist, let us know! > > Thanks! > David > > On 15/06/12 04:16 AM, Francis Giraldeau wrote: > > Le 2012-06-15 09:41, somanath sahoo a ?crit : > >> Hi, > >> > >> As i am new to lttng usage, I have followed the instructions from > >> in order to install the lttng > >> tool set in my ubuntu 11.10. After successful installation, i followed > >> the getting started link on lttng.org in > >> "root mode". > > > > You may verify if kernel modules are loaded correctly. If not, then there > > may be a problem while the installation. > > > > lsmod | grep lttng > > > > If not, you may try to load them and see if any error message occurs: > > > > sudo modprobe lttng_tracer > > > > Francis > > > > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Mon Jun 18 10:11:56 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Mon, 18 Jun 2012 10:11:56 -0400 Subject: [lttng-dev] lttng-ust with --std=c99 -pedantic In-Reply-To: <4FDF1AC2.3070009@toojays.net> References: <20120612153257.GB14189@Krystal> <20120612155539.GA15859@Krystal> <4FDF1AC2.3070009@toojays.net> Message-ID: <20120618141156.GB6865@Krystal> * John Steele Scott (toojays at toojays.net) wrote: > On 13/06/12 01:25, Mathieu Desnoyers wrote: > > * Mathieu Desnoyers (mathieu.desnoyers at efficios.com) wrote: > >> Hi John, > >> > >> * John Steele Scott (toojays at toojays.net) wrote: > >>> I want to add lttng-ust tracepoints to a program which builds with "--std=c99 -pedantic". Right now this does not work. > >>> > >>> Using the demo program as an example, if you enable --std=c99, the first issue looks like: > >>> > >>> jscott at saaz:~/src/lttng-ust/tests/demo$ ccache gcc -std=c99 -DHAVE_CONFIG_H -I. -I../.. -I../../include/lttng -I../../include -Wall -g -O2 -MT demo.o -MD -MP -MF .deps/demo.Tpo -c -o demo.o demo.c > >>> In file included from demo.c:34:0: > >>> ust_tests_demo.h: In function ?__tracepoint_cb_ust_tests_demo___starting?: > >>> ust_tests_demo.h:27:23: warning: implicit declaration of function ?typeof? [-Wimplicit-function-declaration] > >>> ust_tests_demo.h:27:218: error: expected ?;? before ?_________p1? > >>> ust_tests_demo.h:27:395: error: ?_________p1? undeclared (first use in this function) > >>> ust_tests_demo.h:27:395: note: each undeclared identifier is reported only once for each function it appears in > >>> In file included from demo.c:34:0: > >>> ust_tests_demo.h: In function ?__tracepoint_cb_ust_tests_demo___done?: > >>> ust_tests_demo.h:35:214: error: expected ?;? before ?_________p1? > >>> ust_tests_demo.h:35:383: error: ?_________p1? undeclared (first use in this function) > >>> In file included from demo.c:35:0: > >>> ust_tests_demo2.h: In function ?__tracepoint_cb_ust_tests_demo2___loop?: > >>> ust_tests_demo2.h:27:299: error: expected ?;? before ?_________p1? > >>> ust_tests_demo2.h:27:470: error: ?_________p1? undeclared (first use in this function) > >>> In file included from demo.c:36:0: > >>> ust_tests_demo3.h: In function ?__tracepoint_cb_ust_tests_demo3___done?: > >>> ust_tests_demo3.h:27:215: error: expected ?;? before ?_________p1? > >>> ust_tests_demo3.h:27:386: error: ?_________p1? undeclared (first use in this function) > >>> > >>> This can be easily resolved by using __typeof__() instead of typeof(). Then I can build with --std=c99. But adding -pedantic still fails: > >> I pushed a fix for this: > >> > >> commit 6423c3134bf07d4a7db56f69f2c79b540a79c4f1 > >> Author: Mathieu Desnoyers > >> Date: Mon Jun 11 10:15:25 2012 -0400 > >> > >> Fix c99 compatibility: use __typeof__ instead of typeof in public headers > >> > >> Reported-by: John Steele Scott > >> Signed-off-by: Mathieu Desnoyers > >> > >> I also had issues with (void) arg parameters with tracepoints, so > >> pushed: > >> > >> commit 4495dd39c05739d0fb2bc463b7c093d2459ce2b6 > >> Author: Mathieu Desnoyers > >> Date: Tue Jun 12 11:22:46 2012 -0400 > >> > >> Fix: support -std=c99 in tracepoint macros > >> > >> Signed-off-by: Mathieu Desnoyers > >> > >> I had also to fix userspace RCU library, with: > >> > >> > >> commit e51500edbd9919cee53bc85cbb4b22cd4786fc42 > >> Author: Mathieu Desnoyers > >> Date: Tue Jun 12 11:24:31 2012 -0400 > >> > >> Fix c99 compatibility: use __asm__ and __volatile__ in public headers > >> > >> Signed-off-by: Mathieu Desnoyers > >> > >> commit bdffa73aa208ad5f1e5b3a3cb6cbf86ac6996559 > >> Author: Mathieu Desnoyers > >> Date: Mon Jun 11 10:16:35 2012 -0400 > >> > >> Fix c99 compatibility: use __typeof__ instead of typeof in public headers > >> > >> Reported-by: John Steele Scott > >> Signed-off-by: Mathieu Desnoyers > >> > >> > >>> jscott at saaz:~/src/lttng-ust/tests/demo$ ccache gcc -std=c99 -Dtypeof=__typeof__ -pedantic -DHAVE_CONFIG_H -I. -I../.. -I../../include/lttng -I../../include -Wall -g -O2 -MT demo.o -MD -MP -MF .deps/demo.Tpo -c -o demo.o demo.c > >>> In file included from ust_tests_demo.h:25:0, > >>> from demo.c:34: > >>> ../../include/lttng/tracepoint.h: In function ?__tracepoints__init?: > >>> ../../include/lttng/tracepoint.h:251:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > >>> ../../include/lttng/tracepoint.h:255:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > >>> ../../include/lttng/tracepoint.h:260:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > >>> ../../include/lttng/tracepoint.h:264:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > >>> ../../include/lttng/tracepoint.h:268:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > >>> In file included from demo.c:34:0: > >>> ust_tests_demo.h: In function ?__tracepoint_cb_ust_tests_demo___starting?: > >>> ust_tests_demo.h:27:161: warning: ISO C forbids braced-groups within expressions [-pedantic] > >>> ust_tests_demo.h:27:549: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > >>> In file included from demo.c:34:0: > >>> ust_tests_demo.h: In function ?__tracepoint_cb_ust_tests_demo___done?: > >>> ust_tests_demo.h:35:161: warning: ISO C forbids braced-groups within expressions [-pedantic] > >>> ust_tests_demo.h:35:537: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > >>> In file included from demo.c:35:0: > >>> ust_tests_demo2.h: In function ?__tracepoint_cb_ust_tests_demo2___loop?: > >>> ust_tests_demo2.h:27:245: warning: ISO C forbids braced-groups within expressions [-pedantic] > >>> ust_tests_demo2.h:27:624: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > >>> In file included from demo.c:36:0: > >>> ust_tests_demo3.h: In function ?__tracepoint_cb_ust_tests_demo3___done?: > >>> ust_tests_demo3.h:27:161: warning: ISO C forbids braced-groups within expressions [-pedantic] > >>> ust_tests_demo3.h:27:540: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > >> I see these warnings, the only thing that currently fails is due to > >> BYTE_ORDER and BIG_ENDIAN not being defined. By adding: > >> > >> -DBYTE_ORDER=__BYTE_ORDER -DBIG_ENDIAN=__BIG_ENDIAN > >> > >> my tests/hello program, with a Makefile.am modified to do: > >> > >> hello_CFLAGS = -Werror=old-style-definition --std=c99 -pedantic > >> > >> prints many pedantic warnings, and fails with: > >> > >> ././ust_tests_hello.h:55:1: error: zero or negative size array ?__event_fields___ust_tests_hello___tptest_sighandler? > >> > >> which seems to be caused by my event with 0 fields, which try to create > >> an array of length 0. > >> > >> I'll look into this one. > > Please try again with lttng-ust master HEAD, userspace-rcu master HEAD, > > and let me know if you experience problems compiling your instrumented > > application with --std=c99 -pedantic. Please note that the probe object > > (e.g. tp.c in the tests/hello program) needs to be compiled with gnu > > extensions, so you should not use --std=c99 for this specific object. > > Mathieu, > > Thanks for your reply, sorry it has taken me so long to respond. > > I can now build this application with userspace tracing. I was > actually able to compile the probe object with --std=c99 as well (this > time on Centos 6.2, didn't try yet on Ubuntu). Would you expect any > problems from this? I only have a single tracepoint in this app, but > it does seem to work. --std=c99 will not work to compile probes containing tracepoints taking 0 arguments. This is because c99 does not allow 0-sized arrays, and I don't want to complexify the probe generation to take care of this issue. > > The thing I forgot to mention is: okay, I can build now with > -pedantic, but I can't build with "-pedantic -Werror". This > application builds with "--std=c99 -pedantic -Werror" (among many > other flags). I can of course remove -Werror for my tests, but in the > longer term I would like to see lttng-ust enabled in our regular > build, and disabling -Werror for that is not something we want to do. > If I only had to disable it for the trace provider, I could negotiate > that, but disabling it for any module which uses tracepoints is > undesirable. > > I haven't yet looked at how difficult it would be do eliminate these > pedantic warnings. Do you have a feel for what is involved? > Preprocessor tricks warp my mind. :( I don't think we'll want to make the lttng probe module build under --std=c99 -pedandic -Werror, but I think we should focus on making sure the tracepoint part that is built within the application (with TRACEPOINT_CREATE_PROBES _not_ defined) builds fine in --std=c99 -pedantic. Currently, building a simple test program with tracepoints under --std=c99 -pedantic gets me: In file included from ust_tests_hello.h:25:0, from hello.c:34: ../../include/lttng/tracepoint.h: In function ?__tracepoints__init?: ../../include/lttng/tracepoint.h:261:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] ../../include/lttng/tracepoint.h:265:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] ../../include/lttng/tracepoint.h:270:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] ../../include/lttng/tracepoint.h:274:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] ../../include/lttng/tracepoint.h:278:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] ---> see include/lttng/tracepoint.h __tracepoints__init(): ----> This is caused by use of dlsym() to lookup a function pointer from a symbol. dlsym() returns "void *", and we cast it into function pointer type. Ideas on how to make this c99 pedantic compliant are welcome. ------------------ In file included from hello.c:34:0: ust_tests_hello.h: In function ?__tracepoint_cb_ust_tests_hello___tptest?: ust_tests_hello.h:28:1: warning: ISO C forbids braced-groups within expressions [-pedantic] ust_tests_hello.h:28:1: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] ust_tests_hello.h: In function ?__tracepoint_cb_ust_tests_hello___tptest_sighandler?: ust_tests_hello.h:52:1: warning: ISO C forbids braced-groups within expressions [-pedantic] ust_tests_hello.h:52:1: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] ---> see include/lttng/tracepoint.h #define _DECLARE_TRACEPOINT(_provider, _name, ...) \ extern struct tracepoint __tracepoint_##_provider##___##_name; \ static inline void __tracepoint_cb_##_provider##___##_name(_TP_ARGS_PROTO(__VA_ARGS__)) \ [...] ----> this is caused by use of "tp_rcu_dereference_bp()", which is defined in include/lttng/tracepoint-rcu.h (in the #else clause) : #define tp_rcu_dereference_bp(p) \ ({ \ __typeof__(p) _________p1 = URCU_FORCE_CAST(__typeof__(p), \ tracepoint_dlopen.rcu_dereference_sym_bp(URCU_FORCE_CAST(void *, p))); \ (_________p1); \ }) For this one, we should be able to change it into a single-expression and remove the braced-groups, since we are just really evaluating a single expression here. Fixed by the following commit in lttng-ust master: commit a4eaf8eabe829be8f7d7432ffaf83291a068b0ed Author: Mathieu Desnoyers Date: Mon Jun 18 10:10:36 2012 -0400 Fix c99 compatibility: tp_rcu_dereference_bp() should not use braced-groups within expressions ------------------ hello.c: In function ?inthandler?: hello.c:39:47: warning: ISO C99 requires rest arguments to be used [enabled by default] ---> see include/lttng/tracepoint.h tracepoint() macro. ----> this is caused by not passing any argument to the tracepoint. Ideas on how to fix this are welcome, as I don't see any way to do it in C99-pedantic. Thanks! Mathieu > > cheers, > > John > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From alexandre.montplaisir at polymtl.ca Mon Jun 18 10:35:07 2012 From: alexandre.montplaisir at polymtl.ca (Alexandre Montplaisir) Date: Mon, 18 Jun 2012 10:35:07 -0400 Subject: [lttng-dev] lttng - ubuntu 11.10 - problem In-Reply-To: <20120618134251.GA6865@Krystal> References: <1340002781.5676.YahooMailNeo@web161206.mail.bf1.yahoo.com> <20120618134251.GA6865@Krystal> Message-ID: <4FDF3C9B.3020204@polymtl.ca> On 12-06-18 09:42 AM, Mathieu Desnoyers wrote: > * somanath sahoo (bapi_mvit2004 at yahoo.com) wrote: >> Hi, >> >> I have tried to reinstall lttng-tools, lttng-modules-dkms and babeltrace. >> >> >> the reinstallation of lttng-tools and babeltrace has been successful. But reinstallation of lttng-modules-dkms has been failing. the log is provided below >> > [...] >> The content of make.log is provided below : >> >> >> DKMS make.log for lttng-modules-2.0.0 for kernel 3.0.0-21-generic (i686) >> Mon Jun 18 12:05:51 IST 2012 >> make: Entering directory `/usr/src/linux-headers-3.0.0-21-generic' >> CC [M] /var/lib/dkms/lttng-modules/2.0.0/build/lttng-ring-buffer-client-discard.o >> CC [M] /var/lib/dkms/lttng-modules/2.0.0/build/lttng-ring-buffer-client-overwrite.o >> CC [M] /var/lib/dkms/lttng-modules/2.0.0/build/lttng-ring-buffer-metadata-client.o >> CC [M] /var/lib/dkms/lttng-modules/2.0.0/build/lttng-ring-buffer-client-mmap-discard.o >> CC [M] /var/lib/dkms/lttng-modules/2.0.0/build/lttng-ring-buffer-client-mmap-overwrite.o >> CC [M] /var/lib/dkms/lttng-modules/2.0.0/build/lttng-ring-buffer-metadata-mmap-client.o >> CC [M] /var/lib/dkms/lttng-modules/2.0.0/build/lttng-statedump-impl.o >> CC [M] /var/lib/dkms/lttng-modules/2.0.0/build/wrapper/irqdesc.o >> CC [M] /var/lib/dkms/lttng-modules/2.0.0/build/lttng-events.o >> CC [M] /var/lib/dkms/lttng-modules/2.0.0/build/lttng-abi.o >> CC [M] /var/lib/dkms/lttng-modules/2.0.0/build/lttng-probes.o >> CC [M] /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context.o >> CC [M] /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-pid.o >> CC [M] /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-procname.o >> CC [M] /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-prio.o >> CC [M] /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-nice.o >> CC [M] /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-vpid.o >> CC [M] /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-tid.o >> CC [M] /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-vtid.o >> CC [M] /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-ppid.o >> CC [M] /var/lib/dkms/lttng-modules/2.0.0/build/lttng-context-vppid.o >> CC [M] /var/lib/dkms/lttng-modules/2.0.0/build/lttng-calibrate.o >> CC [M] /var/lib/dkms/lttng-modules/2.0.0/build/wrapper/random.o >> CC [M] /var/lib/dkms/lttng-modules/2.0.0/build/lttng-syscalls.o >> /var/lib/dkms/lttng-modules/2.0.0/build/lttng-syscalls.c:32:38: error: macro "is_compat_task" passed 1 arguments, but takes just 0 >> /var/lib/dkms/lttng-modules/2.0.0/build/lttng-syscalls.c:33:1: error: expected ?=?, ?,?, ?;?, ?asm? or ?__attribute__? before ?{? token >> make[1]: *** [/var/lib/dkms/lttng-modules/2.0.0/build/lttng-syscalls.o] Error 1 >> make: *** [_module_/var/lib/dkms/lttng-modules/2.0.0/build] Error 2 >> make: Leaving directory `/usr/src/linux-headers-3.0.0-21-generic' > It looks like the lttng-modules DKMS package is out of date (2.0.0). > Alex, can you update it ? Yeah, we stopped bothering about the Oneiric packages since Precise came out with lttng-modules included ;) Still it's strange, because 2.0.0 used to install fine on Oneiric... I'll try to find some time today to push an update, and see if it helps. Cheers, -- Alexandre Montplaisir DORSAL lab, ?cole Polytechnique de Montr?al From Zheng.Chang at emc.com Mon Jun 18 23:50:04 2012 From: Zheng.Chang at emc.com (Zheng.Chang at emc.com) Date: Mon, 18 Jun 2012 23:50:04 -0400 Subject: [lttng-dev] Some questions about Lttng Message-ID: <6539770C71C3814BB0BFC2DBEBD105087948C2@CORPUSMX30B.corp.emc.com> Hi folks, I'm studying how to use Lttng now. I built a kernel which version is 2.6.38 and ran with lttng 2.0. I got some confused when I started to use it. Here are my questions: 1. I didn't see kernel patches for kernel 3.x. Does it mean kernel 3.x support it already? 2. I tried to do something like, dump the arguments of system call, or dump a backtrace in a specified function. But the output of lttng is very limited. Is there a way to do that with lttng? 3. I looked into some UST examples and found here are three header files: tracepoint.h, tracepoint-event.h and ust-tracepoint-event.h. They have some duplicated macro definitions like TRACEPOINT_EVENT. And the examples includes all of these three header files despite no conflict here. Could someone help to explain the intention? 4. Once I defined a tracepoint in my code, seems some initializations would register default probe into the hook point. How to disable the default probe and register my self-defined probes? 5. Does lttng-ust support dynamic traceing like kprobe? I'm a newbie of lttng. Any help will be appreciated. Thanks Zheng -------------- next part -------------- An HTML attachment was scrubbed... URL: From francis.giraldeau at gmail.com Tue Jun 19 05:28:01 2012 From: francis.giraldeau at gmail.com (Francis Giraldeau) Date: Tue, 19 Jun 2012 11:28:01 +0200 Subject: [lttng-dev] Some questions about Lttng In-Reply-To: <6539770C71C3814BB0BFC2DBEBD105087948C2@CORPUSMX30B.corp.emc.com> References: <6539770C71C3814BB0BFC2DBEBD105087948C2@CORPUSMX30B.corp.emc.com> Message-ID: <4FE04621.3070806@gmail.com> Le 2012-06-19 05:50, Zheng.Chang at emc.com a ?crit : > > Hi folks, > > > > I'm studying how to use Lttng now. I built a kernel which version is > 2.6.38 and ran with lttng 2.0. > > I got some confused when I started to use it. Here are my questions: > > > > 1. I didn't see kernel patches for kernel 3.x. Does it mean kernel 3.x > support it already? > You don't need a kernel patch with lttng 2.0, only modules are required. It uses the tracepoints already present in the kernel, and trace system calls, and uses perf performance counters and kprobes. > 2. I tried to do something like, dump the arguments of system call, or > dump a backtrace in a specified function. But the output of lttng is > very limited. Is there a way to do that with lttng? > If system calls are enabled, arguments are supposed to be dumped (option --syscall to lttng enable-event). If it's not the case, then are you sure you are using lttng 2.0 and not 0.12? ;) Because the older version has a different behavior. One additional note: few system calls do not have all their arguments decoded in lttng 2, but there are only a few. > 3. I looked into some UST examples and found here are three header > files: tracepoint.h, tracepoint-event.h and ust-tracepoint-event.h. > They have some duplicated macro definitions like TRACEPOINT_EVENT. > The macros are quite complicated. Some includes files are included more than once to generate various portion of the tracepoint code. So, my advice here is to take a working example and adapt it to your needs. > And the examples includes all of these three header files despite no > conflict here. Could someone help to explain the intention? > > 4. Once I defined a tracepoint in my code, seems some initializations > would register default probe into the hook point. How to disable the > default probe and register my self-defined probes? > You mean, call a custom function when tracepoint is executed? Maybe somebody else has an answer, but AFAIK this is not possible. But you could make a wrapper to your tracepoint and then call your additional function there. > 5. Does lttng-ust support dynamic traceing like kprobe? > AFAIK, the kernel tracer supports kprobe, but not UST. Maybe somebody else can confirm/infirm the dynamic tracepoint feature in user-space? You can use a feature of GCC to regiter callback on function entry and exit, but since it executes for all functions, then the overhead is very high. You can have a look here: https://github.com/giraldeau/workload-kit/blob/master/ust/cyg.c Cheers, Francis -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4476 bytes Desc: Signature cryptographique S/MIME URL: From mathieu.desnoyers at efficios.com Tue Jun 19 12:36:55 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 19 Jun 2012 12:36:55 -0400 Subject: [lttng-dev] Some questions about Lttng In-Reply-To: <4FE04621.3070806@gmail.com> References: <6539770C71C3814BB0BFC2DBEBD105087948C2@CORPUSMX30B.corp.emc.com> <4FE04621.3070806@gmail.com> Message-ID: <20120619163655.GC6743@Krystal> * Francis Giraldeau (francis.giraldeau at gmail.com) wrote: > Le 2012-06-19 05:50, Zheng.Chang at emc.com a ?crit : > > > > Hi folks, > > > > > > > > I'm studying how to use Lttng now. I built a kernel which version is > > 2.6.38 and ran with lttng 2.0. > > > > I got some confused when I started to use it. Here are my questions: > > > > > > > > 1. I didn't see kernel patches for kernel 3.x. Does it mean kernel 3.x > > support it already? > > > > You don't need a kernel patch with lttng 2.0, only modules are required. > It uses the tracepoints already present in the kernel, and trace system > calls, and uses perf performance counters and kprobes. > > > 2. I tried to do something like, dump the arguments of system call, or > > dump a backtrace in a specified function. But the output of lttng is > > very limited. Is there a way to do that with lttng? > > > > If system calls are enabled, arguments are supposed to be dumped (option > --syscall to lttng enable-event). If it's not the case, then are you > sure you are using lttng 2.0 and not 0.12? ;) Because the older version > has a different behavior. One additional note: few system calls do not > have all their arguments decoded in lttng 2, but there are only a few. There is no backtrace dump feature in lttng 2.0. Arguments of system calls are almost all there on x86 32/64 and ARM. What architecture are you using ? > > > 3. I looked into some UST examples and found here are three header > > files: tracepoint.h, tracepoint-event.h and ust-tracepoint-event.h. > > They have some duplicated macro definitions like TRACEPOINT_EVENT. > > > > The macros are quite complicated. Some includes files are included more > than once to generate various portion of the tracepoint code. So, my > advice here is to take a working example and adapt it to your needs. Good advice. > > > And the examples includes all of these three header files despite no > > conflict here. Could someone help to explain the intention? > > > > 4. Once I defined a tracepoint in my code, seems some initializations > > would register default probe into the hook point. How to disable the > > default probe and register my self-defined probes? > > > > You mean, call a custom function when tracepoint is executed? Maybe > somebody else has an answer, but AFAIK this is not possible. But you > could make a wrapper to your tracepoint and then call your additional > function there. Yep, not possible. You'd have to wrap the tracepoint. > > > 5. Does lttng-ust support dynamic traceing like kprobe? > > > > AFAIK, the kernel tracer supports kprobe, but not UST. Maybe somebody > else can confirm/infirm the dynamic tracepoint feature in user-space? This is correct. Thanks, Mathieu > > You can use a feature of GCC to regiter callback on function entry and > exit, but since it executes for all functions, then the overhead is very > high. You can have a look here: > > https://github.com/giraldeau/workload-kit/blob/master/ust/cyg.c > > Cheers, > > Francis > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Tue Jun 19 13:45:18 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 19 Jun 2012 13:45:18 -0400 Subject: [lttng-dev] lttng-ust with --std=c99 -pedantic In-Reply-To: <20120618141156.GB6865@Krystal> References: <20120612153257.GB14189@Krystal> <20120612155539.GA15859@Krystal> <4FDF1AC2.3070009@toojays.net> <20120618141156.GB6865@Krystal> Message-ID: <20120619174518.GB8776@Krystal> * Mathieu Desnoyers (mathieu.desnoyers at efficios.com) wrote: > * John Steele Scott (toojays at toojays.net) wrote: > > On 13/06/12 01:25, Mathieu Desnoyers wrote: > > > * Mathieu Desnoyers (mathieu.desnoyers at efficios.com) wrote: > > >> Hi John, > > >> > > >> * John Steele Scott (toojays at toojays.net) wrote: > > >>> I want to add lttng-ust tracepoints to a program which builds with "--std=c99 -pedantic". Right now this does not work. > > >>> > > >>> Using the demo program as an example, if you enable --std=c99, the first issue looks like: > > >>> > > >>> jscott at saaz:~/src/lttng-ust/tests/demo$ ccache gcc -std=c99 -DHAVE_CONFIG_H -I. -I../.. -I../../include/lttng -I../../include -Wall -g -O2 -MT demo.o -MD -MP -MF .deps/demo.Tpo -c -o demo.o demo.c > > >>> In file included from demo.c:34:0: > > >>> ust_tests_demo.h: In function ?__tracepoint_cb_ust_tests_demo___starting?: > > >>> ust_tests_demo.h:27:23: warning: implicit declaration of function ?typeof? [-Wimplicit-function-declaration] > > >>> ust_tests_demo.h:27:218: error: expected ?;? before ?_________p1? > > >>> ust_tests_demo.h:27:395: error: ?_________p1? undeclared (first use in this function) > > >>> ust_tests_demo.h:27:395: note: each undeclared identifier is reported only once for each function it appears in > > >>> In file included from demo.c:34:0: > > >>> ust_tests_demo.h: In function ?__tracepoint_cb_ust_tests_demo___done?: > > >>> ust_tests_demo.h:35:214: error: expected ?;? before ?_________p1? > > >>> ust_tests_demo.h:35:383: error: ?_________p1? undeclared (first use in this function) > > >>> In file included from demo.c:35:0: > > >>> ust_tests_demo2.h: In function ?__tracepoint_cb_ust_tests_demo2___loop?: > > >>> ust_tests_demo2.h:27:299: error: expected ?;? before ?_________p1? > > >>> ust_tests_demo2.h:27:470: error: ?_________p1? undeclared (first use in this function) > > >>> In file included from demo.c:36:0: > > >>> ust_tests_demo3.h: In function ?__tracepoint_cb_ust_tests_demo3___done?: > > >>> ust_tests_demo3.h:27:215: error: expected ?;? before ?_________p1? > > >>> ust_tests_demo3.h:27:386: error: ?_________p1? undeclared (first use in this function) > > >>> > > >>> This can be easily resolved by using __typeof__() instead of typeof(). Then I can build with --std=c99. But adding -pedantic still fails: > > >> I pushed a fix for this: > > >> > > >> commit 6423c3134bf07d4a7db56f69f2c79b540a79c4f1 > > >> Author: Mathieu Desnoyers > > >> Date: Mon Jun 11 10:15:25 2012 -0400 > > >> > > >> Fix c99 compatibility: use __typeof__ instead of typeof in public headers > > >> > > >> Reported-by: John Steele Scott > > >> Signed-off-by: Mathieu Desnoyers > > >> > > >> I also had issues with (void) arg parameters with tracepoints, so > > >> pushed: > > >> > > >> commit 4495dd39c05739d0fb2bc463b7c093d2459ce2b6 > > >> Author: Mathieu Desnoyers > > >> Date: Tue Jun 12 11:22:46 2012 -0400 > > >> > > >> Fix: support -std=c99 in tracepoint macros > > >> > > >> Signed-off-by: Mathieu Desnoyers > > >> > > >> I had also to fix userspace RCU library, with: > > >> > > >> > > >> commit e51500edbd9919cee53bc85cbb4b22cd4786fc42 > > >> Author: Mathieu Desnoyers > > >> Date: Tue Jun 12 11:24:31 2012 -0400 > > >> > > >> Fix c99 compatibility: use __asm__ and __volatile__ in public headers > > >> > > >> Signed-off-by: Mathieu Desnoyers > > >> > > >> commit bdffa73aa208ad5f1e5b3a3cb6cbf86ac6996559 > > >> Author: Mathieu Desnoyers > > >> Date: Mon Jun 11 10:16:35 2012 -0400 > > >> > > >> Fix c99 compatibility: use __typeof__ instead of typeof in public headers > > >> > > >> Reported-by: John Steele Scott > > >> Signed-off-by: Mathieu Desnoyers > > >> > > >> > > >>> jscott at saaz:~/src/lttng-ust/tests/demo$ ccache gcc -std=c99 -Dtypeof=__typeof__ -pedantic -DHAVE_CONFIG_H -I. -I../.. -I../../include/lttng -I../../include -Wall -g -O2 -MT demo.o -MD -MP -MF .deps/demo.Tpo -c -o demo.o demo.c > > >>> In file included from ust_tests_demo.h:25:0, > > >>> from demo.c:34: > > >>> ../../include/lttng/tracepoint.h: In function ?__tracepoints__init?: > > >>> ../../include/lttng/tracepoint.h:251:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > > >>> ../../include/lttng/tracepoint.h:255:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > > >>> ../../include/lttng/tracepoint.h:260:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > > >>> ../../include/lttng/tracepoint.h:264:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > > >>> ../../include/lttng/tracepoint.h:268:4: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > > >>> In file included from demo.c:34:0: > > >>> ust_tests_demo.h: In function ?__tracepoint_cb_ust_tests_demo___starting?: > > >>> ust_tests_demo.h:27:161: warning: ISO C forbids braced-groups within expressions [-pedantic] > > >>> ust_tests_demo.h:27:549: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > > >>> In file included from demo.c:34:0: > > >>> ust_tests_demo.h: In function ?__tracepoint_cb_ust_tests_demo___done?: > > >>> ust_tests_demo.h:35:161: warning: ISO C forbids braced-groups within expressions [-pedantic] > > >>> ust_tests_demo.h:35:537: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > > >>> In file included from demo.c:35:0: > > >>> ust_tests_demo2.h: In function ?__tracepoint_cb_ust_tests_demo2___loop?: > > >>> ust_tests_demo2.h:27:245: warning: ISO C forbids braced-groups within expressions [-pedantic] > > >>> ust_tests_demo2.h:27:624: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > > >>> In file included from demo.c:36:0: > > >>> ust_tests_demo3.h: In function ?__tracepoint_cb_ust_tests_demo3___done?: > > >>> ust_tests_demo3.h:27:161: warning: ISO C forbids braced-groups within expressions [-pedantic] > > >>> ust_tests_demo3.h:27:540: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > > >> I see these warnings, the only thing that currently fails is due to > > >> BYTE_ORDER and BIG_ENDIAN not being defined. By adding: > > >> > > >> -DBYTE_ORDER=__BYTE_ORDER -DBIG_ENDIAN=__BIG_ENDIAN > > >> > > >> my tests/hello program, with a Makefile.am modified to do: > > >> > > >> hello_CFLAGS = -Werror=old-style-definition --std=c99 -pedantic > > >> > > >> prints many pedantic warnings, and fails with: > > >> > > >> ././ust_tests_hello.h:55:1: error: zero or negative size array ?__event_fields___ust_tests_hello___tptest_sighandler? > > >> > > >> which seems to be caused by my event with 0 fields, which try to create > > >> an array of length 0. > > >> > > >> I'll look into this one. > > > Please try again with lttng-ust master HEAD, userspace-rcu master HEAD, > > > and let me know if you experience problems compiling your instrumented > > > application with --std=c99 -pedantic. Please note that the probe object > > > (e.g. tp.c in the tests/hello program) needs to be compiled with gnu > > > extensions, so you should not use --std=c99 for this specific object. > > > > Mathieu, > > > > Thanks for your reply, sorry it has taken me so long to respond. > > > > I can now build this application with userspace tracing. I was > > actually able to compile the probe object with --std=c99 as well (this > > time on Centos 6.2, didn't try yet on Ubuntu). Would you expect any > > problems from this? I only have a single tracepoint in this app, but > > it does seem to work. > > --std=c99 will not work to compile probes containing tracepoints taking > 0 arguments. This is because c99 does not allow 0-sized arrays, and I > don't want to complexify the probe generation to take care of this > issue. > > > > > The thing I forgot to mention is: okay, I can build now with > > -pedantic, but I can't build with "-pedantic -Werror". This > > application builds with "--std=c99 -pedantic -Werror" (among many > > other flags). I can of course remove -Werror for my tests, but in the > > longer term I would like to see lttng-ust enabled in our regular > > build, and disabling -Werror for that is not something we want to do. > > If I only had to disable it for the trace provider, I could negotiate > > that, but disabling it for any module which uses tracepoints is > > undesirable. > > > > I haven't yet looked at how difficult it would be do eliminate these > > pedantic warnings. Do you have a feel for what is involved? > > Preprocessor tricks warp my mind. :( > > I don't think we'll want to make the lttng probe module build under > --std=c99 -pedandic -Werror, but I think we should focus on making sure > the tracepoint part that is built within the application (with > TRACEPOINT_CREATE_PROBES _not_ defined) builds fine in --std=c99 > -pedantic. > > Currently, building a simple test program with tracepoints under > --std=c99 -pedantic gets me: > > > In file included from ust_tests_hello.h:25:0, > from hello.c:34: > ../../include/lttng/tracepoint.h: In function ?__tracepoints__init?: > ../../include/lttng/tracepoint.h:261:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > ../../include/lttng/tracepoint.h:265:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > ../../include/lttng/tracepoint.h:270:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > ../../include/lttng/tracepoint.h:274:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > ../../include/lttng/tracepoint.h:278:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > > ---> see include/lttng/tracepoint.h __tracepoints__init(): > ----> This is caused by use of dlsym() to lookup a function pointer from > a symbol. dlsym() returns "void *", and we cast it into function > pointer type. Ideas on how to make this c99 pedantic compliant are > welcome. > > ------------------ > > In file included from hello.c:34:0: > ust_tests_hello.h: In function ?__tracepoint_cb_ust_tests_hello___tptest?: > ust_tests_hello.h:28:1: warning: ISO C forbids braced-groups within expressions [-pedantic] > ust_tests_hello.h:28:1: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > ust_tests_hello.h: In function ?__tracepoint_cb_ust_tests_hello___tptest_sighandler?: > ust_tests_hello.h:52:1: warning: ISO C forbids braced-groups within expressions [-pedantic] > ust_tests_hello.h:52:1: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > > ---> see include/lttng/tracepoint.h > > #define _DECLARE_TRACEPOINT(_provider, _name, ...) \ > extern struct tracepoint __tracepoint_##_provider##___##_name; \ > static inline void __tracepoint_cb_##_provider##___##_name(_TP_ARGS_PROTO(__VA_ARGS__)) \ > [...] > > ----> this is caused by use of "tp_rcu_dereference_bp()", which is > defined in include/lttng/tracepoint-rcu.h (in the #else clause) : > > #define tp_rcu_dereference_bp(p) \ > ({ \ > __typeof__(p) _________p1 = URCU_FORCE_CAST(__typeof__(p), \ > tracepoint_dlopen.rcu_dereference_sym_bp(URCU_FORCE_CAST(void *, p))); \ > (_________p1); \ > }) > > For this one, we should be able to change it into a single-expression > and remove the braced-groups, since we are just really evaluating a > single expression here. Fixed by the following commit in lttng-ust > master: > > commit a4eaf8eabe829be8f7d7432ffaf83291a068b0ed > Author: Mathieu Desnoyers > Date: Mon Jun 18 10:10:36 2012 -0400 > > Fix c99 compatibility: tp_rcu_dereference_bp() should not use braced-groups within expressions > > ------------------ > > hello.c: In function ?inthandler?: > hello.c:39:47: warning: ISO C99 requires rest arguments to be used [enabled by default] > ---> see include/lttng/tracepoint.h tracepoint() macro. > ----> this is caused by not passing any argument to the tracepoint. > Ideas on how to fix this are welcome, as I don't see any way to do it in > C99-pedantic. I think I found a way to fix the last pedantic warnings. Can you try with this master branch commit ? commit fbdeb5ecb8ff9b7d73a72de9fc66d07f7797d93f Author: Mathieu Desnoyers Date: Tue Jun 19 13:45:05 2012 -0400 Fix C99 strict compatibility: don't use void * for function pointers compiling public headers with --std=x99 -pedantic shows: warning: ISO C forbids conversion of object pointer to function pointer type Use "void (*func)(void)" to represent a generic function pointer rather than "void *". Signed-off-by: Mathieu Desnoyers > > Thanks! > > Mathieu > > > > > > > cheers, > > > > John > > > > _______________________________________________ > > lttng-dev mailing list > > lttng-dev at lists.lttng.org > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > -- > Mathieu Desnoyers > Operating System Efficiency R&D Consultant > EfficiOS Inc. > http://www.efficios.com > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From Zheng.Chang at emc.com Wed Jun 20 01:30:56 2012 From: Zheng.Chang at emc.com (Zheng.Chang at emc.com) Date: Wed, 20 Jun 2012 01:30:56 -0400 Subject: [lttng-dev] Some questions about Lttng In-Reply-To: <20120619163655.GC6743@Krystal> References: <6539770C71C3814BB0BFC2DBEBD105087948C2@CORPUSMX30B.corp.emc.com><4FE04621.3070806@gmail.com> <20120619163655.GC6743@Krystal> Message-ID: <6539770C71C3814BB0BFC2DBEBD105087FC418@CORPUSMX30B.corp.emc.com> > -----Original Message----- > From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] > Sent: Wednesday, June 20, 2012 0:37 AM > To: Francis Giraldeau > Cc: lttng-dev at lists.lttng.org > Subject: Re: [lttng-dev] Some questions about Lttng > * Francis Giraldeau (francis.giraldeau at gmail.com) wrote: > > Le 2012-06-19 05:50, Zheng.Chang at emc.com a ?crit : > > > > > > Hi folks, > > > > > > > > > > > > I'm studying how to use Lttng now. I built a kernel which version is > > > 2.6.38 and ran with lttng 2.0. > > > > > > I got some confused when I started to use it. Here are my questions: > > > > > > > > > > > > 1. I didn't see kernel patches for kernel 3.x. Does it mean kernel 3.x > > > support it already? > > > > > > > You don't need a kernel patch with lttng 2.0, only modules are required. > > It uses the tracepoints already present in the kernel, and trace system > > calls, and uses perf performance counters and kprobes. > > > > > 2. I tried to do something like, dump the arguments of system call, or > > > dump a backtrace in a specified function. But the output of lttng is > > > very limited. Is there a way to do that with lttng? > > > > > > > If system calls are enabled, arguments are supposed to be dumped (option > > --syscall to lttng enable-event). If it's not the case, then are you > > sure you are using lttng 2.0 and not 0.12? ;) Because the older version > > has a different behavior. One additional note: few system calls do not > > have all their arguments decoded in lttng 2, but there are only a few. > There is no backtrace dump feature in lttng 2.0. > Arguments of system calls are almost all there on x86 32/64 and ARM. > What architecture are you using ? My tsetbed is x86 32bit and lttng's version is 2.0.1. You're right. I can see the arguments now. > > > > > 3. I looked into some UST examples and found here are three header > > > files: tracepoint.h, tracepoint-event.h and ust-tracepoint-event.h. > > > They have some duplicated macro definitions like TRACEPOINT_EVENT. > > > > > > > The macros are quite complicated. Some includes files are included more > > than once to generate various portion of the tracepoint code. So, my > > advice here is to take a working example and adapt it to your needs. > Good advice. > > > > > And the examples includes all of these three header files despite no > > > conflict here. Could someone help to explain the intention? > > > > > > 4. Once I defined a tracepoint in my code, seems some initializations > > > would register default probe into the hook point. How to disable the > > > default probe and register my self-defined probes? > > > > > > > You mean, call a custom function when tracepoint is executed? Maybe > > somebody else has an answer, but AFAIK this is not possible. But you > > could make a wrapper to your tracepoint and then call your additional > > function there. > Yep, not possible. You'd have to wrap the tracepoint. > > > > > 5. Does lttng-ust support dynamic traceing like kprobe? > > > > > > > AFAIK, the kernel tracer supports kprobe, but not UST. Maybe somebody > > else can confirm/infirm the dynamic tracepoint feature in user-space? > This is correct. > Thanks, > Mathieu > > > > You can use a feature of GCC to regiter callback on function entry and > > exit, but since it executes for all functions, then the overhead is very > > high. You can have a look here: > > > > https://github.com/giraldeau/workload-kit/blob/master/ust/cyg.c > > > > Cheers, > > > > Francis And here are some subsequent questions about lttng: 6. Does lttng-ust support marker? Marker is easier to be compatible with the APIs like printf if we don't care its performance issue. 7. What's supposed to show with 'lttng list -u'? It's empty now. Is that possible to show the events defined in an application? 8. What does disable-event command of lttng do? With the example(hello) provided by lttng-ust, I enabled all events with '-a -u' and then disabled them again. I launched the example with gdb and dumped the tracepoint's structure and then found its state was active. It's supposed to be inactive here, right? BTW: I didn't see any trace generated here with view command. Thanks all for your useful info! Best regards Zheng > > _______________________________________________ > > lttng-dev mailing list > > lttng-dev at lists.lttng.org > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > -- > Mathieu Desnoyers > Operating System Efficiency R&D Consultant > EfficiOS Inc. > http://www.efficios.com > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev From mathieu.desnoyers at efficios.com Wed Jun 20 09:03:38 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Wed, 20 Jun 2012 09:03:38 -0400 Subject: [lttng-dev] Some questions about Lttng In-Reply-To: <6539770C71C3814BB0BFC2DBEBD105087FC418@CORPUSMX30B.corp.emc.com> References: <20120619163655.GC6743@Krystal> <6539770C71C3814BB0BFC2DBEBD105087FC418@CORPUSMX30B.corp.emc.com> Message-ID: <20120620130338.GA25746@Krystal> * Zheng.Chang at emc.com (Zheng.Chang at emc.com) wrote: > > > -----Original Message----- > > From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] > > Sent: Wednesday, June 20, 2012 0:37 AM > > To: Francis Giraldeau > > Cc: lttng-dev at lists.lttng.org > > Subject: Re: [lttng-dev] Some questions about Lttng > > > * Francis Giraldeau (francis.giraldeau at gmail.com) wrote: > > > Le 2012-06-19 05:50, Zheng.Chang at emc.com a ?crit : > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > I'm studying how to use Lttng now. I built a kernel which version is > > > > 2.6.38 and ran with lttng 2.0. > > > > > > > > I got some confused when I started to use it. Here are my questions: > > > > > > > > > > > > > > > > 1. I didn't see kernel patches for kernel 3.x. Does it mean kernel 3.x > > > > support it already? > > > > > > > > > > You don't need a kernel patch with lttng 2.0, only modules are required. > > > It uses the tracepoints already present in the kernel, and trace system > > > calls, and uses perf performance counters and kprobes. > > > > > > > 2. I tried to do something like, dump the arguments of system call, or > > > > dump a backtrace in a specified function. But the output of lttng is > > > > very limited. Is there a way to do that with lttng? > > > > > > > > > > If system calls are enabled, arguments are supposed to be dumped (option > > > --syscall to lttng enable-event). If it's not the case, then are you > > > sure you are using lttng 2.0 and not 0.12? ;) Because the older version > > > has a different behavior. One additional note: few system calls do not > > > have all their arguments decoded in lttng 2, but there are only a few. > > > There is no backtrace dump feature in lttng 2.0. > > > Arguments of system calls are almost all there on x86 32/64 and ARM. > > What architecture are you using ? > > > My tsetbed is x86 32bit and lttng's version is 2.0.1. You're right. I can see the arguments now. > > > > > > > > > 3. I looked into some UST examples and found here are three header > > > > files: tracepoint.h, tracepoint-event.h and ust-tracepoint-event.h. > > > > They have some duplicated macro definitions like TRACEPOINT_EVENT. > > > > > > > > > > The macros are quite complicated. Some includes files are included more > > > than once to generate various portion of the tracepoint code. So, my > > > advice here is to take a working example and adapt it to your needs. > > > Good advice. > > > > > > > > And the examples includes all of these three header files despite no > > > > conflict here. Could someone help to explain the intention? > > > > > > > > 4. Once I defined a tracepoint in my code, seems some initializations > > > > would register default probe into the hook point. How to disable the > > > > default probe and register my self-defined probes? > > > > > > > > > > You mean, call a custom function when tracepoint is executed? Maybe > > > somebody else has an answer, but AFAIK this is not possible. But you > > > could make a wrapper to your tracepoint and then call your additional > > > function there. > > > Yep, not possible. You'd have to wrap the tracepoint. > > > > > > > > 5. Does lttng-ust support dynamic traceing like kprobe? > > > > > > > > > > AFAIK, the kernel tracer supports kprobe, but not UST. Maybe somebody > > > else can confirm/infirm the dynamic tracepoint feature in user-space? > > > This is correct. > > > Thanks, > > > Mathieu > > > > > > > You can use a feature of GCC to regiter callback on function entry and > > > exit, but since it executes for all functions, then the overhead is very > > > high. You can have a look here: > > > > > > https://github.com/giraldeau/workload-kit/blob/master/ust/cyg.c > > > > > > Cheers, > > > > > > Francis > > > And here are some subsequent questions about lttng: > > 6. Does lttng-ust support marker? Marker is easier to be compatible > with the APIs like printf if we don't care its performance issue. Not currently. We plan to implement a "tracepoint_printf" (or trace_printf, not sure yet) that will behave similarly to the Linux Kernel Markers (back in the early lttng 0.x days). This will definitely fulfill your use-case, but it's just not there yet. > 7. What's supposed to show with 'lttng list -u'? It's empty now. Is > that possible to show the events defined in an application? List of tracepoints in _currently running_ _registered_ applications only. An application registers when linked with liblttng-ust. > 8. What does disable-event command of lttng do? With the > example(hello) provided by lttng-ust, I enabled all events with '-a > -u' and then disabled them again. I launched the example with gdb and > dumped the tracepoint's structure and then found its state was active. > It's supposed to be inactive here, right? Can you provide the detail of commands you launched, and the result of gdb printout ? Please run "hello" with LTTNG_UST_DEBUG=1 env. var set. > BTW: I didn't see any trace generated here with view command. That is after the disable, right ? Also, did you do a lttng start at some point in your test ? Thanks, Mathieu > > Thanks all for your useful info! > > Best regards > Zheng > > > > _______________________________________________ > > > lttng-dev mailing list > > > lttng-dev at lists.lttng.org > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > > > -- > > Mathieu Desnoyers > > Operating System Efficiency R&D Consultant > > EfficiOS Inc. > > http://www.efficios.com > > > _______________________________________________ > > lttng-dev mailing list > > lttng-dev at lists.lttng.org > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From toojays at toojays.net Wed Jun 20 09:10:12 2012 From: toojays at toojays.net (John Steele Scott) Date: Wed, 20 Jun 2012 22:40:12 +0930 Subject: [lttng-dev] lttng-ust with --std=c99 -pedantic In-Reply-To: <20120619174518.GB8776@Krystal> References: <20120612153257.GB14189@Krystal> <20120612155539.GA15859@Krystal> <4FDF1AC2.3070009@toojays.net> <20120618141156.GB6865@Krystal> <20120619174518.GB8776@Krystal> Message-ID: <4FE1CBB4.7040007@toojays.net> On 20/06/12 03:15, Mathieu Desnoyers wrote: > * Mathieu Desnoyers (mathieu.desnoyers at efficios.com) wrote: >> * John Steele Scott (toojays at toojays.net) wrote: >> snip snip >>> The thing I forgot to mention is: okay, I can build now with >>> -pedantic, but I can't build with "-pedantic -Werror". This >>> application builds with "--std=c99 -pedantic -Werror" (among many >>> other flags). I can of course remove -Werror for my tests, but in the >>> longer term I would like to see lttng-ust enabled in our regular >>> build, and disabling -Werror for that is not something we want to do. >>> If I only had to disable it for the trace provider, I could negotiate >>> that, but disabling it for any module which uses tracepoints is >>> undesirable. >>> >>> I haven't yet looked at how difficult it would be do eliminate these >>> pedantic warnings. Do you have a feel for what is involved? >>> Preprocessor tricks warp my mind. :( >> I don't think we'll want to make the lttng probe module build under >> --std=c99 -pedandic -Werror, but I think we should focus on making sure >> the tracepoint part that is built within the application (with >> TRACEPOINT_CREATE_PROBES _not_ defined) builds fine in --std=c99 >> -pedantic. >> >> Currently, building a simple test program with tracepoints under >> --std=c99 -pedantic gets me: >> >> >> In file included from ust_tests_hello.h:25:0, >> from hello.c:34: >> ../../include/lttng/tracepoint.h: In function ?__tracepoints__init?: >> ../../include/lttng/tracepoint.h:261:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] >> ../../include/lttng/tracepoint.h:265:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] >> ../../include/lttng/tracepoint.h:270:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] >> ../../include/lttng/tracepoint.h:274:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] >> ../../include/lttng/tracepoint.h:278:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] >> >> ---> see include/lttng/tracepoint.h __tracepoints__init(): >> ----> This is caused by use of dlsym() to lookup a function pointer from >> a symbol. dlsym() returns "void *", and we cast it into function >> pointer type. Ideas on how to make this c99 pedantic compliant are >> welcome. I found this warning discussed in a GCC bug report. Jakub Jelinek's suggestion at http://gcc.gnu.org/bugzilla/show_bug.cgi?id=45289#c6 (load the result of dlsym into a void *, then memcpy to the function pointer), does make these go away (using GCC 4.4.6 on Centos 6.2). >> >> ------------------ >> >> In file included from hello.c:34:0: >> ust_tests_hello.h: In function ?__tracepoint_cb_ust_tests_hello___tptest?: >> ust_tests_hello.h:28:1: warning: ISO C forbids braced-groups within expressions [-pedantic] >> ust_tests_hello.h:28:1: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] >> ust_tests_hello.h: In function ?__tracepoint_cb_ust_tests_hello___tptest_sighandler?: >> ust_tests_hello.h:52:1: warning: ISO C forbids braced-groups within expressions [-pedantic] >> ust_tests_hello.h:52:1: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] >> >> ---> see include/lttng/tracepoint.h >> >> #define _DECLARE_TRACEPOINT(_provider, _name, ...) \ >> extern struct tracepoint __tracepoint_##_provider##___##_name; \ >> static inline void __tracepoint_cb_##_provider##___##_name(_TP_ARGS_PROTO(__VA_ARGS__)) \ >> [...] >> >> ----> this is caused by use of "tp_rcu_dereference_bp()", which is >> defined in include/lttng/tracepoint-rcu.h (in the #else clause) : >> >> #define tp_rcu_dereference_bp(p) \ >> ({ \ >> __typeof__(p) _________p1 = URCU_FORCE_CAST(__typeof__(p), \ >> tracepoint_dlopen.rcu_dereference_sym_bp(URCU_FORCE_CAST(void *, p))); \ >> (_________p1); \ >> }) >> >> For this one, we should be able to change it into a single-expression >> and remove the braced-groups, since we are just really evaluating a >> single expression here. Fixed by the following commit in lttng-ust >> master: >> >> commit a4eaf8eabe829be8f7d7432ffaf83291a068b0ed >> Author: Mathieu Desnoyers >> Date: Mon Jun 18 10:10:36 2012 -0400 >> >> Fix c99 compatibility: tp_rcu_dereference_bp() should not use braced-groups within expressions >> >> ------------------ >> >> hello.c: In function ?inthandler?: >> hello.c:39:47: warning: ISO C99 requires rest arguments to be used [enabled by default] >> ---> see include/lttng/tracepoint.h tracepoint() macro. >> ----> this is caused by not passing any argument to the tracepoint. >> Ideas on how to fix this are welcome, as I don't see any way to do it in >> C99-pedantic. > I think I found a way to fix the last pedantic warnings. Can you try > with this master branch commit ? > > > commit fbdeb5ecb8ff9b7d73a72de9fc66d07f7797d93f > Author: Mathieu Desnoyers > Date: Tue Jun 19 13:45:05 2012 -0400 > > Fix C99 strict compatibility: don't use void * for function pointers > > compiling public headers with --std=x99 -pedantic shows: > > warning: ISO C forbids conversion of object pointer to function pointer type > > Use "void (*func)(void)" to represent a generic function pointer rather > than "void *". > > Signed-off-by: Mathieu Desnoyers > Unfortunately, this isn't sufficient to fix these warnings on Centos 6.2. However, I can now make the warnings go away by using dlsym+memcpy (instead of dlsym+cast), as mentioned above. I haven't had a chance to actually use the resulting binary yet, but I don't see why it shouldn't work. Thanks, John From mathieu.desnoyers at efficios.com Wed Jun 20 09:47:46 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Wed, 20 Jun 2012 09:47:46 -0400 Subject: [lttng-dev] lttng-ust with --std=c99 -pedantic In-Reply-To: <4FE1CBB4.7040007@toojays.net> References: <20120612153257.GB14189@Krystal> <20120612155539.GA15859@Krystal> <4FDF1AC2.3070009@toojays.net> <20120618141156.GB6865@Krystal> <20120619174518.GB8776@Krystal> <4FE1CBB4.7040007@toojays.net> Message-ID: <20120620134746.GB25924@Krystal> * John Steele Scott (toojays at toojays.net) wrote: > On 20/06/12 03:15, Mathieu Desnoyers wrote: > > * Mathieu Desnoyers (mathieu.desnoyers at efficios.com) wrote: > >> * John Steele Scott (toojays at toojays.net) wrote: > >> > snip snip > >>> The thing I forgot to mention is: okay, I can build now with > >>> -pedantic, but I can't build with "-pedantic -Werror". This > >>> application builds with "--std=c99 -pedantic -Werror" (among many > >>> other flags). I can of course remove -Werror for my tests, but in the > >>> longer term I would like to see lttng-ust enabled in our regular > >>> build, and disabling -Werror for that is not something we want to do. > >>> If I only had to disable it for the trace provider, I could negotiate > >>> that, but disabling it for any module which uses tracepoints is > >>> undesirable. > >>> > >>> I haven't yet looked at how difficult it would be do eliminate these > >>> pedantic warnings. Do you have a feel for what is involved? > >>> Preprocessor tricks warp my mind. :( > >> I don't think we'll want to make the lttng probe module build under > >> --std=c99 -pedandic -Werror, but I think we should focus on making sure > >> the tracepoint part that is built within the application (with > >> TRACEPOINT_CREATE_PROBES _not_ defined) builds fine in --std=c99 > >> -pedantic. > >> > >> Currently, building a simple test program with tracepoints under > >> --std=c99 -pedantic gets me: > >> > >> > >> In file included from ust_tests_hello.h:25:0, > >> from hello.c:34: > >> ../../include/lttng/tracepoint.h: In function ?__tracepoints__init?: > >> ../../include/lttng/tracepoint.h:261:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > >> ../../include/lttng/tracepoint.h:265:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > >> ../../include/lttng/tracepoint.h:270:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > >> ../../include/lttng/tracepoint.h:274:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > >> ../../include/lttng/tracepoint.h:278:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] > >> > >> ---> see include/lttng/tracepoint.h __tracepoints__init(): > >> ----> This is caused by use of dlsym() to lookup a function pointer from > >> a symbol. dlsym() returns "void *", and we cast it into function > >> pointer type. Ideas on how to make this c99 pedantic compliant are > >> welcome. > > I found this warning discussed in a GCC bug report. Jakub Jelinek's suggestion at http://gcc.gnu.org/bugzilla/show_bug.cgi?id=45289#c6 (load the result of dlsym into a void *, then memcpy to the function pointer), does make these go away (using GCC 4.4.6 on Centos 6.2). Can you check if these can be reproduced with gcc 4.5 or 4.6 ? (without the memcpy trick) The bugzilla entry seems to imply that the warning went away from -pedantic mode at some point. Thanks, Mathieu -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From toojays at toojays.net Wed Jun 20 20:51:01 2012 From: toojays at toojays.net (John Steele Scott) Date: Thu, 21 Jun 2012 10:21:01 +0930 Subject: [lttng-dev] lttng-ust with --std=c99 -pedantic In-Reply-To: <20120620134746.GB25924@Krystal> References: <20120612153257.GB14189@Krystal> <20120612155539.GA15859@Krystal> <4FDF1AC2.3070009@toojays.net> <20120618141156.GB6865@Krystal> <20120619174518.GB8776@Krystal> <4FE1CBB4.7040007@toojays.net> <20120620134746.GB25924@Krystal> Message-ID: <8762alsd1m.fsf@quantum.com> Mathieu Desnoyers writes: > * John Steele Scott (toojays at toojays.net) wrote: >> On 20/06/12 03:15, Mathieu Desnoyers wrote: >> > * Mathieu Desnoyers (mathieu.desnoyers at efficios.com) wrote: >> >> * John Steele Scott (toojays at toojays.net) wrote: >> >> >> snip snip >> >>> The thing I forgot to mention is: okay, I can build now with >> >>> -pedantic, but I can't build with "-pedantic -Werror". This >> >>> application builds with "--std=c99 -pedantic -Werror" (among many >> >>> other flags). I can of course remove -Werror for my tests, but in the >> >>> longer term I would like to see lttng-ust enabled in our regular >> >>> build, and disabling -Werror for that is not something we want to do. >> >>> If I only had to disable it for the trace provider, I could negotiate >> >>> that, but disabling it for any module which uses tracepoints is >> >>> undesirable. >> >>> >> >>> I haven't yet looked at how difficult it would be do eliminate these >> >>> pedantic warnings. Do you have a feel for what is involved? >> >>> Preprocessor tricks warp my mind. :( >> >> I don't think we'll want to make the lttng probe module build under >> >> --std=c99 -pedandic -Werror, but I think we should focus on making sure >> >> the tracepoint part that is built within the application (with >> >> TRACEPOINT_CREATE_PROBES _not_ defined) builds fine in --std=c99 >> >> -pedantic. >> >> >> >> Currently, building a simple test program with tracepoints under >> >> --std=c99 -pedantic gets me: >> >> >> >> >> >> In file included from ust_tests_hello.h:25:0, >> >> from hello.c:34: >> >> ../../include/lttng/tracepoint.h: In function ?__tracepoints__init?: >> >> ../../include/lttng/tracepoint.h:261:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] >> >> ../../include/lttng/tracepoint.h:265:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] >> >> ../../include/lttng/tracepoint.h:270:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] >> >> ../../include/lttng/tracepoint.h:274:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] >> >> ../../include/lttng/tracepoint.h:278:3: warning: ISO C forbids conversion of object pointer to function pointer type [-pedantic] >> >> >> >> ---> see include/lttng/tracepoint.h __tracepoints__init(): >> >> ----> This is caused by use of dlsym() to lookup a function pointer from >> >> a symbol. dlsym() returns "void *", and we cast it into function >> >> pointer type. Ideas on how to make this c99 pedantic compliant are >> >> welcome. >> >> I found this warning discussed in a GCC bug report. Jakub Jelinek's suggestion at http://gcc.gnu.org/bugzilla/show_bug.cgi?id=45289#c6 (load the result of dlsym into a void *, then memcpy to the function pointer), does make these go away (using GCC 4.4.6 on Centos 6.2). > > Can you check if these can be reproduced with gcc 4.5 or 4.6 ? (without > the memcpy trick) > > The bugzilla entry seems to imply that the warning went away from > -pedantic mode at some point. That was my reading of the bugzilla entry as well. However, I can still reproduce those warnings with GCC 4.5.4 and 4.6.1 (both installed on Ubuntu 11.10). I don't have a GCC 4.7 install to test with. An alternative approach to the memcpy thing is to wrap dlsym in a function which returns the symbol in an output parameter, rather than a return value. It looks a little less ugly. Something like: diff --git a/include/lttng/tracepoint.h b/include/lttng/tracepoint.h index 5bab476..8d2afc1 100644 --- a/include/lttng/tracepoint.h +++ b/include/lttng/tracepoint.h @@ -249,6 +249,14 @@ int __tracepoint_registered struct tracepoint_dlopen tracepoint_dlopen __attribute__((weak, visibility("hidden"))); +#ifndef LTTNG_HAVE_PEDANTIC_C99_DLSYM_WORKAROUND +#define LTTNG_HAVE_PEDANTIC_C99_DLSYM_WORKAROUND +static void lttng_c99_dlsym(void *handle, const char* symbol, void **addr) +{ + *addr = dlsym(handle, symbol); +} +#endif + static void __attribute__((constructor)) __tracepoints__init(void) { if (__tracepoint_registered++) @@ -258,27 +266,22 @@ static void __attribute__((constructor)) __tracepoints__init(void) dlopen("liblttng-ust-tracepoint.so.0", RTLD_NOW | RTLD_GLOBAL); if (!tracepoint_dlopen.liblttngust_handle) return; - tracepoint_dlopen.tracepoint_register_lib = - URCU_FORCE_CAST(int (*)(struct tracepoint * const *, int), - dlsym(tracepoint_dlopen.liblttngust_handle, - "tracepoint_register_lib")); - tracepoint_dlopen.tracepoint_unregister_lib = - URCU_FORCE_CAST(int (*)(struct tracepoint * const *), - dlsym(tracepoint_dlopen.liblttngust_handle, - "tracepoint_unregister_lib")); + lttng_c99_dlsym(tracepoint_dlopen.liblttngust_handle, + "tracepoint_register_lib", + (void **)&tracepoint_dlopen.tracepoint_register_lib); + lttng_c99_dlsym(tracepoint_dlopen.liblttngust_handle, + "tracepoint_unregister_lib", + (void **)&tracepoint_dlopen.tracepoint_unregister_lib); #ifndef _LGPL_SOURCE - tracepoint_dlopen.rcu_read_lock_sym_bp = - URCU_FORCE_CAST(void (*)(void), - dlsym(tracepoint_dlopen.liblttngust_handle, - "tp_rcu_read_lock_bp")); - tracepoint_dlopen.rcu_read_unlock_sym_bp = - URCU_FORCE_CAST(void (*)(void), - dlsym(tracepoint_dlopen.liblttngust_handle, - "tp_rcu_read_unlock_bp")); - tracepoint_dlopen.rcu_dereference_sym_bp = - URCU_FORCE_CAST(void *(*)(void *p), - dlsym(tracepoint_dlopen.liblttngust_handle, - "tp_rcu_dereference_sym_bp")); + lttng_c99_dlsym(tracepoint_dlopen.liblttngust_handle, + "tp_rcu_read_lock_bp", + (void **)&tracepoint_dlopen.rcu_read_lock_sym_bp); + lttng_c99_dlsym(tracepoint_dlopen.liblttngust_handle, + "tp_rcu_read_unlock_bp", + (void **)&tracepoint_dlopen.rcu_read_unlock_sym_bp); + lttng_c99_dlsym(tracepoint_dlopen.liblttngust_handle, + "tp_rcu_dereference_sym_bp", + (void **)&tracepoint_dlopen.rcu_dereference_sym_bp); #endif tracepoint_dlopen.tracepoint_register_lib(__start___tracepoints_ptrs, __stop___tracepoints_ptrs - On the system I'm trying to build which has the aggressive GCC warnings enabled, this gets me down to just one warning, which is due to redundantly declaring the __tracepoint_provider_##_provider symbol. That one is from -Wredundant-decls, not -pedantic; and I'll leave it for another day. cheers, John From mathieu.desnoyers at efficios.com Wed Jun 20 21:26:22 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Wed, 20 Jun 2012 21:26:22 -0400 Subject: [lttng-dev] Query about LTTng 2.0 Debian packages Message-ID: <20120621012622.GA1387@Krystal> Hi Jon, I wanted to get in touch with you regarding LTTng 2.0 Debian packages. Is there anything we can do to help you getting those packages into Debian ? They have been out for a while now, and even Ubuntu picked them up. I understand that you are probably busy, hence my offer for help. Thanks, Mathieu -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From toojays at toojays.net Wed Jun 20 21:30:12 2012 From: toojays at toojays.net (John Steele Scott) Date: Thu, 21 Jun 2012 11:00:12 +0930 Subject: [lttng-dev] Enumerated types in lttng-ust traces? Message-ID: It seems that the CTF supports enumerations, and babeltrace has support for pretty-printing them. But how do you get lttng-ust to emit them? Is this possible? Right now I'm just using ctf_integer, but it seems like there should be way to show human-readable enumeration names in the trace. cheers, John From mathieu.desnoyers at efficios.com Thu Jun 21 00:00:07 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Thu, 21 Jun 2012 00:00:07 -0400 Subject: [lttng-dev] Enumerated types in lttng-ust traces? In-Reply-To: References: Message-ID: <20120621040007.GA3080@Krystal> * John Steele Scott (toojays at toojays.net) wrote: > It seems that the CTF supports enumerations, and babeltrace has > support for pretty-printing them. But how do you get lttng-ust to emit > them? Is this possible? > > Right now I'm just using ctf_integer, but it seems like there should > be way to show human-readable enumeration names in the trace. Hi John, You are right, CTF supports enumerations. Currently, neither lttng-modules nor lttng-ust allow emitting those enumerations from tracepoints (it's just not been implemented yet). We'd have to add that to LTTng-UST tracepoints, e.g., something resembling to: TRACEPOINT_ENUM(enum_name, TP_ENUM_V(SOMENAMEA) /* value 0 */ TP_ENUM_V(SOMENAMEB) /* value 1 */ TP_ENUM_V(SOMENAMEC, 5) /* value 5 */ TP_ENUM_V(SOMENAMED) /* value 6 */ TP_ENUM_R(SOMENAMEE, 8, 10) /* range 8 to 10 (inclusive) */ TP_ENUM_V(SOMENAMEF) /* value 11 */ ) then: TRACEPOINT_EVENT(prog_module, event, TP_ARGS(int, someint), TP_FIELDS( ctf_enum(enum_name, int, fieldname, someint) ) ) where ctf_enum would be: ctf_enum(name of enumeration, type of enumeration container, name of the field, name of the TP_ARGS entry to use as source ) I did early experiments with this in lttng-modules (code unfinished and unused at this stage). You can dig it up in the files: probes/lttng-type-list.h probes/lttng-types.h of lttng-modules. Please keep in mind that these are unfinished and might need a lot of changes to get straight. Thoughts ? Thanks, Mathieu -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From Zheng.Chang at emc.com Thu Jun 21 00:59:20 2012 From: Zheng.Chang at emc.com (Zheng.Chang at emc.com) Date: Thu, 21 Jun 2012 00:59:20 -0400 Subject: [lttng-dev] Some questions about Lttng In-Reply-To: <20120620130338.GA25746@Krystal> References: <20120619163655.GC6743@Krystal> <6539770C71C3814BB0BFC2DBEBD105087FC418@CORPUSMX30B.corp.emc.com> <20120620130338.GA25746@Krystal> Message-ID: <6539770C71C3814BB0BFC2DBEBD105087FC8FC@CORPUSMX30B.corp.emc.com> > -----Original Message----- > From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] > Sent: Wednesday, June 20, 2012 21:04 PM > To: Chang, Zheng > Cc: francis.giraldeau at gmail.com; lttng-dev at lists.lttng.org > Subject: Re: [lttng-dev] Some questions about Lttng > > * Zheng.Chang at emc.com (Zheng.Chang at emc.com) wrote: > > > > > -----Original Message----- > > > From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] > > > Sent: Wednesday, June 20, 2012 0:37 AM > > > To: Francis Giraldeau > > > Cc: lttng-dev at lists.lttng.org > > > Subject: Re: [lttng-dev] Some questions about Lttng > > > > > * Francis Giraldeau (francis.giraldeau at gmail.com) wrote: > > > > Le 2012-06-19 05:50, Zheng.Chang at emc.com a ?crit : > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > > > > > I'm studying how to use Lttng now. I built a kernel which version is > > > > > 2.6.38 and ran with lttng 2.0. > > > > > > > > > > I got some confused when I started to use it. Here are my questions: > > > > > > > > > > > > > > > > > > > > 1. I didn't see kernel patches for kernel 3.x. Does it mean kernel 3.x > > > > > support it already? > > > > > > > > > > > > > You don't need a kernel patch with lttng 2.0, only modules are required. > > > > It uses the tracepoints already present in the kernel, and trace system > > > > calls, and uses perf performance counters and kprobes. > > > > > > > > > 2. I tried to do something like, dump the arguments of system call, or > > > > > dump a backtrace in a specified function. But the output of lttng is > > > > > very limited. Is there a way to do that with lttng? > > > > > > > > > > > > > If system calls are enabled, arguments are supposed to be dumped (option > > > > --syscall to lttng enable-event). If it's not the case, then are you > > > > sure you are using lttng 2.0 and not 0.12? ;) Because the older version > > > > has a different behavior. One additional note: few system calls do not > > > > have all their arguments decoded in lttng 2, but there are only a few. > > > > > There is no backtrace dump feature in lttng 2.0. > > > > > Arguments of system calls are almost all there on x86 32/64 and ARM. > > > What architecture are you using ? > > > > > > My tsetbed is x86 32bit and lttng's version is 2.0.1. You're right. I can see the arguments now. > > > > > > > > > > > > > 3. I looked into some UST examples and found here are three header > > > > > files: tracepoint.h, tracepoint-event.h and ust-tracepoint-event.h. > > > > > They have some duplicated macro definitions like TRACEPOINT_EVENT. > > > > > > > > > > > > > The macros are quite complicated. Some includes files are included more > > > > than once to generate various portion of the tracepoint code. So, my > > > > advice here is to take a working example and adapt it to your needs. > > > > > Good advice. > > > > > > > > > > > And the examples includes all of these three header files despite no > > > > > conflict here. Could someone help to explain the intention? > > > > > > > > > > 4. Once I defined a tracepoint in my code, seems some initializations > > > > > would register default probe into the hook point. How to disable the > > > > > default probe and register my self-defined probes? > > > > > > > > > > > > > You mean, call a custom function when tracepoint is executed? Maybe > > > > somebody else has an answer, but AFAIK this is not possible. But you > > > > could make a wrapper to your tracepoint and then call your additional > > > > function there. > > > > > Yep, not possible. You'd have to wrap the tracepoint. > > > > > > > > > > > 5. Does lttng-ust support dynamic traceing like kprobe? > > > > > > > > > > > > > AFAIK, the kernel tracer supports kprobe, but not UST. Maybe somebody > > > > else can confirm/infirm the dynamic tracepoint feature in user-space? > > > > > This is correct. > > > > > Thanks, > > > > > Mathieu > > > > > > > > > > You can use a feature of GCC to regiter callback on function entry and > > > > exit, but since it executes for all functions, then the overhead is very > > > > high. You can have a look here: > > > > > > > > https://github.com/giraldeau/workload-kit/blob/master/ust/cyg.c > > > > > > > > Cheers, > > > > > > > > Francis > > > > > > And here are some subsequent questions about lttng: > > > > 6. Does lttng-ust support marker? Marker is easier to be compatible > > with the APIs like printf if we don't care its performance issue. > > Not currently. We plan to implement a "tracepoint_printf" (or > trace_printf, not sure yet) that will behave similarly to the Linux > Kernel Markers (back in the early lttng 0.x days). This will definitely > fulfill your use-case, but it's just not there yet. > > > 7. What's supposed to show with 'lttng list -u'? It's empty now. Is > > that possible to show the events defined in an application? > > List of tracepoints in _currently running_ _registered_ applications > only. An application registers when linked with liblttng-ust. > > > 8. What does disable-event command of lttng do? With the > > example(hello) provided by lttng-ust, I enabled all events with '-a > > -u' and then disabled them again. I launched the example with gdb and > > dumped the tracepoint's structure and then found its state was active. > > It's supposed to be inactive here, right? > > Can you provide the detail of commands you launched, and the result of > gdb printout ? Please run "hello" with LTTNG_UST_DEBUG=1 env. var set. > > > BTW: I didn't see any trace generated here with view command. > > That is after the disable, right ? Yes > Also, did you do a lttng start at some point in your test ? Yes. Following three examples(A/B/C) provide the testing info for question 8: Example A: None event lttng create hello lttng list hello Tracing session hello: [active] Trace path: /root/lttng-traces/hello-20120621-071913 lttng start gdb ./hello (gdb) file .libs/hello (gdb) set env LTTNG_UST_DEBUG=1 (gdb) show env ... LTTNG_UST_DEBUG=1 (gdb) br main Breakpoint 1 at 0x80489b9: file hello.c, line 84. (gdb) r Starting program: /root/project/lttng/lttng-ust-2.0.2/tests/hello/.libs/hello [Thread debugging using libthread_db enabled] liblttng_ust_tracepoint[3341/3341]: just registered a tracepoints section from 0xb7fddd64 and having 1 tracepoints (in tracepoint_register_lib() at tracepoint.c:638) liblttng_ust_tracepoint[3341/3341]: registered tracepoint: lttng_ust:metadata (in tracepoint_register_lib() at tracepoint.c:643) libust[3341/3341]: just registered probe lttng_ust containing 1 events (in ltt_probe_register() at ltt-probes.c:109) libust[3341/3341]: Registered event probe "lttng_ust:metadata" with signature "const char *, str" (in ltt_probe_register() at ltt-probes.c:118) libust[3341/3341]: LTT : ltt ring buffer client init (in ltt_ring_buffer_metadata_client_init() at ltt-ring-buffer-metadata-client.h:334) libust[3341/3341]: LTT : ltt ring buffer client init (in ltt_ring_buffer_client_overwrite_init() at ltt-ring-buffer-client.h:584) libust[3341/3341]: LTT : ltt ring buffer client init (in ltt_ring_buffer_client_discard_init() at ltt-ring-buffer-client.h:584) [New Thread 0xb7dddb70 (LWP 3344)] [New Thread 0xb75dcb70 (LWP 3345)] libust[3341/3344]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) libust[3341/3344]: Waiting for local apps sessiond (in wait_for_sessiond() at lttng-ust-comm.c:621) libust[3341/3344]: Linux kernels 2.6.33 to 3.0 (with the exception of stable versions) do not support FUTEX_WAKE on read-only memory mappings correctly. Please upgrade your kernel (fix is commit 9ea71503a8ed9184d2d0b8ccc4d269d05f7940ae in Linux kernel mainline). LTTng-UST will use polling mode fallback. (in wait_for_sessiond() at lttng-ust-comm.c:634) libust[3341/3344]: Error: futex: Bad address (in wait_for_sessiond() at lttng-ust-comm.c:636) libust[3341/3344]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) libust[3341/3345]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3341/3345]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3341/3345]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3341/3345]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3341/3341]: just registered probe ust_tests_hello containing 2 events (in ltt_probe_register() at ltt-probes.c:109) libust[3341/3341]: Registered event probe "ust_tests_hello:tptest" with signature "int, anint, int, netint, long *, values, char *, text, size_t, textlen, double, doublearg, float, floatarg" (in ltt_probe_register() at ltt-probes.c:118) libust[3341/3341]: Registered event probe "ust_tests_hello:tptest_sighandler" with signature "" (in ltt_probe_register() at ltt-probes.c:118) liblttng_ust_tracepoint[3341/3341]: just registered a tracepoints section from 0x804b6c4 and having 2 tracepoints (in tracepoint_register_lib() at tracepoint.c:638) liblttng_ust_tracepoint[3341/3341]: registered tracepoint: ust_tests_hello:tptest_sighandler (in tracepoint_register_lib() at tracepoint.c:643) liblttng_ust_tracepoint[3341/3341]: registered tracepoint: ust_tests_hello:tptest (in tracepoint_register_lib() at tracepoint.c:643) Breakpoint 1, main (argc=1, argv=0xbffff754) at hello.c:84 84 if (argc == 2) Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.47.el6_2.12.i686 libuuid-2.17.2-12.4.el6.i686 (gdb) p __tracepoint_ust_tests_hello___tptest $1 = {name = 0x804a380 "ust_tests_hello:tptest", state = 0, probes = 0x0, tracepoint_provider_ref = 0x804b724, signature = 0x8049240 "int, anint, int, netint, long *, values, char *, text, size_t, textlen, double, doublearg, float, floatarg", padding = '\000' } Notice here the __tracepoint_ust_tests_hello___tptest.state and __tracepoint_ust_tests_hello___tptest.probes both are 0, which means no register and API tracepoint doesn't invoke probe. Example B: enable event ust_tests_hello:tptest lttng create hello lttng enable-event ust_tests_hello:tptest -u lttng list hello Tracing session hello: [inactive] Trace path: /root/lttng-traces/hello-20120621-073315 === Domain: UST global === Channels: ------------- - channel0: [enabled] Attributes: overwrite mode: 0 subbufers size: 4096 number of subbufers: 4 switch timer interval: 0 read timer interval: 200 output: mmap() Events: ust_tests_hello:tptest (type: tracepoint) [enabled] lttng start gdb ./hello (gdb) file .libs/hello (gdb) set env LTTNG_UST_DEBUG=1 (gdb) show env ... LTTNG_UST_DEBUG=1 (gdb) br main Breakpoint 1 at 0x80489b9: file hello.c, line 84. (gdb) r Starting program: /root/project/lttng/lttng-ust-2.0.2/tests/hello/.libs/hello [Thread debugging using libthread_db enabled] liblttng_ust_tracepoint[3365/3365]: just registered a tracepoints section from 0xb7fddd64 and having 1 tracepoints (in tracepoint_register_lib() at tracepoint.c:638) liblttng_ust_tracepoint[3365/3365]: registered tracepoint: lttng_ust:metadata (in tracepoint_register_lib() at tracepoint.c:643) libust[3365/3365]: just registered probe lttng_ust containing 1 events (in ltt_probe_register() at ltt-probes.c:109) libust[3365/3365]: Registered event probe "lttng_ust:metadata" with signature "const char *, str" (in ltt_probe_register() at ltt-probes.c:118) libust[3365/3365]: LTT : ltt ring buffer client init (in ltt_ring_buffer_metadata_client_init() at ltt-ring-buffer-metadata-client.h:334) libust[3365/3365]: LTT : ltt ring buffer client init (in ltt_ring_buffer_client_overwrite_init() at ltt-ring-buffer-client.h:584) libust[3365/3365]: LTT : ltt ring buffer client init (in ltt_ring_buffer_client_discard_init() at ltt-ring-buffer-client.h:584) [New Thread 0xb7dddb70 (LWP 3368)] libust[3365/3368]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) [New Thread 0xb75dcb70 (LWP 3369)] libust[3365/3368]: Waiting for local apps sessiond (in wait_for_sessiond() at lttng-ust-comm.c:621) libust[3365/3368]: Linux kernels 2.6.33 to 3.0 (with the exception of stable versions) do not support FUTEX_WAKE on read-only memory mappings correctly. Please upgrade your kernel (fix is commit 9ea71503a8ed9184d2d0b8ccc4d269d05f7940ae in Linux kernel mainline). LTTng-UST will use polling mode fallback. (in wait_for_sessiond() at lttng-ust-comm.c:634) libust[3365/3368]: Error: futex: Bad address (in wait_for_sessiond() at lttng-ust-comm.c:636) libust[3365/3368]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) libust[3365/3369]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3365/3369]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3365/3369]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3365/3369]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3365/3369]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) liblttng_ust_tracepoint[3365/3369]: Registering probe to tracepoint lttng_ust:metadata (in __tracepoint_probe_register() at tracepoint.c:426) libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3365/3369]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3365/3369]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3365/3369]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3365/3369]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3365/3369]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3365/3369]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3365/3369]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3365/3365]: just registered probe ust_tests_hello containing 2 events (in ltt_probe_register() at ltt-probes.c:109) libust[3365/3365]: Registered event probe "ust_tests_hello:tptest" with signature "int, anint, int, netint, long *, values, char *, text, size_t, textlen, double, doublearg, float, floatarg" (in ltt_probe_register() at ltt-probes.c:118) liblttng_ust_tracepoint[3365/3365]: Registering probe to tracepoint ust_tests_hello:tptest (in __tracepoint_probe_register() at tracepoint.c:426) libust[3365/3365]: Registered event probe "ust_tests_hello:tptest_sighandler" with signature "" (in ltt_probe_register() at ltt-probes.c:118) liblttng_ust_tracepoint[3365/3365]: just registered a tracepoints section from 0x804b6c4 and having 2 tracepoints (in tracepoint_register_lib() at tracepoint.c:638) liblttng_ust_tracepoint[3365/3365]: registered tracepoint: ust_tests_hello:tptest_sighandler (in tracepoint_register_lib() at tracepoint.c:643) liblttng_ust_tracepoint[3365/3365]: registered tracepoint: ust_tests_hello:tptest (in tracepoint_register_lib() at tracepoint.c:643) Breakpoint 1, main (argc=1, argv=0xbffff754) at hello.c:84 84 if (argc == 2) Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.47.el6_2.12.i686 libuuid-2.17.2-12.4.el6.i686 (gdb) p __tracepoint_ust_tests_hello___tptest $1 = {name = 0x804a380 "ust_tests_hello:tptest", state = 1, probes = 0x804c258, tracepoint_provider_ref = 0x804b724, signature = 0x8049240 "int, anint, int, netint, long *, values, char *, text, size_t, textlen, double, doublearg, float, floatarg", padding = '\000' } We can see __tracepoint_ust_tests_hello___tptest.state and __tracepoint_ust_tests_hello___tptest.probes have been set, which means API tracepoint can invoke probe callback function and generate trace info. Example C: enable event and then disable it lttng create hello Session hello created. Traces will be written in /root/lttng-traces/hello-20120621-074221 lttng enable-event ust_tests_hello:tptest -u UST event ust_tests_hello:tptest created in channel channel0 lttng disable-event ust_tests_hello:tptest -u UST event ust_tests_hello:tptest disabled in channel channel0 for session hello lttng list hello Tracing session hello: [inactive] Trace path: /root/lttng-traces/hello-20120621-074221 === Domain: UST global === Channels: ------------- - channel0: [enabled] Attributes: overwrite mode: 0 subbufers size: 4096 number of subbufers: 4 switch timer interval: 0 read timer interval: 200 output: mmap() Events: ust_tests_hello:tptest (type: tracepoint) [disabled] lttng start gdb ./hello (gdb) file .libs/hello (gdb) set env LTTNG_UST_DEBUG=1 (gdb) show env ... LTTNG_UST_DEBUG=1 (gdb) br main Breakpoint 1 at 0x80489b9: file hello.c, line 84. (gdb) r Starting program: /root/project/lttng/lttng-ust-2.0.2/tests/hello/.libs/hello [Thread debugging using libthread_db enabled] liblttng_ust_tracepoint[3384/3384]: just registered a tracepoints section from 0xb7fddd64 and having 1 tracepoints (in tracepoint_register_lib() at tracepoint.c:638) liblttng_ust_tracepoint[3384/3384]: registered tracepoint: lttng_ust:metadata (in tracepoint_register_lib() at tracepoint.c:643) libust[3384/3384]: just registered probe lttng_ust containing 1 events (in ltt_probe_register() at ltt-probes.c:109) libust[3384/3384]: Registered event probe "lttng_ust:metadata" with signature "const char *, str" (in ltt_probe_register() at ltt-probes.c:118) libust[3384/3384]: LTT : ltt ring buffer client init (in ltt_ring_buffer_metadata_client_init() at ltt-ring-buffer-metadata-client.h:334) libust[3384/3384]: LTT : ltt ring buffer client init (in ltt_ring_buffer_client_overwrite_init() at ltt-ring-buffer-client.h:584) libust[3384/3384]: LTT : ltt ring buffer client init (in ltt_ring_buffer_client_discard_init() at ltt-ring-buffer-client.h:584) [New Thread 0xb7dddb70 (LWP 3387)] [New Thread 0xb75dcb70 (LWP 3388)] libust[3384/3387]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) libust[3384/3387]: Waiting for local apps sessiond (in wait_for_sessiond() at lttng-ust-comm.c:621) libust[3384/3387]: Linux kernels 2.6.33 to 3.0 (with the exception of stable versions) do not support FUTEX_WAKE on read-only memory mappings correctly. Please upgrade your kernel (fix is commit 9ea71503a8ed9184d2d0b8ccc4d269d05f7940ae in Linux kernel mainline). LTTng-UST will use polling mode fallback. (in wait_for_sessiond() at lttng-ust-comm.c:634) libust[3384/3387]: Error: futex: Bad address (in wait_for_sessiond() at lttng-ust-comm.c:636) libust[3384/3387]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) libust[3384/3388]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3384/3388]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3384/3388]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3384/3388]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3384/3388]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3384/3388]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) liblttng_ust_tracepoint[3384/3388]: Registering probe to tracepoint lttng_ust:metadata (in __tracepoint_probe_register() at tracepoint.c:426) libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3384/3388]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3384/3388]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3384/3388]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3384/3388]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3384/3388]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3384/3388]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3384/3388]: message received (in ust_listener_thread() at lttng-ust-comm.c:766) libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) libust[3384/3384]: just registered probe ust_tests_hello containing 2 events (in ltt_probe_register() at ltt-probes.c:109) libust[3384/3384]: Registered event probe "ust_tests_hello:tptest" with signature "int, anint, int, netint, long *, values, char *, text, size_t, textlen, double, doublearg, float, floatarg" (in ltt_probe_register() at ltt-probes.c:118) liblttng_ust_tracepoint[3384/3384]: Registering probe to tracepoint ust_tests_hello:tptest (in __tracepoint_probe_register() at tracepoint.c:426) libust[3384/3384]: Registered event probe "ust_tests_hello:tptest_sighandler" with signature "" (in ltt_probe_register() at ltt-probes.c:118) liblttng_ust_tracepoint[3384/3384]: just registered a tracepoints section from 0x804b6c4 and having 2 tracepoints (in tracepoint_register_lib() at tracepoint.c:638) liblttng_ust_tracepoint[3384/3384]: registered tracepoint: ust_tests_hello:tptest_sighandler (in tracepoint_register_lib() at tracepoint.c:643) liblttng_ust_tracepoint[3384/3384]: registered tracepoint: ust_tests_hello:tptest (in tracepoint_register_lib() at tracepoint.c:643) Breakpoint 1, main (argc=1, argv=0xbffff754) at hello.c:84 84 if (argc == 2) Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.47.el6_2.12.i686 libuuid-2.17.2-12.4.el6.i686 (gdb) p __tracepoint_ust_tests_hello___tptest $1 = {name = 0x804a380 "ust_tests_hello:tptest", state = 1, probes = 0x804c258, tracepoint_provider_ref = 0x804b724, signature = 0x8049240 "int, anint, int, netint, long *, values, char *, text, size_t, textlen, double, doublearg, float, floatarg", padding = '\000' } Here __tracepoint_ust_tests_hello___tptest.state and __tracepoint_ust_tests_hello___tptest.probes have been set, which means probe callback function wiil be called by API tracepoint. But the event has been disabled, is this necessary to call into the probe again? If this behavior is normal, what's suppoed to do with disable-event command? Any misunderstanding please fix me. Thanks Best Regards Zheng > Thanks, > Mathieu > > Thanks all for your useful info! > > Best regards > Zheng > > > > _______________________________________________ > > > lttng-dev mailing list > > > lttng-dev at lists.lttng.org > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > > > -- > > Mathieu Desnoyers > > Operating System Efficiency R&D Consultant > > EfficiOS Inc. > > http://www.efficios.com > > > _______________________________________________ > > lttng-dev mailing list > > lttng-dev at lists.lttng.org > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > -- > Mathieu Desnoyers > Operating System Efficiency R&D Consultant > EfficiOS Inc. > http://www.efficios.com From mathieu.desnoyers at efficios.com Thu Jun 21 08:45:32 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Thu, 21 Jun 2012 08:45:32 -0400 Subject: [lttng-dev] Some questions about Lttng In-Reply-To: <6539770C71C3814BB0BFC2DBEBD105087FC8FC@CORPUSMX30B.corp.emc.com> References: <20120619163655.GC6743@Krystal> <6539770C71C3814BB0BFC2DBEBD105087FC418@CORPUSMX30B.corp.emc.com> <20120620130338.GA25746@Krystal> <6539770C71C3814BB0BFC2DBEBD105087FC8FC@CORPUSMX30B.corp.emc.com> Message-ID: <20120621124532.GA7255@Krystal> * Zheng.Chang at emc.com (Zheng.Chang at emc.com) wrote: > > -----Original Message----- > > From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] [...] > > > 8. What does disable-event command of lttng do? With the > > > example(hello) provided by lttng-ust, I enabled all events with '-a > > > -u' and then disabled them again. I launched the example with gdb and > > > dumped the tracepoint's structure and then found its state was active. > > > It's supposed to be inactive here, right? > > > > Can you provide the detail of commands you launched, and the result of > > gdb printout ? Please run "hello" with LTTNG_UST_DEBUG=1 env. var set. > > > > > BTW: I didn't see any trace generated here with view command. > > > > That is after the disable, right ? > > Yes > > > Also, did you do a lttng start at some point in your test ? > > > Yes. Following three examples(A/B/C) provide the testing info for question 8: > > Example A: None event > lttng create hello > lttng list hello > Tracing session hello: [active] > Trace path: /root/lttng-traces/hello-20120621-071913 > > lttng start > > gdb ./hello > (gdb) file .libs/hello > (gdb) set env LTTNG_UST_DEBUG=1 > (gdb) show env > ... > LTTNG_UST_DEBUG=1 > (gdb) br main > Breakpoint 1 at 0x80489b9: file hello.c, line 84. > (gdb) r > Starting program: /root/project/lttng/lttng-ust-2.0.2/tests/hello/.libs/hello > [Thread debugging using libthread_db enabled] > liblttng_ust_tracepoint[3341/3341]: just registered a tracepoints section from 0xb7fddd64 and having 1 tracepoints (in tracepoint_register_lib() at tracepoint.c:638) > liblttng_ust_tracepoint[3341/3341]: registered tracepoint: lttng_ust:metadata (in tracepoint_register_lib() at tracepoint.c:643) > libust[3341/3341]: just registered probe lttng_ust containing 1 events (in ltt_probe_register() at ltt-probes.c:109) > libust[3341/3341]: Registered event probe "lttng_ust:metadata" with signature "const char *, str" (in ltt_probe_register() at ltt-probes.c:118) > libust[3341/3341]: LTT : ltt ring buffer client init > (in ltt_ring_buffer_metadata_client_init() at ltt-ring-buffer-metadata-client.h:334) > libust[3341/3341]: LTT : ltt ring buffer client init > (in ltt_ring_buffer_client_overwrite_init() at ltt-ring-buffer-client.h:584) > libust[3341/3341]: LTT : ltt ring buffer client init > (in ltt_ring_buffer_client_discard_init() at ltt-ring-buffer-client.h:584) > [New Thread 0xb7dddb70 (LWP 3344)] > [New Thread 0xb75dcb70 (LWP 3345)] > libust[3341/3344]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) > libust[3341/3344]: Waiting for local apps sessiond (in wait_for_sessiond() at lttng-ust-comm.c:621) > libust[3341/3344]: Linux kernels 2.6.33 to 3.0 (with the exception of stable versions) do not support FUTEX_WAKE on read-only memory mappings correctly. Please upgrade your kernel (fix is commit 9ea71503a8ed9184d2d0b8ccc4d269d05f7940ae in Linux kernel mainline). LTTng-UST will use polling mode fallback. (in wait_for_sessiond() at lttng-ust-comm.c:634) > libust[3341/3344]: Error: futex: Bad address (in wait_for_sessiond() at lttng-ust-comm.c:636) > libust[3341/3344]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) > libust[3341/3345]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3341/3345]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3341/3345]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3341/3345]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3341/3341]: just registered probe ust_tests_hello containing 2 events (in ltt_probe_register() at ltt-probes.c:109) > libust[3341/3341]: Registered event probe "ust_tests_hello:tptest" with signature "int, anint, int, netint, long *, values, char *, text, size_t, textlen, double, doublearg, float, floatarg" (in ltt_probe_register() at ltt-probes.c:118) > libust[3341/3341]: Registered event probe "ust_tests_hello:tptest_sighandler" with signature "" (in ltt_probe_register() at ltt-probes.c:118) > liblttng_ust_tracepoint[3341/3341]: just registered a tracepoints section from 0x804b6c4 and having 2 tracepoints (in tracepoint_register_lib() at tracepoint.c:638) > liblttng_ust_tracepoint[3341/3341]: registered tracepoint: ust_tests_hello:tptest_sighandler (in tracepoint_register_lib() at tracepoint.c:643) > liblttng_ust_tracepoint[3341/3341]: registered tracepoint: ust_tests_hello:tptest (in tracepoint_register_lib() at tracepoint.c:643) > > Breakpoint 1, main (argc=1, argv=0xbffff754) at hello.c:84 > 84 if (argc == 2) > Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.47.el6_2.12.i686 libuuid-2.17.2-12.4.el6.i686 > (gdb) p __tracepoint_ust_tests_hello___tptest > $1 = {name = 0x804a380 "ust_tests_hello:tptest", state = 0, probes = 0x0, tracepoint_provider_ref = 0x804b724, > signature = 0x8049240 "int, anint, int, netint, long *, values, char *, text, size_t, textlen, double, doublearg, float, floatarg", padding = '\000' } > > Notice here the __tracepoint_ust_tests_hello___tptest.state and __tracepoint_ust_tests_hello___tptest.probes both are 0, which means no register and API tracepoint doesn't invoke probe. > > Example B: enable event ust_tests_hello:tptest > lttng create hello > lttng enable-event ust_tests_hello:tptest -u > lttng list hello > Tracing session hello: [inactive] > Trace path: /root/lttng-traces/hello-20120621-073315 > > === Domain: UST global === > > Channels: > ------------- > - channel0: [enabled] > > Attributes: > overwrite mode: 0 > subbufers size: 4096 > number of subbufers: 4 > switch timer interval: 0 > read timer interval: 200 > output: mmap() > > Events: > ust_tests_hello:tptest (type: tracepoint) [enabled] > lttng start > gdb ./hello > (gdb) file .libs/hello > (gdb) set env LTTNG_UST_DEBUG=1 > (gdb) show env > ... > LTTNG_UST_DEBUG=1 > (gdb) br main > Breakpoint 1 at 0x80489b9: file hello.c, line 84. > (gdb) r > Starting program: /root/project/lttng/lttng-ust-2.0.2/tests/hello/.libs/hello > [Thread debugging using libthread_db enabled] > liblttng_ust_tracepoint[3365/3365]: just registered a tracepoints section from 0xb7fddd64 and having 1 tracepoints (in tracepoint_register_lib() at tracepoint.c:638) > liblttng_ust_tracepoint[3365/3365]: registered tracepoint: lttng_ust:metadata (in tracepoint_register_lib() at tracepoint.c:643) > libust[3365/3365]: just registered probe lttng_ust containing 1 events (in ltt_probe_register() at ltt-probes.c:109) > libust[3365/3365]: Registered event probe "lttng_ust:metadata" with signature "const char *, str" (in ltt_probe_register() at ltt-probes.c:118) > libust[3365/3365]: LTT : ltt ring buffer client init > (in ltt_ring_buffer_metadata_client_init() at ltt-ring-buffer-metadata-client.h:334) > libust[3365/3365]: LTT : ltt ring buffer client init > (in ltt_ring_buffer_client_overwrite_init() at ltt-ring-buffer-client.h:584) > libust[3365/3365]: LTT : ltt ring buffer client init > (in ltt_ring_buffer_client_discard_init() at ltt-ring-buffer-client.h:584) > [New Thread 0xb7dddb70 (LWP 3368)] > libust[3365/3368]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) > [New Thread 0xb75dcb70 (LWP 3369)] > libust[3365/3368]: Waiting for local apps sessiond (in wait_for_sessiond() at lttng-ust-comm.c:621) > libust[3365/3368]: Linux kernels 2.6.33 to 3.0 (with the exception of stable versions) do not support FUTEX_WAKE on read-only memory mappings correctly. Please upgrade your kernel (fix is commit 9ea71503a8ed9184d2d0b8ccc4d269d05f7940ae in Linux kernel mainline). LTTng-UST will use polling mode fallback. (in wait_for_sessiond() at lttng-ust-comm.c:634) > libust[3365/3368]: Error: futex: Bad address (in wait_for_sessiond() at lttng-ust-comm.c:636) > libust[3365/3368]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) > libust[3365/3369]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3365/3369]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3365/3369]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3365/3369]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3365/3369]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > liblttng_ust_tracepoint[3365/3369]: Registering probe to tracepoint lttng_ust:metadata (in __tracepoint_probe_register() at tracepoint.c:426) > libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3365/3369]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3365/3369]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3365/3369]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3365/3369]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3365/3369]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3365/3369]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3365/3369]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3365/3365]: just registered probe ust_tests_hello containing 2 events (in ltt_probe_register() at ltt-probes.c:109) > libust[3365/3365]: Registered event probe "ust_tests_hello:tptest" with signature "int, anint, int, netint, long *, values, char *, text, size_t, textlen, double, doublearg, float, floatarg" (in ltt_probe_register() at ltt-probes.c:118) > liblttng_ust_tracepoint[3365/3365]: Registering probe to tracepoint ust_tests_hello:tptest (in __tracepoint_probe_register() at tracepoint.c:426) > libust[3365/3365]: Registered event probe "ust_tests_hello:tptest_sighandler" with signature "" (in ltt_probe_register() at ltt-probes.c:118) > liblttng_ust_tracepoint[3365/3365]: just registered a tracepoints section from 0x804b6c4 and having 2 tracepoints (in tracepoint_register_lib() at tracepoint.c:638) > liblttng_ust_tracepoint[3365/3365]: registered tracepoint: ust_tests_hello:tptest_sighandler (in tracepoint_register_lib() at tracepoint.c:643) > liblttng_ust_tracepoint[3365/3365]: registered tracepoint: ust_tests_hello:tptest (in tracepoint_register_lib() at tracepoint.c:643) > > Breakpoint 1, main (argc=1, argv=0xbffff754) at hello.c:84 > 84 if (argc == 2) > Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.47.el6_2.12.i686 libuuid-2.17.2-12.4.el6.i686 > (gdb) p __tracepoint_ust_tests_hello___tptest > $1 = {name = 0x804a380 "ust_tests_hello:tptest", state = 1, probes = 0x804c258, tracepoint_provider_ref = 0x804b724, > signature = 0x8049240 "int, anint, int, netint, long *, values, char *, text, size_t, textlen, double, doublearg, float, floatarg", padding = '\000' } > > We can see __tracepoint_ust_tests_hello___tptest.state and __tracepoint_ust_tests_hello___tptest.probes have been set, which means API tracepoint can invoke probe callback function and generate trace info. > > Example C: enable event and then disable it > lttng create hello > Session hello created. > Traces will be written in /root/lttng-traces/hello-20120621-074221 > lttng enable-event ust_tests_hello:tptest -u > UST event ust_tests_hello:tptest created in channel channel0 > lttng disable-event ust_tests_hello:tptest -u > UST event ust_tests_hello:tptest disabled in channel channel0 for session hello > lttng list hello > Tracing session hello: [inactive] > Trace path: /root/lttng-traces/hello-20120621-074221 > > === Domain: UST global === > > Channels: > ------------- > - channel0: [enabled] > > Attributes: > overwrite mode: 0 > subbufers size: 4096 > number of subbufers: 4 > switch timer interval: 0 > read timer interval: 200 > output: mmap() > > Events: > ust_tests_hello:tptest (type: tracepoint) [disabled] > > lttng start > gdb ./hello > (gdb) file .libs/hello > (gdb) set env LTTNG_UST_DEBUG=1 > (gdb) show env > ... > LTTNG_UST_DEBUG=1 > (gdb) br main > Breakpoint 1 at 0x80489b9: file hello.c, line 84. > (gdb) r > Starting program: /root/project/lttng/lttng-ust-2.0.2/tests/hello/.libs/hello > [Thread debugging using libthread_db enabled] > liblttng_ust_tracepoint[3384/3384]: just registered a tracepoints section from 0xb7fddd64 and having 1 tracepoints (in tracepoint_register_lib() at tracepoint.c:638) > liblttng_ust_tracepoint[3384/3384]: registered tracepoint: lttng_ust:metadata (in tracepoint_register_lib() at tracepoint.c:643) > libust[3384/3384]: just registered probe lttng_ust containing 1 events (in ltt_probe_register() at ltt-probes.c:109) > libust[3384/3384]: Registered event probe "lttng_ust:metadata" with signature "const char *, str" (in ltt_probe_register() at ltt-probes.c:118) > libust[3384/3384]: LTT : ltt ring buffer client init > (in ltt_ring_buffer_metadata_client_init() at ltt-ring-buffer-metadata-client.h:334) > libust[3384/3384]: LTT : ltt ring buffer client init > (in ltt_ring_buffer_client_overwrite_init() at ltt-ring-buffer-client.h:584) > libust[3384/3384]: LTT : ltt ring buffer client init > (in ltt_ring_buffer_client_discard_init() at ltt-ring-buffer-client.h:584) > [New Thread 0xb7dddb70 (LWP 3387)] > [New Thread 0xb75dcb70 (LWP 3388)] > libust[3384/3387]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) > libust[3384/3387]: Waiting for local apps sessiond (in wait_for_sessiond() at lttng-ust-comm.c:621) > libust[3384/3387]: Linux kernels 2.6.33 to 3.0 (with the exception of stable versions) do not support FUTEX_WAKE on read-only memory mappings correctly. Please upgrade your kernel (fix is commit 9ea71503a8ed9184d2d0b8ccc4d269d05f7940ae in Linux kernel mainline). LTTng-UST will use polling mode fallback. (in wait_for_sessiond() at lttng-ust-comm.c:634) > libust[3384/3387]: Error: futex: Bad address (in wait_for_sessiond() at lttng-ust-comm.c:636) > libust[3384/3387]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) > libust[3384/3388]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3384/3388]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3384/3388]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3384/3388]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3384/3388]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3384/3388]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > liblttng_ust_tracepoint[3384/3388]: Registering probe to tracepoint lttng_ust:metadata (in __tracepoint_probe_register() at tracepoint.c:426) > libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3384/3388]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3384/3388]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3384/3388]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3384/3388]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3384/3388]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3384/3388]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3384/3388]: message received > (in ust_listener_thread() at lttng-ust-comm.c:766) > libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) > libust[3384/3384]: just registered probe ust_tests_hello containing 2 events (in ltt_probe_register() at ltt-probes.c:109) > libust[3384/3384]: Registered event probe "ust_tests_hello:tptest" with signature "int, anint, int, netint, long *, values, char *, text, size_t, textlen, double, doublearg, float, floatarg" (in ltt_probe_register() at ltt-probes.c:118) > liblttng_ust_tracepoint[3384/3384]: Registering probe to tracepoint ust_tests_hello:tptest (in __tracepoint_probe_register() at tracepoint.c:426) > libust[3384/3384]: Registered event probe "ust_tests_hello:tptest_sighandler" with signature "" (in ltt_probe_register() at ltt-probes.c:118) > liblttng_ust_tracepoint[3384/3384]: just registered a tracepoints section from 0x804b6c4 and having 2 tracepoints (in tracepoint_register_lib() at tracepoint.c:638) > liblttng_ust_tracepoint[3384/3384]: registered tracepoint: ust_tests_hello:tptest_sighandler (in tracepoint_register_lib() at tracepoint.c:643) > liblttng_ust_tracepoint[3384/3384]: registered tracepoint: ust_tests_hello:tptest (in tracepoint_register_lib() at tracepoint.c:643) > > Breakpoint 1, main (argc=1, argv=0xbffff754) at hello.c:84 > 84 if (argc == 2) > Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.47.el6_2.12.i686 libuuid-2.17.2-12.4.el6.i686 > (gdb) p __tracepoint_ust_tests_hello___tptest > $1 = {name = 0x804a380 "ust_tests_hello:tptest", state = 1, probes = 0x804c258, tracepoint_provider_ref = 0x804b724, > signature = 0x8049240 "int, anint, int, netint, long *, values, char *, text, size_t, textlen, double, doublearg, float, floatarg", padding = '\000' } > > Here __tracepoint_ust_tests_hello___tptest.state and __tracepoint_ust_tests_hello___tptest.probes have been set, which means probe callback function wiil be called by API tracepoint. But the event has been disabled, is this necessary to call into the probe again? If this behavior is normal, what's suppoed to do with disable-event command? > > Any misunderstanding please fix me. Thanks disabling an event ends up calling, in the program: lttng-ust-abi.c:lttng_event_cmd() with "LTTNG_UST_DISABLE", which calls ltt_event_disable(). Looking at this function: int ltt_event_disable(struct ltt_event *event) { int old; if (event->chan == event->chan->session->metadata) return -EPERM; old = uatomic_xchg(&event->enabled, 0); if (!old) return -EEXIST; return 0; } shows us that disabling an event sets the "enabled" flag within the event (struct ltt_event) to 0. This flag is tested in include/lttng/ust-tracepoint-event.h: #undef TRACEPOINT_EVENT_CLASS #define TRACEPOINT_EVENT_CLASS(_provider, _name, _args, _fields) \ static void __event_probe__##_provider##___##_name(_TP_ARGS_DATA_PROTO(_args))\ [...] if (caa_unlikely(!CMM_ACCESS_ONCE(__chan->session->active))) \ return; \ if (caa_unlikely(!CMM_ACCESS_ONCE(__chan->enabled))) \ return; \ if (caa_unlikely(!CMM_ACCESS_ONCE(__event->enabled))) \ return; What you are looking at with the debugger are the tracepoint call sites, which still have a handler connected to them when you disable the event. So yes, in theory, we could lessen the overhead of the enable followed by disable case and ensure that we disconnect the probe from the call site, but diminishing the performance impact of this rare use-case has not been high on my priority list so far. Hoping that my explanation helps clarifying things, Thanks, Mathieu > > Best Regards > Zheng > > > > Thanks, > > Mathieu > > > > Thanks all for your useful info! > > > > Best regards > > Zheng > > > > > > _______________________________________________ > > > > lttng-dev mailing list > > > > lttng-dev at lists.lttng.org > > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > > > > > > -- > > > Mathieu Desnoyers > > > Operating System Efficiency R&D Consultant > > > EfficiOS Inc. > > > http://www.efficios.com > > > > > _______________________________________________ > > > lttng-dev mailing list > > > lttng-dev at lists.lttng.org > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > > > > -- > > Mathieu Desnoyers > > Operating System Efficiency R&D Consultant > > EfficiOS Inc. > > http://www.efficios.com > -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From pavan.anumula at sasken.com Thu Jun 21 11:24:22 2012 From: pavan.anumula at sasken.com (Pavan Anumula) Date: Thu, 21 Jun 2012 20:54:22 +0530 Subject: [lttng-dev] Empty traces for UST In-Reply-To: <4FDB4D5B.6050701@efficios.com> References: <47D610AD9C485E458630BA38C324D3B60113FB00FBB0@EXGMBX01.sasken.com> <4FDB4D5B.6050701@efficios.com> Message-ID: <47D610AD9C485E458630BA38C324D3B60113FB9D4B72@EXGMBX01.sasken.com> Hi David, We tried to debug lttng commands with gdbserver on the arm target from host gdb remote target session following were obersevations: 1. create Session gdbserver --remote-debug localhost:5560 arm-none-linux-gnueabi-lttng -vvv create sesAA >>passed with out and errors 2. Enable events for session gdbserver --remote-debug localhost:5560 arm-none-linux-gnueabi-lttng -vvv enable-event -u -a >> failed randomly with segment faults and memeory allocation error see below trace error >>> (gdb) n >>>312 goto error; >>> (gdb) n >>>318 handle = lttng_create_handle(session_name, &dom); >>> (gdb) s >>>lttng_create_handle (session_name=0x40015154 "", domain=0xbe810bbc) at lttng-ctl.c:395 >>>395 perror("malloc handle"); Some times it used to print: >>>/lib/ld-linux.so.3: No such file or directory Or Segment faults Here is the /proc/meminfo of device: root at arago:~# cat /proc/meminfo MemTotal: 28220 kB MemFree: 12704 kB Buffers: 0 kB Cached: 7520 kB SwapCached: 0 kB Active: 4804 kB Inactive: 4984 kB Active(anon): 2296 kB Inactive(anon): 84 kB Active(file): 2508 kB Inactive(file): 4900 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 0 kB SwapFree: 0 kB Dirty: 0 kB Writeback: 0 kB AnonPages: 2296 kB Mapped: 2000 kB Shmem: 112 kB Slab: 3188 kB SReclaimable: 808 kB SUnreclaim: 2380 kB KernelStack: 576 kB PageTables: 288 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 14108 kB Committed_AS: 47704 kB VmallocTotal: 985088 kB VmallocUsed: 400 kB VmallocChunk: 984612 kB Please suggest how to overcome this issue. Regards, Pavan -----Original Message----- From: David Goulet [mailto:dgoulet at efficios.com] Sent: Friday, June 15, 2012 8:28 PM To: Pavan Anumula Cc: lttng-dev at lists.lttng.org Subject: Re: [lttng-dev] Empty traces for UST -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Pavan, Glad for the kernel! :) For UST, I'll recommend you start a session daemon like so "lttng-sessiond - -vvv". After that, use the lttng-ust/tests/hello program and execute it. You should see on the debug output of the session daemon that the application "hello" has registered and is ready for tracing. Your output below does NOT show those debug messages so it would be a good start to see if there is a problem at that point. Also, make sure that the liblttng-ust is installed upon compiling lttng-tools so that the user space tracer support is enabled in the tools. Thanks! David On 15/06/12 10:50 AM, Pavan Anumula wrote: > Hi David, > > Finally I was able to get kernel traces by referring to one of your > other posts , Actually lttng-consumerd binary was actually not > present at the needed path, Thanks for your help. > > And sorry for bugging you with mails :) . > > > But this time I am facing issues at UST tracing . Please kindly help > me on this. > > When I try to trace for User space it is creating an empty folder with > no traces on ARM target, I tried with all the sample trace programs in > LTTng-UST/tests folder. > > I am using NFS to load binaries and for tracing will that causes problem. > Please also find the debug logs attached, > > Please also let me know if you want some more information. > > > ************************************** fcntl64(12, F_SETFD, FD_CLOEXEC) > = 0 fcntl64(13, F_SETFD, FD_CLOEXEC) = 0 pipe([14, 15]) > = 0 fcntl64(14, F_SETFD, FD_CLOEXEC) = 0 fcntl64(15, F_SETFD, > FD_CLOEXEC) = 0 getrlimit(RLIMIT_NOFILE, {rlim_cur=65535, > rlim_max=65535}) = 0 write(2, "DEBUG1: poll set max size set to"..., > 92DEBUG1: poll set max size set to 65535 [in > compat_poll_set_max_size() at compat-poll.c:196] ) = 92 mmap2(NULL, > 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4024b000 mprotect(0x4024b000, 4096, > PROT_NONE) = 0 clone(child_stack=0x40a49ef8, > flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_S > YSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, > parent_tidptr=0x40a4a3e8, tls=0x40a4a840, child_tidptr=0x40a4a3e8) = > 921 mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) = 0x40a4b000 mprotect(0x40a4b000, 4096, PROT_NONE) = 0 > clone(child_stack=0x41249ef8, > flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_S > YSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, > parent_tidptr=0x4124a3e8, tls=0x4124a840, child_tidptr=0x4124a3e8) = > 922 mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) = 0x4124b000 mprotect(0x4124b000, 4096, PROT_NONE) = 0 > clone(child_stack=0x41a49ef8, > flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_S > YSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, > parent_tidptr=0x41a4a3e8, tls=0x41a4a840, child_tidptr=0x41a4a3e8) = > 923 mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) = 0x41a4b000 mprotect(0x41a4b000, 4096, PROT_NONE) = 0 clone(DEBUG1: > [thread] Dispatch UST command started [in > thread_dispatch_ust_registration() at main.c:1324] DEBUG1: Futex n to > 1 prepare done [in futex_nto1_prepare() at futex.c:73] DEBUG1: Woken > up but nothing in the UST command queue [in > thread_dispatch_ust_registration() at main.c:1334] DEBUG1: [thread] > Manage application registration started [in > thread_registration_apps() at main.c:1392] DEBUG1: fd 3 of 1 added to > pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 11 of 2 > added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: > Notifying applications of session daemon state: 1 [in > notify_ust_apps() at main.c:687] DEBUG1: Got the wait shm fd 16 [in > get_wait_shm() at shm.c:117] DEBUG1: Futex wait update active 1 [in > futex_wait_update() at futex.c:62] DEBUG1: Accepting application > registration [in > thread_registration_apps() at main.c:1423] DEBUG1: [thread] Manage > application started [in thread_manage_apps() at main.c:1179] DEBUG1: > fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: fd > 14 of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: > Apps thread polling on 2 fds [in thread_manage_apps() at main.c:1200] > DEBUG1: [thread] Manage client started [in thread_manage_clients() at > main.c:3794] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() > at compat-poll.c:91] DEBUG1: fd 10 of 2 added to pollfd [in > compat_poll_add() at compat-poll.c:91] DEBUG1: Accepting client > command ... [in > thread_manage_clients() at main.c:3826] child_stack=0x42249ef8, > flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_S > YSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, > parent_tidptr=0x4224a3e8, tls=0x4224a840, child_tidptr=0x4224a3e8) = > 924 mmap2(NULL, 8388608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) = 0x4224b000 mprotect(0x4224b000, 4096, PROT_NONE) = 0 > clone(child_stack=0x42a49ef8, > flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_S > YSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, > parent_tidptr=0x42a4a3e8, tls=0x42a4a840, child_tidptr=0x42a4a3e8) = > 925 futex(0x42a4a3e8, FUTEX_WAIT, 925, NULLDEBUG1: Thread manage > kernel started [in thread_manage_kernel() at main.c:876] DEBUG1: fd 3 > of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] > DEBUG1: fd 12 of 2 added to pollfd [in compat_poll_add() at > compat-poll.c:91] DEBUG1: Updating kernel poll set [in > update_kernel_poll() at main.c:748] DEBUG1: Thread kernel polling on 2 > fds [in thread_manage_kernel() at main.c:905] DEBUG1: Wait for client response [in thread_manage_clients() at main.c:3863] DEBUG1: > Receiving data from client ... [in thread_manage_clients() at > main.c:3898] > DEBUG1: Nothing recv() from client... continuing [in > thread_manage_clients() at main.c:3902] DEBUG1: Clean command context > structure [in clean_command_ctx() at main.c:534] DEBUG1: Accepting > client command ... [in thread_manage_clients() at main.c:3826] DEBUG1: > Wait for client response [in thread_manage_clients() at main.c:3863] DEBUG1: > Receiving data from client ... [in thread_manage_clients() at > main.c:3898] > DEBUG1: Processing client command 8 [in process_client_msg() at > main.c:3309] DEBUG2: Trying to find session by name sesCCC [in > session_find_by_name() at session.c:126] DEBUG3: mkdir() recursive > /home/root/lttng-traces/sesCCC-20110327-045731 with mode 504 for uid 0 > and gid 0 [in run_as_mkdir_recursive() at runas.c:338] DEBUG1: Using > run_as_clone [in run_as() at runas.c:322] DEBUG1: Tracing session > sesCCC created in /home/root/lttng-traces/sesCCC-20110327-045731 with > ID 1 by UID > 0 GID 0 [in session_create() at session.c:234] DEBUG1: Sending > response > (size: 16, retcode: Success) [in thread_manage_clients() at > main.c:3936] > DEBUG1: Clean command context structure [in clean_command_ctx() at > main.c:534] DEBUG1: Accepting client command ... [in > thread_manage_clients() at main.c:3826] DEBUG1: Wait for client > response [in thread_manage_clients() at main.c:3863] DEBUG1: Receiving > data from client ... [in thread_manage_clients() at main.c:3898] > DEBUG1: Nothing > recv() from client... continuing [in thread_manage_clients() at > main.c:3902] DEBUG1: Clean command context structure [in > clean_command_ctx() at main.c:534] DEBUG1: Accepting client command > ... [in > thread_manage_clients() at main.c:3826] DEBUG1: Wait for client > response [in thread_manage_clients() at main.c:3863] DEBUG1: Receiving > data from client ... [in thread_manage_clients() at main.c:3898] > DEBUG1: Processing client command 6 [in process_client_msg() at > main.c:3309] DEBUG1: Getting session sesCCC by name [in process_client_msg() at main.c:3364] DEBUG2: > Trying to find session by name sesCCC [in session_find_by_name() at > session.c:126] DEBUG1: Creating UST session [in create_ust_session() > at main.c:1965] DEBUG3: Created hashtable size 4 at 0x4fa08 of type 1 > [in > lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 at > 0x4fb28 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: > Created hashtable size 4 at 0x4fc48 of type 0 [in lttng_ht_new() at > hashtable.c:96] DEBUG2: UST trace session create successful [in > trace_ust_create_session() at trace-ust.c:119] DEBUG3: mkdir() > recursive /home/root/lttng-traces/sesCCC-20110327-045731/ust with mode > 504 for uid 0 and gid 0 [in run_as_mkdir_recursive() at runas.c:338] > DEBUG1: Using run_as_clone [in run_as() at runas.c:322] DEBUG1: > Spawning consumerd [in > spawn_consumerd() at main.c:1659] DEBUG1: Using 32-bit UST consumer at: > /root/armfilesystem/lib/lttng/libexec/lttng-consumerd [in > spawn_consumerd() at main.c:1777] DEBUG2: Consumer pid 934 [in > start_consumerd() at main.c:1830] DEBUG2: Spawning consumer control > thread [in start_consumerd() at main.c:1833] DEBUG1: [thread] Manage > consumer started [in > thread_manage_consumer() at main.c:982] DEBUG1: fd 3 of 1 added to > pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 9 of 2 > added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG2: > Receiving code from consumer err_sock [in thread_manage_consumer() at main.c:1043] DEBUG1: > consumer command socket ready [in thread_manage_consumer() at > main.c:1062] > DEBUG1: fd 17 of 2 added to pollfd [in compat_poll_add() at > compat-poll.c:91] DEBUG2: Trace UST channel channel0 not found by name > [in > trace_ust_find_channel_by_name() at trace-ust.c:52] DEBUG3: Created > hashtable size 4 at 0x51310 of type 0 [in lttng_ht_new() at > hashtable.c:96] DEBUG3: Created hashtable size 4 at 0x51430 of type 1 > [in > lttng_ht_new() at hashtable.c:96] DEBUG2: Trace UST channel channel0 > created [in trace_ust_create_channel() at trace-ust.c:181] DEBUG2: > Channel > channel0 being created in UST global domain [in channel_ust_create() > at channel.c:267] DEBUG2: UST app adding channel channel0 to global > domain for session id 1 [in ust_app_create_channel_glb() at ust-app.c:1792] DEBUG2: > Channel channel0 created successfully [in channel_ust_create() at > channel.c:292] DEBUG2: Trace UST channel channel0 found by name [in > trace_ust_find_channel_by_name() at trace-ust.c:47] DEBUG2: Trace UST > event NOT found by name * [in trace_ust_find_event_by_name() at > trace-ust.c:79] > DEBUG3: Created hashtable size 4 at 0x51550 of type 1 [in > lttng_ht_new() at hashtable.c:96] DEBUG2: Trace UST event *, loglevel > (0,-1) created [in > trace_ust_create_event() at trace-ust.c:260] DEBUG1: UST app creating > event > * for all apps for session id 1 [in ust_app_create_event_glb() at > ust-app.c:1916] DEBUG1: Event UST * created in channel channel0 [in > event_ust_enable_tracepoint() at event.c:446] DEBUG1: Sending response > (size: 16, retcode: Success) [in thread_manage_clients() at > main.c:3936] > DEBUG1: Clean command context structure [in clean_command_ctx() at > main.c:534] DEBUG1: Accepting client command ... [in > thread_manage_clients() at main.c:3826] DEBUG1: Wait for client > response [in thread_manage_clients() at main.c:3863] DEBUG1: Receiving > data from client ... [in thread_manage_clients() at main.c:3898] > DEBUG1: Nothing > recv() from client... continuing [in thread_manage_clients() at > main.c:3902] DEBUG1: Clean command context structure [in > clean_command_ctx() at main.c:534] DEBUG1: Accepting client command > ... [in > thread_manage_clients() at main.c:3826] DEBUG1: Wait for client > response [in thread_manage_clients() at main.c:3863] DEBUG1: Receiving > data from client ... [in thread_manage_clients() at main.c:3898] > DEBUG1: Processing client command 16 [in process_client_msg() at > main.c:3309] DEBUG1: Getting session sesCCC by name [in process_client_msg() at main.c:3364] DEBUG2: > Trying to find session by name sesCCC [in session_find_by_name() at > session.c:126] DEBUG1: Starting all UST traces [in > ust_app_start_trace_all() at ust-app.c:2207] DEBUG1: Sending response > (size: 16, retcode: Success) [in thread_manage_clients() at > main.c:3936] > DEBUG1: Clean command context structure [in clean_command_ctx() at > main.c:534] DEBUG1: Accepting client command ... [in > thread_manage_clients() at main.c:3826] > ************************************************ > > > > > > Thanks in Advance, Pavan > > > -----Original Message----- From: David Goulet > [mailto:dgoulet at efficios.com] Sent: Tuesday, June 12, 2012 9:05 PM To: > Pavan Anumula Cc: lttng-dev at lists.lttng.org Subject: Re: [lttng-dev] > Lttng start failed > > Hi Pavan, > > The output below is normal and should be working fine. However, the > problem is that you have _NO_ debug message after the "lttng start". > That command should at least produce 10+ lines of messages after this last line: > > "DEBUG1: Accepting application registration [in > thread_registration_apps() at main.c:1423]" > > Can you enlight me and tell me if by doing "lttng create newsessiom" > and "lttng enable-event -a -k", you have output messages coming out of > the session daemon in -vvv mode ? > > If NOT, you are talking to another daemon! A quick "ps faux | grep > lttng-sessiond" could help clear that question. > > If only one daemon is running and stuck at the above debug message, > the next step is to tell me on what the daemon is waiting on using > either "strace -p" or "gdb" also. There is 6 threads so having the > strace -p output for all of them would help me greatly. > > Thanks! David > > On 12/06/12 10:05 AM, Mathieu Desnoyers wrote: >> * Pavan Anumula (pavan.anumula at sasken.com) wrote: >>> Hi Mathieu, Still I am unable to start tracing, And I am facing the >>> same start errors. Please kindly help me on this. This time I >>> loaded the modules by modprobe( not with insmod as I did before), >>> Then I run the command " arm-none-linux-gnueabi-lttng-sessiond >>> -vvv " (before arm-none-linux-gnueabi-lttng-sessiond -d), > >> When using "arm-none-linux-gnueabi-lttng-sessiond -vvv", you should >> _not_ run "arm-none-linux-gnueabi-lttng-sessiond -d". > > >>> The below is the Output, And It got hanged overthere. > >> This is normal: the sessiond is waiting for applications to connect. >> It does not hang, it just waits. > >> David, can you follow up on this issue ? It seems to be >> sessiond-related. > >> Thanks, > >> Mathieu > > >>> Later when I started lttng start I am acing the same erros below: >>> root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng start LTTng: >>> Failure to write metadata to buffers (timeout) Error: Starting >>> kernel trace failed >>> >>> ******************************************************************** >>> * >>> *************************** >>> >>> > DEBUG3: Creating LTTng run directory: /var/run/lttng [in > create_lttng_rundir() at main.c:4315] >>> DEBUG2: Kernel consumer err path: /var/run/lttng/kconsumerd/error >>> [in >>> main() at main.c:4543] DEBUG2: Kernel consumer cmd path: >>> /var/run/lttng/kconsumerd/command [in main() at main.c:4545] DEBUG1: >>> Client socket path /var/run/lttng/client-lttng-sessiond [in main() >>> at main.c:4592] DEBUG1: Application socket path >>> /var/run/lttng/apps-lttng-sessiond [in main() at main.c:4593] DEBUG1: >>> LTTng run directory path: /var/run/lttng [in main() at main.c:4594] >>> DEBUG2: UST consumer 32 bits err path: >>> /var/run/lttng/ustconsumerd32/error [in main() at main.c:4603] DEBUG2: >>> UST consumer 32 bits cmd path: /var/run/lttng/ustconsumerd32/command >>> [in main() at main.c:4605] DEBUG2: UST consumer 64 bits err path: >>> /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] DEBUG2: >>> UST consumer 64 bits cmd path: /var/run/lttng/ustconsumerd64/command >>> [in main() at main.c:4616] DEBUG3: Created hashtable size 4 at >>> 0x4a080 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG3: >>> Created hashtable size 4 at 0x4a168 of type 1 [in lttng_ht_new() at >>> hashtable.c:96] DEBUG2: Creating consumer directory: >>> /var/run/lttng/kconsumerd [in set_consumer_sockets() at main.c:4357] >>> DEBUG1: Modprobe successfully lttng-tracer [in >>> modprobe_lttng_control() at modprobe.c:163] DEBUG2: Kernel tracer >>> version validated (major version 2) [in kernel_validate_version() at kernel.c:675] DEBUG1: >>> Modprobe successfully lttng-ftrace [in modprobe_lttng_data() at >>> modprobe.c:199] DEBUG1: Modprobe successfully lttng-kprobes [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>> successfully lttng-kretprobes [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: >>> Modprobe successfully lttng-lib-ring-buffer [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>> successfully lttng-ring-buffer-client-discard [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>> successfully lttng-ring-buffer-client-overwrite [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>> successfully lttng-ring-buffer-metadata-client [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>> successfully lttng-ring-buffer-client-mmap-discard [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>> successfully lttng-ring-buffer-client-mmap-overwrite [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>> successfully lttng-ring-buffer-metadata-mmap-client [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>> successfully lttng-probe-lttng [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>> successfully lttng-types [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: >>> Modprobe successfully lttng-probe-block [in modprobe_lttng_data() at >>> modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-irq [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>> successfully lttng-probe-kvm [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: >>> Modprobe successfully lttng-probe-sched [in modprobe_lttng_data() at >>> modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-signal [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>> successfully lttng-probe-statedump [in modprobe_lttng_data() at >>> modprobe.c:199] >>> DEBUG1: Modprobe successfully lttng-probe-timer [in >>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Kernel tracer fd 6 >>> [in >>> init_kernel_tracer() at main.c:1887] DEBUG2: Creating consumer >>> directory: /var/run/lttng/ustconsumerd64 [in set_consumer_sockets() >>> at main.c:4357] DEBUG2: Creating consumer directory: >>> /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() at >>> main.c:4357] DEBUG1: Signal handler set for SIGTERM, SIGPIPE and >>> SIGINT [in set_signal_handler() at main.c:4449] DEBUG1: All >>> permissions are set [in set_permissions() at main.c:4250] DEBUG1: >>> poll set max size set to 65535 [in compat_poll_set_max_size() at compat-poll.c:196] DEBUG1: >>> [thread] Manage application started [in thread_manage_apps() at >>> main.c:1179] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() >>> at compat-poll.c:91] DEBUG1: fd 13 of 2 added to pollfd [in >>> compat_poll_add() at compat-poll.c:91] DEBUG1: Apps thread polling >>> on 2 fds [in thread_manage_apps() at main.c:1200] DEBUG1: Thread >>> manage kernel started [in thread_manage_kernel() at main.c:876] >>> DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: >>> fd 11 of 2 added to pollfd [in compat_poll_add() at >>> compat-poll.c:91] >>> DEBUG1: Updating kernel poll set [in update_kernel_poll() at >>> main.c:748] DEBUG1: Thread kernel polling on 2 fds [in >>> thread_manage_kernel() at main.c:905] DEBUG1: [thread] Manage >>> application registration started [in thread_registration_apps() at >>> main.c:1392] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() >>> at compat-poll.c:91] DEBUG1: fd 10 of 2 added to pollfd [in >>> compat_poll_add() at compat-poll.c:91] DEBUG1: Notifying >>> applications of session daemon state: 1 [in notify_ust_apps() at >>> main.c:687] >>> DEBUG1: [thread] Dispatch UST command started [in >>> thread_dispatch_ust_registration() at main.c:1324] DEBUG1: Futex n >>> to 1 prepare done [in futex_nto1_prepare() at futex.c:73] DEBUG1: >>> Woken up but nothing in the UST command queue [in >>> thread_dispatch_ust_registration() at main.c:1334] DEBUG1: [thread] >>> Manage client started [in thread_manage_clients() at main.c:3794] >>> DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at >>> compat-poll.c:91] DEBUG1: fd 9 of 2 added to pollfd [in >>> compat_poll_add() at compat-poll.c:91] DEBUG1: Accepting client >>> command ... [in thread_manage_clients() at main.c:3826] DEBUG1: Got >>> the wait shm fd 15 [in get_wait_shm() at shm.c:117] DEBUG1: Futex >>> wait update active 1 [in futex_wait_update() at futex.c:62] DEBUG1: >>> Accepting application registration [in thread_registration_apps() at >>> main.c:1423] >>> ******************************************************************** >>> * >>> *********************************** >>> >>> > Am I missing anything?? >>> >>> Thanks in Advance, Pavan >>> >>> >>> ________________________________________ From: Mathieu Desnoyers >>> [mathieu.desnoyers at efficios.com] Sent: Monday, June 11, 2012 7:24 PM >>> To: Pavan Anumula Cc: lttng-dev at lists.lttng.org Subject: Re: >>> [lttng-dev] Lttng start failed >>> >>> * Pavan Anumula (pavan.anumula at sasken.com) wrote: >>>> Hi Mathieu, >>>> >>>> Please find the info below as per your comments >>>> >>>> >>>> ########## Output of command >>>> "arm-none-linux-gnueabi-lttng-sessiond >>>> -vvv " , After loading the modules ####### >>>> >>>> DEBUG3: Creating LTTng run directory: /var/run/lttng [in >>>> create_lttng_rundir() at main.c:4315] DEBUG2: Kernel consumer err >>>> path: /var/run/lttng/kconsumerd/error [in main() at main.c:4543] >>>> DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command >>>> [in main() at main.c:4545] DEBUG1: Client socket path >>>> /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] >>>> DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond >>>> [in main() at main.c:4593] DEBUG1: LTTng run directory path: >>>> /var/run/lttng [in main() at main.c:4594] DEBUG2: UST consumer 32 >>>> bits err path: /var/run/lttng/ustconsumerd32/error [in main() at >>>> main.c:4603] DEBUG2: UST consumer 32 bits cmd path: >>>> /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] >>>> DEBUG2: UST consumer 64 bits err path: >>>> /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] >>>> DEBUG2: UST consumer 64 bits cmd path: >>>> /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] >>>> DEBUG3: Created hashtable size 4 at 0x4a080 of type 1 [in >>>> lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 >>>> at >>>> 0x4a168 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: >>>> Creating consumer directory: /var/run/lttng/kconsumerd [in >>>> set_consumer_sockets() at main.c:4357] FATAL: Module lttng_tracer >>>> not found. Error: Unable to load module lttng-tracer >>> >>> Hrm. You want to do kernel tracing, but modprobe cannot find the >>> lttng kernel tracer modules. You might want to run depmod -a or >>> something like that on your target, and ensure that modprobe works properly. >>> >>> Thanks, >>> >>> Mathieu >>> >>>> DEBUG2: Kernel tracer version validated (major version 2) [in >>>> kernel_validate_version() at kernel.c:675] DEBUG1: Modprobe >>>> successfully lttng-ftrace [in modprobe_lttng_data() at >>>> modprobe.c:199] DEBUG1: Modprobe successfully lttng-kprobes [in >>>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>>> successfully lttng-kretprobes [in modprobe_lttng_data() at >>>> modprobe.c:199] FATAL: Module lttng_lib_ring_buffer not found. Error: >>>> Unable to load module lttng-lib-ring-buffer FATAL: Module >>>> lttng_ring_buffer_client_discard not found. Error: Unable to load >>>> module lttng-ring-buffer-client-discard FATAL: Module >>>> lttng_ring_buffer_client_overwrite not found. Error: Unable to load >>>> module lttng-ring-buffer-client-overwrite FATAL: Module >>>> lttng_ring_buffer_metadata_client not found. Error: Unable to load >>>> module lttng-ring-buffer-metadata-client FATAL: Module >>>> lttng_ring_buffer_client_mmap_discard not found. Error: Unable to >>>> load module lttng-ring-buffer-client-mmap-discard FATAL: Module >>>> lttng_ring_buffer_client_mmap_overwrite not found. Error: Unable to >>>> load module lttng-ring-buffer-client-mmap-overwrite FATAL: Module >>>> lttng_ring_buffer_metadata_mmap_client not found. Error: Unable to >>>> load module lttng-ring-buffer-metadata-mmap-client FATAL: Module >>>> lttng_probe_lttng not found. Error: Unable to load module >>>> lttng-probe-lttng DEBUG1: Modprobe successfully lttng-types [in >>>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>>> successfully lttng-probe-block [in modprobe_lttng_data() at >>>> modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-irq [in >>>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>>> successfully lttng-probe-kvm [in modprobe_lttng_data() at >>>> modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-sched [in >>>> modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>>> successfully lttng-probe-signal [in modprobe_lttng_data() at >>>> modprobe.c:199] DEBUG1: Modprobe successfully lttng-probe-statedump >>>> [in modprobe_lttng_data() at modprobe.c:199] DEBUG1: Modprobe >>>> successfully lttng-probe-timer [in modprobe_lttng_data() at >>>> modprobe.c:199] DEBUG1: Kernel tracer fd 6 [in init_kernel_tracer() >>>> at main.c:1887] DEBUG2: Creating consumer directory: >>>> /var/run/lttng/ustconsumerd64 [in set_consumer_sockets() at >>>> main.c:4357] DEBUG2: Creating consumer directory: >>>> /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() at >>>> main.c:4357] DEBUG1: Signal handler set for SIGTERM, SIGPIPE and >>>> SIGINT [in set_signal_handler() at main.c:4449] DEBUG1: All >>>> permissions are set [in set_permissions() at main.c:4250] DEBUG1: >>>> poll set max size set to 65535 [in compat_poll_set_max_size() at >>>> compat-poll.c:196] DEBUG1: [thread] Manage client started [in >>>> thread_manage_clients() at main.c:3794] DEBUG1: fd 3 of 1 added to >>>> pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 9 of 2 >>>> added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: >>>> Accepting client command ... [in thread_manage_clients() at >>>> main.c:3826] DEBUG1: [thread] Dispatch UST command started [in >>>> thread_dispatch_ust_registration() at main.c:1324] DEBUG1: Futex n >>>> to >>>> 1 prepare done [in futex_nto1_prepare() at futex.c:73] DEBUG1: >>>> Woken up but nothing in the UST command queue [in >>>> thread_dispatch_ust_registration() at main.c:1334] DEBUG1: Thread >>>> manage kernel started [in thread_manage_kernel() at main.c:876] >>>> DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() at >>>> compat-poll.c:91] DEBUG1: fd 11 of 2 added to pollfd [in >>>> compat_poll_add() at compat-poll.c:91] DEBUG1: Updating kernel poll >>>> set [in update_kernel_poll() at main.c:748] DEBUG1: Thread kernel >>>> polling on 2 fds [in thread_manage_kernel() at main.c:905] DEBUG1: >>>> [thread] Manage application started [in thread_manage_apps() at >>>> main.c:1179] DEBUG1: fd 3 of 1 added to pollfd [in >>>> compat_poll_add() at compat-poll.c:91] DEBUG1: fd 13 of 2 added to >>>> pollfd [in >>>> compat_poll_add() at compat-poll.c:91] DEBUG1: Apps thread polling >>>> on >>>> 2 fds [in thread_manage_apps() at main.c:1200] DEBUG1: [thread] >>>> Manage application registration started [in >>>> thread_registration_apps() at main.c:1392] DEBUG1: fd 3 of 1 added >>>> to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 10 >>>> of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: >>>> Notifying applications of session daemon state: 1 [in >>>> notify_ust_apps() at main.c:687] DEBUG1: Got the wait shm fd 15 [in >>>> get_wait_shm() at shm.c:117] DEBUG1: Futex wait update active 1 [in >>>> futex_wait_update() at futex.c:62] DEBUG1: Accepting application >>>> registration [in thread_registration_apps() at main.c:1423] >>>> >>>> >>>> >>>> #### Output of command "arm-none-linux-gnueabi-lttng-sessiond -vvv >>>> " before loading the modules ############# >>>> >>>> DEBUG3: Creating LTTng run directory: /var/run/lttng [in >>>> create_lttng_rundir() at main.c:4315] DEBUG2: Kernel consumer err >>>> path: /var/run/lttng/kconsumerd/error [in main() at main.c:4543] >>>> DEBUG2: Kernel consumer cmd path: /var/run/lttng/kconsumerd/command >>>> [in main() at main.c:4545] DEBUG1: Client socket path >>>> /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] >>>> DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond >>>> [in main() at main.c:4593] DEBUG1: LTTng run directory path: >>>> /var/run/lttng [in main() at main.c:4594] DEBUG2: UST consumer 32 >>>> bits err path: /var/run/lttng/ustconsumerd32/error [in main() at >>>> main.c:4603] DEBUG2: UST consumer 32 bits cmd path: >>>> /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] >>>> DEBUG2: UST consumer 64 bits err path: >>>> /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] >>>> DEBUG2: UST consumer 64 bits cmd path: >>>> /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] >>>> DEBUG3: Created hashtable size 4 at 0x4a080 of type 1 [in >>>> lttng_ht_new() at hashtable.c:96] DEBUG3: Created hashtable size 4 >>>> at >>>> 0x4a168 of type 1 [in lttng_ht_new() at hashtable.c:96] DEBUG2: >>>> Creating consumer directory: /var/run/lttng/kconsumerd [in >>>> set_consumer_sockets() at main.c:4357] FATAL: Module lttng_tracer >>>> not found. Error: Unable to load module lttng-tracer DEBUG1: Failed >>>> to open /proc/lttng [in init_kernel_tracer() at main.c:1871] Error: >>>> Unable to remove module lttng-tracer Warning: No kernel tracer >>>> available DEBUG2: Creating consumer directory: >>>> /var/run/lttng/ustconsumerd64 [in set_consumer_sockets() at >>>> main.c:4357] DEBUG2: Creating consumer directory: >>>> /var/run/lttng/ustconsumerd32 [in set_consumer_sockets() at >>>> main.c:4357] DEBUG1: Signal handler set for SIGTERM, SIGPIPE and >>>> SIGINT [in set_signal_handler() at main.c:4449] DEBUG1: All >>>> permissions are set [in set_permissions() at main.c:4250] DEBUG1: >>>> poll set max size set to 65535 [in compat_poll_set_max_size() at >>>> compat-poll.c:196] DEBUG1: [thread] Manage application started [in >>>> thread_manage_apps() at main.c:1179] DEBUG1: fd 3 of 1 added to >>>> pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 12 of >>>> 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: >>>> Apps thread polling on 2 fds [in thread_manage_apps() at >>>> main.c:1200] >>>> DEBUG1: Thread manage kernel started [in thread_manage_kernel() at >>>> main.c:876] DEBUG1: fd 3 of 1 added to pollfd [in compat_poll_add() >>>> at compat-poll.c:91] DEBUG1: fd 10 of 2 added to pollfd [in >>>> compat_poll_add() at compat-poll.c:91] DEBUG1: Updating kernel poll >>>> set [in update_kernel_poll() at main.c:748] DEBUG1: Thread kernel >>>> polling on 2 fds [in thread_manage_kernel() at main.c:905] DEBUG1: >>>> [thread] Manage application registration started [in >>>> thread_registration_apps() at main.c:1392] DEBUG1: fd 3 of 1 added >>>> to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 9 >>>> of 2 added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: >>>> Notifying applications of session daemon state: 1 [in >>>> notify_ust_apps() at main.c:687] DEBUG1: [thread] Dispatch UST >>>> command started [in thread_dispatch_ust_registration() at >>>> main.c:1324] DEBUG1: Futex n to 1 prepare done [in >>>> futex_nto1_prepare() at futex.c:73] DEBUG1: Woken up but nothing in >>>> the UST command queue [in thread_dispatch_ust_registration() at >>>> main.c:1334] DEBUG1: [thread] Manage client started [in >>>> thread_manage_clients() at main.c:3794] DEBUG1: fd 3 of 1 added to >>>> pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: fd 8 of 2 >>>> added to pollfd [in compat_poll_add() at compat-poll.c:91] DEBUG1: >>>> Accepting client command ... [in thread_manage_clients() at >>>> main.c:3826] DEBUG1: Got the wait shm fd 14 [in get_wait_shm() at >>>> shm.c:117] DEBUG1: Futex wait update active 1 [in >>>> futex_wait_update() at futex.c:62] DEBUG1: Accepting application >>>> registration [in >>>> thread_registration_apps() at main.c:1423] >>>> >>>> >>>> Regards, Pavan >>>> >>>> -----Original Message----- From: Mathieu Desnoyers >>>> [mailto:mathieu.desnoyers at efficios.com] Sent: Saturday, June 09, >>>> 2012 6:03 PM To: Pavan Anumula Cc: lttng-dev at lists.lttng.org Subject: >>>> Re: [lttng-dev] Lttng start failed >>>> >>>> * Pavan Anumula (pavan.anumula at sasken.com) wrote: >>>>> Hi Mathue, >>>>> >>>>> Thanks for the quick reply, >>>>> >>>>> After inserting lttng modules , I had given the command >>>>> "arm-none-linux-gnueabi-lttng-sessiond -vvv" as you said, Below >>>>> is the output where there are error messages. Please help me in >>>>> resolving the issue, SO that I can catch kernel and user traces. >>>>> >>>>> >>>>> root at arago:/usr/lttng/modules# >>>>> arm-none-linux-gnueabi-lttng-sessiond -d >>>>> root at arago:/usr/lttng/modules# >>>>> arm-none-linux-gnueabi-lttng-sessiond -vvv DEBUG3: Creating LTTng >>>>> run directory: /var/run/lttng [in create_lttng_rundir() at >>>>> main.c:4315] DEBUG2: Kernel consumer err path: >>>>> /var/run/lttng/kconsumerd/error [in main() at main.c:4543] DEBUG2: >>>>> Kernel consumer cmd path: /var/run/lttng/kconsumerd/command [in >>>>> main() at main.c:4545] DEBUG1: Client socket path >>>>> /var/run/lttng/client-lttng-sessiond [in main() at main.c:4592] >>>>> DEBUG1: Application socket path /var/run/lttng/apps-lttng-sessiond >>>>> [in main() at main.c:4593] DEBUG1: LTTng run directory path: >>>>> /var/run/lttng [in main() at main.c:4594] DEBUG2: UST consumer 32 >>>>> bits err path: /var/run/lttng/ustconsumerd32/error [in main() at >>>>> main.c:4603] DEBUG2: UST consumer 32 bits cmd path: >>>>> /var/run/lttng/ustconsumerd32/command [in main() at main.c:4605] >>>>> DEBUG2: UST consumer 64 bits err path: >>>>> /var/run/lttng/ustconsumerd64/error [in main() at main.c:4614] >>>>> DEBUG2: UST consumer 64 bits cmd path: >>>>> /var/run/lttng/ustconsumerd64/command [in main() at main.c:4616] >>>>> Error: Already running daemon. >>>> >>>> Please kill the lttng-sessiond that is already started (the one >>>> with -d). Instead of that one, run the sessiond with: >>>> >>>> lttng-sessiond -vvv >>>> >>>> (don't run lttng-sessiond -d before) >>>> >>>> Thanks, >>>> >>>> Mathieu >>>> >>>>> >>>>> >>>>> >>>>> >>>>> Thanks in advance, Pavan >>>>> >>>>> -----Original Message----- From: Mathieu Desnoyers >>>>> [mailto:mathieu.desnoyers at efficios.com] Sent: Friday, June 08, >>>>> 2012 11:12 PM To: Pavan Anumula Cc: lttng-dev at lists.lttng.org >>>>> Subject: Re: [lttng-dev] Lttng start failed >>>>> >>>>> * Pavan Anumula (pavan.anumula at sasken.com) wrote: >>>>>> Hi , >>>>>> >>>>>> I am new to LTTng usage, I am trying to use LTTng 2.0.1 and >>>>>> LTTng-modules-2.0.2 for ARM(omapL138), with Linux kernel 2.6.33 >>>>>> wit RT-29 patch on it. >>>>>> >>>>>> After enabling all the kernel events I am facing the below >>>>>> errors. I loaded all the modules manually by insmod. >>>>>> >>>>>> >>>>>> >>>>>> root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng create >>>>>> newsessiom Session newsessiom created. Traces will be written in >>>>>> /home/root/lttng-traces/newsessiom-20110325-154922 >>>>>> >>>>>> root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng >>>>>> enable-event -a --kernel All kernel events are enabled in channel >>>>>> channel0 >>>>>> >>>>>> root at arago:~/lttng-traces# arm-none-linux-gnueabi-lttng start >>>>>> LTTng: Failure to write metadata to buffers (timeout) Error: >>>>>> Starting kernel trace failed >>>>>> >>>>>> Please kindly help me on this. >>>>> >>>>> I think you should look into the lttng-sessiond --help : >>>>> >>>>> --consumerd32-path PATH Specify path for the 32-bit UST >>>>> consumer daemon binary --consumerd32-libdir PATH Specify path for >>>>> the 32-bit UST consumer daemon libraries --consumerd64-path PATH >>>>> Specify path for the 64-bit UST consumer daemon binary >>>>> --consumerd64-libdir PATH Specify path for the 64-bit UST >>>>> consumer daemon libraries >>>>> >>>>> options. My guess is that lttng-sessiond is not able to find the >>>>> consumerd binary files, maybe due to a rename or because they have >>>>> been moved after install. >>>>> >>>>> One more thing that might help is to launch the lttng-sessiond >>>>> with "-vvv" : it will provide verbose output and let us know where >>>>> things fall apart. >>>>> >>>>> Thanks, >>>>> >>>>> Mathieu >>>>> >>>>> >>>>>> >>>>>> >>>>>> Thanks in advance, Pavan >>>>>> >>>>>> ________________________________ SASKEN BUSINESS DISCLAIMER: >>>>>> This message may contain confidential, proprietary or legally >>>>>> privileged information. In case you are not the original intended >>>>>> Recipient of the message, you must not, directly or indirectly, >>>>>> use, disclose, distribute, print, or copy any part of this >>>>>> message and you are requested to delete it and inform the sender. >>>>>> Any views expressed in this message are those of the individual >>>>>> sender unless otherwise stated. Nothing contained in this message >>>>>> shall be construed as an offer or acceptance of any offer by >>>>>> Sasken Communication Technologies Limited ("Sasken") unless sent >>>>>> with that express intent and with due authority of Sasken. Sasken >>>>>> has taken enough precautions to prevent the spread of viruses. >>>>>> However the company accepts no liability for any damage caused by >>>>>> any virus transmitted by this email. Read Disclaimer at >>>>>> http://www.sasken.com/extras/mail_disclaimer.html >>>>> >>>>>> _______________________________________________ lttng-dev mailing >>>>>> list lttng-dev at lists.lttng.org >>>>>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev >>>>> >>>>> >>>>> -- Mathieu Desnoyers Operating System Efficiency R&D Consultant >>>>> EfficiOS Inc. http://www.efficios.com >>>> >>>> -- Mathieu Desnoyers Operating System Efficiency R&D Consultant >>>> EfficiOS Inc. http://www.efficios.com >>> >>> -- Mathieu Desnoyers Operating System Efficiency R&D Consultant >>> EfficiOS Inc. http://www.efficios.com > > > _______________________________________________ lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBAgAGBQJP201YAAoJEELoaioR9I02a7AH/3sTHsbxRJ3z+//6+wPQu7GC 6KIZxfBhhq4WSSZRJGHvD9O6QCHPgUmrYjR4leONB9//FfoWdzzRZb0IDErpJ3+L B8I49nd7X2UHp1jf7CpbJUzrs1AQZa3K32D/nRlF3LceOGnvIpkKstur5msXvN5B msGap8UQ8NsxkyCRW8cmPNt/Qm7lMsrg9zeP5VGEncj7dywIqaPfjz7MGFwNp4Vz uxxBusAjnI50Kmx+ETYZmTiqEIPCV9rnpVK4T2LtZgX/+rh06gDkkvW6xy0wO7pf ymZ0MytfvuKaHXmQnXyGvWhrodmn+D9E32DwyrcuhZa6K0ZPE5eDRbxUT5WGbuQ= =VYvD -----END PGP SIGNATURE----- From mathieu.desnoyers at efficios.com Thu Jun 21 12:41:13 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Thu, 21 Jun 2012 12:41:13 -0400 Subject: [lttng-dev] [RFC] Userspace RCU library internal error handling Message-ID: <20120621164113.GA21197@Krystal> Hi, Currently, liburcu calls "exit(-1)" upon internal consistency error. This is not pretty, and usually frowned upon in libraries. One example of failure path where we use this is if pthread_mutex_lock() would happen to fail within synchronize_rcu(). Clearly, this should _never_ happen: it would typically be triggered only by memory corruption (or other terrible things like that). That being said, we clearly don't want to make "synchronize_rcu()" return errors like that to the application, because it would complexify the application error handling needlessly. So instead of calling exit(-1), one possibility would be to do something like this: #include #include #include #define urcu_die(fmt, ...) \ do { \ fprintf(stderr, fmt, ##__VA_ARGS__); \ (void) pthread_kill(pthread_self(), SIGBUS); \ } while (0) and call urcu_die(); in those "unrecoverable error" cases, instead of calling exit(-1). Therefore, if an application chooses to trap those signals, it can, which is otherwise not possible with a direct call to exit(). Thoughts ? Thanks, Mathieu -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From paulmck at linux.vnet.ibm.com Thu Jun 21 12:53:34 2012 From: paulmck at linux.vnet.ibm.com (Paul E. McKenney) Date: Thu, 21 Jun 2012 09:53:34 -0700 Subject: [lttng-dev] [RFC] Userspace RCU library internal error handling In-Reply-To: <20120621164113.GA21197@Krystal> References: <20120621164113.GA21197@Krystal> Message-ID: <20120621165334.GE2397@linux.vnet.ibm.com> On Thu, Jun 21, 2012 at 12:41:13PM -0400, Mathieu Desnoyers wrote: > Hi, > > Currently, liburcu calls "exit(-1)" upon internal consistency error. > This is not pretty, and usually frowned upon in libraries. > > One example of failure path where we use this is if pthread_mutex_lock() > would happen to fail within synchronize_rcu(). Clearly, this should > _never_ happen: it would typically be triggered only by memory > corruption (or other terrible things like that). That being said, we > clearly don't want to make "synchronize_rcu()" return errors like that > to the application, because it would complexify the application error > handling needlessly. > > So instead of calling exit(-1), one possibility would be to do something > like this: > > #include > #include > #include > > #define urcu_die(fmt, ...) \ > do { \ > fprintf(stderr, fmt, ##__VA_ARGS__); \ > (void) pthread_kill(pthread_self(), SIGBUS); \ > } while (0) > > and call urcu_die(); in those "unrecoverable error" cases, instead of > calling exit(-1). Therefore, if an application chooses to trap those > signals, it can, which is otherwise not possible with a direct call to > exit(). This approach makes a lot of sense to me. Thanx, Paul From josh at joshtriplett.org Thu Jun 21 14:59:42 2012 From: josh at joshtriplett.org (Josh Triplett) Date: Thu, 21 Jun 2012 11:59:42 -0700 Subject: [lttng-dev] [rp] [RFC] Userspace RCU library internal error handling In-Reply-To: <20120621164113.GA21197@Krystal> References: <20120621164113.GA21197@Krystal> Message-ID: <20120621185941.GC26361@leaf> On Thu, Jun 21, 2012 at 12:41:13PM -0400, Mathieu Desnoyers wrote: > Currently, liburcu calls "exit(-1)" upon internal consistency error. > This is not pretty, and usually frowned upon in libraries. Agreed. > One example of failure path where we use this is if pthread_mutex_lock() > would happen to fail within synchronize_rcu(). Clearly, this should > _never_ happen: it would typically be triggered only by memory > corruption (or other terrible things like that). That being said, we > clearly don't want to make "synchronize_rcu()" return errors like that > to the application, because it would complexify the application error > handling needlessly. I think you can safely ignore any error conditions you know you can't trigger. pthread_mutex_lock can only return an error under two conditions: an uninitialized mutex, or an error-checking mutex already locked by the current thread. Neither of those can happen in this case. Given that, I'd suggest either calling pthread_mutex_lock and ignoring any possibility of error, or adding an assert. > So instead of calling exit(-1), one possibility would be to do something > like this: > > #include > #include > #include > > #define urcu_die(fmt, ...) \ > do { \ > fprintf(stderr, fmt, ##__VA_ARGS__); \ > (void) pthread_kill(pthread_self(), SIGBUS); \ > } while (0) > > and call urcu_die(); in those "unrecoverable error" cases, instead of > calling exit(-1). Therefore, if an application chooses to trap those > signals, it can, which is otherwise not possible with a direct call to > exit(). It looks like you want to use signals as a kind of exception mechanism, to allow the application to clean up (though not to recover). assert seems much clearer to me for "this can't happen" cases, and assert also generates a signal that the application can catch and clean up. - Josh Triplett From mathieu.desnoyers at efficios.com Thu Jun 21 15:03:06 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Thu, 21 Jun 2012 15:03:06 -0400 Subject: [lttng-dev] [rp] [RFC] Userspace RCU library internal error handling In-Reply-To: <20120621185941.GC26361@leaf> References: <20120621164113.GA21197@Krystal> <20120621185941.GC26361@leaf> Message-ID: <20120621190306.GB24179@Krystal> * Josh Triplett (josh at joshtriplett.org) wrote: > On Thu, Jun 21, 2012 at 12:41:13PM -0400, Mathieu Desnoyers wrote: > > Currently, liburcu calls "exit(-1)" upon internal consistency error. > > This is not pretty, and usually frowned upon in libraries. > > Agreed. > > > One example of failure path where we use this is if pthread_mutex_lock() > > would happen to fail within synchronize_rcu(). Clearly, this should > > _never_ happen: it would typically be triggered only by memory > > corruption (or other terrible things like that). That being said, we > > clearly don't want to make "synchronize_rcu()" return errors like that > > to the application, because it would complexify the application error > > handling needlessly. > > I think you can safely ignore any error conditions you know you can't > trigger. pthread_mutex_lock can only return an error under two > conditions: an uninitialized mutex, or an error-checking mutex already > locked by the current thread. Neither of those can happen in this case. > Given that, I'd suggest either calling pthread_mutex_lock and ignoring > any possibility of error, or adding an assert. > > > So instead of calling exit(-1), one possibility would be to do something > > like this: > > > > #include > > #include > > #include > > > > #define urcu_die(fmt, ...) \ > > do { \ > > fprintf(stderr, fmt, ##__VA_ARGS__); \ > > (void) pthread_kill(pthread_self(), SIGBUS); \ > > } while (0) > > > > and call urcu_die(); in those "unrecoverable error" cases, instead of > > calling exit(-1). Therefore, if an application chooses to trap those > > signals, it can, which is otherwise not possible with a direct call to > > exit(). > > It looks like you want to use signals as a kind of exception mechanism, > to allow the application to clean up (though not to recover). assert > seems much clearer to me for "this can't happen" cases, and assert also > generates a signal that the application can catch and clean up. Within discussions with other LTTng developers, we considered the the assert, but the thought that this case might be silently ignored on production systems (which compile with assertions disabled) makes me uncomfortable. This is why I would prefer a SIGBUS to an assertion. Using assert() would be similar to turning the Linux kernel BUG_ON() mechanism into no-ops on production systems because "it should never happen" (tm) ;-) Thoughts ? Thanks, Mathieu -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Thu Jun 21 15:14:51 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Thu, 21 Jun 2012 15:14:51 -0400 Subject: [lttng-dev] [RFC PATCH] Userspace RCU library internal error handling In-Reply-To: <20120621190306.GB24179@Krystal> References: <20120621164113.GA21197@Krystal> <20120621185941.GC26361@leaf> <20120621190306.GB24179@Krystal> Message-ID: <20120621191451.GA25058@Krystal> * Mathieu Desnoyers (mathieu.desnoyers at efficios.com) wrote: > * Josh Triplett (josh at joshtriplett.org) wrote: > > On Thu, Jun 21, 2012 at 12:41:13PM -0400, Mathieu Desnoyers wrote: > > > Currently, liburcu calls "exit(-1)" upon internal consistency error. > > > This is not pretty, and usually frowned upon in libraries. > > > > Agreed. > > > > > One example of failure path where we use this is if pthread_mutex_lock() > > > would happen to fail within synchronize_rcu(). Clearly, this should > > > _never_ happen: it would typically be triggered only by memory > > > corruption (or other terrible things like that). That being said, we > > > clearly don't want to make "synchronize_rcu()" return errors like that > > > to the application, because it would complexify the application error > > > handling needlessly. > > > > I think you can safely ignore any error conditions you know you can't > > trigger. pthread_mutex_lock can only return an error under two > > conditions: an uninitialized mutex, or an error-checking mutex already > > locked by the current thread. Neither of those can happen in this case. > > Given that, I'd suggest either calling pthread_mutex_lock and ignoring > > any possibility of error, or adding an assert. > > > > > So instead of calling exit(-1), one possibility would be to do something > > > like this: > > > > > > #include > > > #include > > > #include > > > > > > #define urcu_die(fmt, ...) \ > > > do { \ > > > fprintf(stderr, fmt, ##__VA_ARGS__); \ > > > (void) pthread_kill(pthread_self(), SIGBUS); \ > > > } while (0) > > > > > > and call urcu_die(); in those "unrecoverable error" cases, instead of > > > calling exit(-1). Therefore, if an application chooses to trap those > > > signals, it can, which is otherwise not possible with a direct call to > > > exit(). > > > > It looks like you want to use signals as a kind of exception mechanism, > > to allow the application to clean up (though not to recover). assert > > seems much clearer to me for "this can't happen" cases, and assert also > > generates a signal that the application can catch and clean up. > > Within discussions with other LTTng developers, we considered the the > assert, but the thought that this case might be silently ignored on > production systems (which compile with assertions disabled) makes me > uncomfortable. This is why I would prefer a SIGBUS to an assertion. > > Using assert() would be similar to turning the Linux kernel BUG_ON() > mechanism into no-ops on production systems because "it should never > happen" (tm) ;-) Here is the patch for comments. --- diff --git a/Makefile.am b/Makefile.am index 933e538..2396fcf 100644 --- a/Makefile.am +++ b/Makefile.am @@ -22,6 +22,8 @@ nobase_dist_include_HEADERS = urcu/compiler.h urcu/hlist.h urcu/list.h \ urcu/tls-compat.h nobase_nodist_include_HEADERS = urcu/arch.h urcu/uatomic.h urcu/config.h +dist_noinst_HEADERS = urcu-die.h + EXTRA_DIST = $(top_srcdir)/urcu/arch/*.h $(top_srcdir)/urcu/uatomic/*.h \ gpl-2.0.txt lgpl-2.1.txt lgpl-relicensing.txt \ LICENSE compat_arch_x86.c \ diff --git a/urcu-bp.c b/urcu-bp.c index 67eae07..b9c89d8 100644 --- a/urcu-bp.c +++ b/urcu-bp.c @@ -42,6 +42,8 @@ #include "urcu-pointer.h" #include "urcu/tls-compat.h" +#include "urcu-die.h" + /* Do not #define _LGPL_SOURCE to ensure we can emit the wrapper symbols */ #undef _LGPL_SOURCE #include "urcu-bp.h" @@ -142,17 +144,12 @@ static void mutex_lock(pthread_mutex_t *mutex) #ifndef DISTRUST_SIGNALS_EXTREME ret = pthread_mutex_lock(mutex); - if (ret) { - perror("Error in pthread mutex lock"); - exit(-1); - } + if (ret) + urcu_die(ret); #else /* #ifndef DISTRUST_SIGNALS_EXTREME */ while ((ret = pthread_mutex_trylock(mutex)) != 0) { - if (ret != EBUSY && ret != EINTR) { - printf("ret = %d, errno = %d\n", ret, errno); - perror("Error in pthread mutex lock"); - exit(-1); - } + if (ret != EBUSY && ret != EINTR) + urcu_die(ret); poll(NULL,0,10); } #endif /* #else #ifndef DISTRUST_SIGNALS_EXTREME */ @@ -163,10 +160,8 @@ static void mutex_unlock(pthread_mutex_t *mutex) int ret; ret = pthread_mutex_unlock(mutex); - if (ret) { - perror("Error in pthread mutex unlock"); - exit(-1); - } + if (ret) + urcu_die(ret); } void update_counter_and_wait(void) diff --git a/urcu-die.h b/urcu-die.h new file mode 100644 index 0000000..e925d74 --- /dev/null +++ b/urcu-die.h @@ -0,0 +1,38 @@ +#ifndef _URCU_DIE_H +#define _URCU_DIE_H + +/* + * urcu-die.h + * + * Userspace RCU library unrecoverable error handling + * + * Copyright (c) 2012 Mathieu Desnoyers + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include +#include +#include + +#define urcu_die(cause) \ +do { \ + fprintf(stderr, "(" __FILE__ ":%s@%u) Unrecoverable error: %s\n", \ + __func__, __LINE__, strerror(cause)); \ + (void) pthread_kill(pthread_self(), SIGBUS); \ +} while (0) + +#endif /* _URCU_DIE_H */ diff --git a/urcu-qsbr.c b/urcu-qsbr.c index b20d564..d3a6849 100644 --- a/urcu-qsbr.c +++ b/urcu-qsbr.c @@ -42,6 +42,8 @@ #include "urcu-pointer.h" #include "urcu/tls-compat.h" +#include "urcu-die.h" + /* Do not #define _LGPL_SOURCE to ensure we can emit the wrapper symbols */ #undef _LGPL_SOURCE #include "urcu-qsbr.h" @@ -82,17 +84,12 @@ static void mutex_lock(pthread_mutex_t *mutex) #ifndef DISTRUST_SIGNALS_EXTREME ret = pthread_mutex_lock(mutex); - if (ret) { - perror("Error in pthread mutex lock"); - exit(-1); - } + if (ret) + urcu_die(ret); #else /* #ifndef DISTRUST_SIGNALS_EXTREME */ while ((ret = pthread_mutex_trylock(mutex)) != 0) { - if (ret != EBUSY && ret != EINTR) { - printf("ret = %d, errno = %d\n", ret, errno); - perror("Error in pthread mutex lock"); - exit(-1); - } + if (ret != EBUSY && ret != EINTR) + urcu_die(ret); poll(NULL,0,10); } #endif /* #else #ifndef DISTRUST_SIGNALS_EXTREME */ @@ -103,10 +100,8 @@ static void mutex_unlock(pthread_mutex_t *mutex) int ret; ret = pthread_mutex_unlock(mutex); - if (ret) { - perror("Error in pthread mutex unlock"); - exit(-1); - } + if (ret) + urcu_die(ret); } /* diff --git a/urcu.c b/urcu.c index 5fb4db8..a5178c0 100644 --- a/urcu.c +++ b/urcu.c @@ -42,6 +42,8 @@ #include "urcu-pointer.h" #include "urcu/tls-compat.h" +#include "urcu-die.h" + /* Do not #define _LGPL_SOURCE to ensure we can emit the wrapper symbols */ #undef _LGPL_SOURCE #include "urcu.h" @@ -110,17 +112,12 @@ static void mutex_lock(pthread_mutex_t *mutex) #ifndef DISTRUST_SIGNALS_EXTREME ret = pthread_mutex_lock(mutex); - if (ret) { - perror("Error in pthread mutex lock"); - exit(-1); - } + if (ret) + urcu_die(ret); #else /* #ifndef DISTRUST_SIGNALS_EXTREME */ while ((ret = pthread_mutex_trylock(mutex)) != 0) { - if (ret != EBUSY && ret != EINTR) { - printf("ret = %d, errno = %d\n", ret, errno); - perror("Error in pthread mutex lock"); - exit(-1); - } + if (ret != EBUSY && ret != EINTR) + urcu_die(ret); if (CMM_LOAD_SHARED(URCU_TLS(rcu_reader).need_mb)) { cmm_smp_mb(); _CMM_STORE_SHARED(URCU_TLS(rcu_reader).need_mb, 0); @@ -136,10 +133,8 @@ static void mutex_unlock(pthread_mutex_t *mutex) int ret; ret = pthread_mutex_unlock(mutex); - if (ret) { - perror("Error in pthread mutex unlock"); - exit(-1); - } + if (ret) + urcu_die(ret); } #ifdef RCU_MEMBARRIER @@ -432,10 +427,8 @@ void rcu_init(void) act.sa_flags = SA_SIGINFO | SA_RESTART; sigemptyset(&act.sa_mask); ret = sigaction(SIGRCU, &act, NULL); - if (ret) { - perror("Error in sigaction"); - exit(-1); - } + if (ret) + urcu_die(errno); } void rcu_exit(void) @@ -444,10 +437,8 @@ void rcu_exit(void) int ret; ret = sigaction(SIGRCU, NULL, &act); - if (ret) { - perror("Error in sigaction"); - exit(-1); - } + if (ret) + urcu_die(errno); assert(act.sa_sigaction == sigrcu_handler); assert(cds_list_empty(®istry)); } -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From jbernard at debian.org Thu Jun 21 15:14:52 2012 From: jbernard at debian.org (Jon Bernard) Date: Thu, 21 Jun 2012 15:14:52 -0400 Subject: [lttng-dev] Query about LTTng 2.0 Debian packages In-Reply-To: <20120621012622.GA1387@Krystal> References: <20120621012622.GA1387@Krystal> Message-ID: <20120621191452.GA18195@quintessa> * Mathieu Desnoyers wrote: > Hi Jon, > > I wanted to get in touch with you regarding LTTng 2.0 Debian packages. > Is there anything we can do to help you getting those packages into > Debian ? They have been out for a while now, and even Ubuntu picked them > up. I understand that you are probably busy, hence my offer for help. I've just uploaded another package I've been working on and LTTng 2.0 packages are next on my list. Give me a few days to get things sorted, I'll post an update next week with my status. Thanks for your patience, I know I'm behind but I think there is still time to get the packages uploaded before the freeze occurs. Cheers -- Jon From josh at joshtriplett.org Thu Jun 21 15:28:24 2012 From: josh at joshtriplett.org (Josh Triplett) Date: Thu, 21 Jun 2012 12:28:24 -0700 Subject: [lttng-dev] [rp] [RFC] Userspace RCU library internal error handling In-Reply-To: <20120621190306.GB24179@Krystal> References: <20120621164113.GA21197@Krystal> <20120621185941.GC26361@leaf> <20120621190306.GB24179@Krystal> Message-ID: <20120621192824.GE26361@leaf> On Thu, Jun 21, 2012 at 03:03:06PM -0400, Mathieu Desnoyers wrote: > * Josh Triplett (josh at joshtriplett.org) wrote: > > On Thu, Jun 21, 2012 at 12:41:13PM -0400, Mathieu Desnoyers wrote: > > > Currently, liburcu calls "exit(-1)" upon internal consistency error. > > > This is not pretty, and usually frowned upon in libraries. > > > > Agreed. > > > > > One example of failure path where we use this is if pthread_mutex_lock() > > > would happen to fail within synchronize_rcu(). Clearly, this should > > > _never_ happen: it would typically be triggered only by memory > > > corruption (or other terrible things like that). That being said, we > > > clearly don't want to make "synchronize_rcu()" return errors like that > > > to the application, because it would complexify the application error > > > handling needlessly. > > > > I think you can safely ignore any error conditions you know you can't > > trigger. pthread_mutex_lock can only return an error under two > > conditions: an uninitialized mutex, or an error-checking mutex already > > locked by the current thread. Neither of those can happen in this case. > > Given that, I'd suggest either calling pthread_mutex_lock and ignoring > > any possibility of error, or adding an assert. > > > > > So instead of calling exit(-1), one possibility would be to do something > > > like this: > > > > > > #include > > > #include > > > #include > > > > > > #define urcu_die(fmt, ...) \ > > > do { \ > > > fprintf(stderr, fmt, ##__VA_ARGS__); \ > > > (void) pthread_kill(pthread_self(), SIGBUS); \ > > > } while (0) > > > > > > and call urcu_die(); in those "unrecoverable error" cases, instead of > > > calling exit(-1). Therefore, if an application chooses to trap those > > > signals, it can, which is otherwise not possible with a direct call to > > > exit(). > > > > It looks like you want to use signals as a kind of exception mechanism, > > to allow the application to clean up (though not to recover). assert > > seems much clearer to me for "this can't happen" cases, and assert also > > generates a signal that the application can catch and clean up. > > Within discussions with other LTTng developers, we considered the the > assert, but the thought that this case might be silently ignored on > production systems (which compile with assertions disabled) makes me > uncomfortable. This is why I would prefer a SIGBUS to an assertion. > > Using assert() would be similar to turning the Linux kernel BUG_ON() > mechanism into no-ops on production systems because "it should never > happen" (tm) ;-) Just don't define NDEBUG then. :) - Josh Triplett From mathieu.desnoyers at efficios.com Thu Jun 21 15:48:38 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Thu, 21 Jun 2012 15:48:38 -0400 Subject: [lttng-dev] [rp] [RFC] Userspace RCU library internal error handling In-Reply-To: <20120621192824.GE26361@leaf> References: <20120621164113.GA21197@Krystal> <20120621185941.GC26361@leaf> <20120621190306.GB24179@Krystal> <20120621192824.GE26361@leaf> Message-ID: <20120621194838.GB25571@Krystal> * Josh Triplett (josh at joshtriplett.org) wrote: > On Thu, Jun 21, 2012 at 03:03:06PM -0400, Mathieu Desnoyers wrote: > > * Josh Triplett (josh at joshtriplett.org) wrote: > > > On Thu, Jun 21, 2012 at 12:41:13PM -0400, Mathieu Desnoyers wrote: > > > > Currently, liburcu calls "exit(-1)" upon internal consistency error. > > > > This is not pretty, and usually frowned upon in libraries. > > > > > > Agreed. > > > > > > > One example of failure path where we use this is if pthread_mutex_lock() > > > > would happen to fail within synchronize_rcu(). Clearly, this should > > > > _never_ happen: it would typically be triggered only by memory > > > > corruption (or other terrible things like that). That being said, we > > > > clearly don't want to make "synchronize_rcu()" return errors like that > > > > to the application, because it would complexify the application error > > > > handling needlessly. > > > > > > I think you can safely ignore any error conditions you know you can't > > > trigger. pthread_mutex_lock can only return an error under two > > > conditions: an uninitialized mutex, or an error-checking mutex already > > > locked by the current thread. Neither of those can happen in this case. > > > Given that, I'd suggest either calling pthread_mutex_lock and ignoring > > > any possibility of error, or adding an assert. > > > > > > > So instead of calling exit(-1), one possibility would be to do something > > > > like this: > > > > > > > > #include > > > > #include > > > > #include > > > > > > > > #define urcu_die(fmt, ...) \ > > > > do { \ > > > > fprintf(stderr, fmt, ##__VA_ARGS__); \ > > > > (void) pthread_kill(pthread_self(), SIGBUS); \ > > > > } while (0) > > > > > > > > and call urcu_die(); in those "unrecoverable error" cases, instead of > > > > calling exit(-1). Therefore, if an application chooses to trap those > > > > signals, it can, which is otherwise not possible with a direct call to > > > > exit(). > > > > > > It looks like you want to use signals as a kind of exception mechanism, > > > to allow the application to clean up (though not to recover). assert > > > seems much clearer to me for "this can't happen" cases, and assert also > > > generates a signal that the application can catch and clean up. > > > > Within discussions with other LTTng developers, we considered the the > > assert, but the thought that this case might be silently ignored on > > production systems (which compile with assertions disabled) makes me > > uncomfortable. This is why I would prefer a SIGBUS to an assertion. > > > > Using assert() would be similar to turning the Linux kernel BUG_ON() > > mechanism into no-ops on production systems because "it should never > > happen" (tm) ;-) > > Just don't define NDEBUG then. :) Well, AFAIK, it is usual for some distribution packages to define NDEBUG (maybe distro maintainers reading lttng-dev could confirm or infirm this assumption ?). So in a context where upstream does not have total control on the specific tweaks done by distro packages, I prefer not to rely on NDEBUG not being defined to catch internal consistency errors in the wild. Thanks, Mathieu -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From josh at joshtriplett.org Thu Jun 21 17:21:02 2012 From: josh at joshtriplett.org (Josh Triplett) Date: Thu, 21 Jun 2012 14:21:02 -0700 Subject: [lttng-dev] [rp] [RFC] Userspace RCU library internal error handling In-Reply-To: <20120621194838.GB25571@Krystal> References: <20120621164113.GA21197@Krystal> <20120621185941.GC26361@leaf> <20120621190306.GB24179@Krystal> <20120621192824.GE26361@leaf> <20120621194838.GB25571@Krystal> Message-ID: <20120621212102.GF26361@leaf> On Thu, Jun 21, 2012 at 03:48:38PM -0400, Mathieu Desnoyers wrote: > * Josh Triplett (josh at joshtriplett.org) wrote: > > On Thu, Jun 21, 2012 at 03:03:06PM -0400, Mathieu Desnoyers wrote: > > > * Josh Triplett (josh at joshtriplett.org) wrote: > > > > On Thu, Jun 21, 2012 at 12:41:13PM -0400, Mathieu Desnoyers wrote: > > > > > Currently, liburcu calls "exit(-1)" upon internal consistency error. > > > > > This is not pretty, and usually frowned upon in libraries. > > > > > > > > Agreed. > > > > > > > > > One example of failure path where we use this is if pthread_mutex_lock() > > > > > would happen to fail within synchronize_rcu(). Clearly, this should > > > > > _never_ happen: it would typically be triggered only by memory > > > > > corruption (or other terrible things like that). That being said, we > > > > > clearly don't want to make "synchronize_rcu()" return errors like that > > > > > to the application, because it would complexify the application error > > > > > handling needlessly. > > > > > > > > I think you can safely ignore any error conditions you know you can't > > > > trigger. pthread_mutex_lock can only return an error under two > > > > conditions: an uninitialized mutex, or an error-checking mutex already > > > > locked by the current thread. Neither of those can happen in this case. > > > > Given that, I'd suggest either calling pthread_mutex_lock and ignoring > > > > any possibility of error, or adding an assert. > > > > > > > > > So instead of calling exit(-1), one possibility would be to do something > > > > > like this: > > > > > > > > > > #include > > > > > #include > > > > > #include > > > > > > > > > > #define urcu_die(fmt, ...) \ > > > > > do { \ > > > > > fprintf(stderr, fmt, ##__VA_ARGS__); \ > > > > > (void) pthread_kill(pthread_self(), SIGBUS); \ > > > > > } while (0) > > > > > > > > > > and call urcu_die(); in those "unrecoverable error" cases, instead of > > > > > calling exit(-1). Therefore, if an application chooses to trap those > > > > > signals, it can, which is otherwise not possible with a direct call to > > > > > exit(). > > > > > > > > It looks like you want to use signals as a kind of exception mechanism, > > > > to allow the application to clean up (though not to recover). assert > > > > seems much clearer to me for "this can't happen" cases, and assert also > > > > generates a signal that the application can catch and clean up. > > > > > > Within discussions with other LTTng developers, we considered the the > > > assert, but the thought that this case might be silently ignored on > > > production systems (which compile with assertions disabled) makes me > > > uncomfortable. This is why I would prefer a SIGBUS to an assertion. > > > > > > Using assert() would be similar to turning the Linux kernel BUG_ON() > > > mechanism into no-ops on production systems because "it should never > > > happen" (tm) ;-) > > > > Just don't define NDEBUG then. :) > > Well, AFAIK, it is usual for some distribution packages to define > NDEBUG (maybe distro maintainers reading lttng-dev could confirm or > infirm this assumption ?). So in a context where upstream does not have > total control on the specific tweaks done by distro packages, I prefer > not to rely on NDEBUG not being defined to catch internal consistency > errors in the wild. #undef NDEBUG #include Or if you don't consider that sufficient for some reason, you could define your own assert(), but that seems like an odd thing to not count on. Nonetheless, if you define your own assert, I'd still suggest making it look as much like assert() as possible, including the call to abort(). - Josh Triplett From simon.marchi at polymtl.ca Fri Jun 22 02:29:27 2012 From: simon.marchi at polymtl.ca (simon.marchi at polymtl.ca) Date: Fri, 22 Jun 2012 02:29:27 -0400 Subject: [lttng-dev] [PATCH] Add --simple option to the list command Message-ID: <1340346567-11748-1-git-send-email-simon.marchi@polymtl.ca> From: Simon Marchi The --simple option makes "list" print one item per line, a nice format to be parsed by scripts. For example, the bash completion script could benefit from this feature. For now, only session listing is affected, but it would be useful to have other kind of listing behave the same way. Also left some comments on my path. Signed-off-by: Simon Marchi --- src/bin/lttng/commands/list.c | 41 ++++++++++++++++++++++++++++------------- 1 file changed, 28 insertions(+), 13 deletions(-) diff --git a/src/bin/lttng/commands/list.c b/src/bin/lttng/commands/list.c index 966be2d..cb4fb58 100644 --- a/src/bin/lttng/commands/list.c +++ b/src/bin/lttng/commands/list.c @@ -30,6 +30,7 @@ static int opt_kernel; static char *opt_channel; static int opt_domain; static int opt_fields; +static int simple; #if 0 /* Not implemented yet */ static char *opt_cmd_name; @@ -44,6 +45,7 @@ enum { OPT_HELP = 1, OPT_USERSPACE, OPT_LIST_OPTIONS, + OPT_SIMPLE, }; static struct lttng_handle *handle; @@ -63,6 +65,7 @@ static struct poptOption long_options[] = { {"domain", 'd', POPT_ARG_VAL, &opt_domain, 1, 0, 0}, {"fields", 'f', POPT_ARG_VAL, &opt_fields, 1, 0, 0}, {"list-options", 0, POPT_ARG_NONE, NULL, OPT_LIST_OPTIONS, NULL, NULL}, + {"simple", 0, POPT_ARG_NONE, &simple, 0, NULL, NULL}, {0, 0, 0, 0, 0, 0, 0} }; @@ -80,6 +83,8 @@ static void usage(FILE *ofp) fprintf(ofp, "\n"); fprintf(ofp, " -h, --help Show this help\n"); fprintf(ofp, " --list-options Simple listing of options\n"); + fprintf(ofp, " --simple Prints one item per line, suitable " + "for parsing\n"); fprintf(ofp, " -k, --kernel Select kernel domain\n"); fprintf(ofp, " -u, --userspace Select user-space domain.\n"); fprintf(ofp, " -f, --fields List event fields.\n"); @@ -566,33 +571,43 @@ static int list_sessions(const char *session_name) count = lttng_list_sessions(&sessions); DBG("Session count %d", count); if (count < 0) { + /* Some error happened */ ret = count; goto error; } else if (count == 0) { - MSG("Currently no available tracing session"); + /* No session available */ + if (!simple) { + MSG("Currently no available tracing session"); + } goto end; } - if (session_name == NULL) { + if (session_name == NULL && !simple) { MSG("Available tracing sessions:"); } for (i = 0; i < count; i++) { if (session_name != NULL) { + /* We want to list info only about a particular session */ if (strncmp(sessions[i].name, session_name, NAME_MAX) == 0) { session_found = 1; - MSG("Tracing session %s:%s", session_name, active_string(sessions[i].enabled)); - MSG("%sTrace path: %s\n", indent4, sessions[i].path); + + if (simple) { + MSG("%s", session_name); + } else { + MSG("Tracing session %s:%s", session_name, active_string(sessions[i].enabled)); + MSG("%sTrace path: %s\n", indent4, sessions[i].path); + } break; } - continue; - } - - MSG(" %d) %s (%s)%s", i + 1, sessions[i].name, sessions[i].path, - active_string(sessions[i].enabled)); - - if (session_found) { - break; + } else { + /* We want to list all sessions */ + if (simple) { + MSG("%s", sessions[i].name); + } else { + MSG(" %d) %s (%s)%s", i + 1, sessions[i].name, sessions[i].path, + active_string(sessions[i].enabled)); + } } } @@ -604,7 +619,7 @@ static int list_sessions(const char *session_name) goto error; } - if (session_name == NULL) { + if (session_name == NULL && !simple) { MSG("\nUse lttng list for more details"); } -- 1.7.10 From mathieu.desnoyers at efficios.com Fri Jun 22 11:22:31 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Fri, 22 Jun 2012 11:22:31 -0400 Subject: [lttng-dev] [rp] [RFC] Userspace RCU library internal error handling In-Reply-To: <20120621212102.GF26361@leaf> References: <20120621164113.GA21197@Krystal> <20120621185941.GC26361@leaf> <20120621190306.GB24179@Krystal> <20120621192824.GE26361@leaf> <20120621194838.GB25571@Krystal> <20120621212102.GF26361@leaf> Message-ID: <20120622152231.GA10788@Krystal> * Josh Triplett (josh at joshtriplett.org) wrote: > On Thu, Jun 21, 2012 at 03:48:38PM -0400, Mathieu Desnoyers wrote: > > * Josh Triplett (josh at joshtriplett.org) wrote: > > > On Thu, Jun 21, 2012 at 03:03:06PM -0400, Mathieu Desnoyers wrote: > > > > * Josh Triplett (josh at joshtriplett.org) wrote: > > > > > On Thu, Jun 21, 2012 at 12:41:13PM -0400, Mathieu Desnoyers wrote: > > > > > > Currently, liburcu calls "exit(-1)" upon internal consistency error. > > > > > > This is not pretty, and usually frowned upon in libraries. > > > > > > > > > > Agreed. > > > > > > > > > > > One example of failure path where we use this is if pthread_mutex_lock() > > > > > > would happen to fail within synchronize_rcu(). Clearly, this should > > > > > > _never_ happen: it would typically be triggered only by memory > > > > > > corruption (or other terrible things like that). That being said, we > > > > > > clearly don't want to make "synchronize_rcu()" return errors like that > > > > > > to the application, because it would complexify the application error > > > > > > handling needlessly. > > > > > > > > > > I think you can safely ignore any error conditions you know you can't > > > > > trigger. pthread_mutex_lock can only return an error under two > > > > > conditions: an uninitialized mutex, or an error-checking mutex already > > > > > locked by the current thread. Neither of those can happen in this case. > > > > > Given that, I'd suggest either calling pthread_mutex_lock and ignoring > > > > > any possibility of error, or adding an assert. > > > > > > > > > > > So instead of calling exit(-1), one possibility would be to do something > > > > > > like this: > > > > > > > > > > > > #include > > > > > > #include > > > > > > #include > > > > > > > > > > > > #define urcu_die(fmt, ...) \ > > > > > > do { \ > > > > > > fprintf(stderr, fmt, ##__VA_ARGS__); \ > > > > > > (void) pthread_kill(pthread_self(), SIGBUS); \ > > > > > > } while (0) > > > > > > > > > > > > and call urcu_die(); in those "unrecoverable error" cases, instead of > > > > > > calling exit(-1). Therefore, if an application chooses to trap those > > > > > > signals, it can, which is otherwise not possible with a direct call to > > > > > > exit(). > > > > > > > > > > It looks like you want to use signals as a kind of exception mechanism, > > > > > to allow the application to clean up (though not to recover). assert > > > > > seems much clearer to me for "this can't happen" cases, and assert also > > > > > generates a signal that the application can catch and clean up. > > > > > > > > Within discussions with other LTTng developers, we considered the the > > > > assert, but the thought that this case might be silently ignored on > > > > production systems (which compile with assertions disabled) makes me > > > > uncomfortable. This is why I would prefer a SIGBUS to an assertion. > > > > > > > > Using assert() would be similar to turning the Linux kernel BUG_ON() > > > > mechanism into no-ops on production systems because "it should never > > > > happen" (tm) ;-) > > > > > > Just don't define NDEBUG then. :) > > > > Well, AFAIK, it is usual for some distribution packages to define > > NDEBUG (maybe distro maintainers reading lttng-dev could confirm or > > infirm this assumption ?). So in a context where upstream does not have > > total control on the specific tweaks done by distro packages, I prefer > > not to rely on NDEBUG not being defined to catch internal consistency > > errors in the wild. > > #undef NDEBUG > #include > > Or if you don't consider that sufficient for some reason, you could > define your own assert(), but that seems like an odd thing to not count > on. Nonetheless, if you define your own assert, I'd still suggest > making it look as much like assert() as possible, including the call to > abort(). #undef NDEBUG is unwanted, due to its side-effects. We use "assert()" in other locations of the code, for which we want the assertion check to be disabled if NDEBUG is defined in production. I agree with you that calling "abort()" is exactly what we want, and it's much more standard that sending a signal chosen with a fair roll of dices. How about the following ? [...] diff --git a/urcu-die.h b/urcu-die.h new file mode 100644 index 0000000..227c8dc --- /dev/null +++ b/urcu-die.h @@ -0,0 +1,37 @@ +#ifndef _URCU_DIE_H +#define _URCU_DIE_H + +/* + * urcu-die.h + * + * Userspace RCU library unrecoverable error handling + * + * Copyright (c) 2012 Mathieu Desnoyers + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include +#include + +#define urcu_die(cause) \ +do { \ + fprintf(stderr, "(" __FILE__ ":%s@%u) Unrecoverable error: %s\n", \ + __func__, __LINE__, strerror(cause)); \ + abort(); \ +} while (0) + +#endif /* _URCU_DIE_H */ [...] -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From dgoulet at efficios.com Fri Jun 22 11:27:45 2012 From: dgoulet at efficios.com (David Goulet) Date: Fri, 22 Jun 2012 11:27:45 -0400 Subject: [lttng-dev] [RELEASE] LTTng-tools 2.0.3 Message-ID: <4FE48EF1.2080409@efficios.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The lttng-tools project provides a session daemon (lttng-sessiond) that acts as a tracing registry, the "lttng" command line for tracing control and the liblttng-ctl library also for tracing control. This release fixes a show stopper in version 2.0.2. Enabling an event with the loglevel type ALL was failing due to an error in the loglevel match function on the session daemon side. Sorry all for this inconvenience! 2012-06-22 lttng-tools 2.0.3 * Fix: enable event loglevel match function * Fix: unchecked pointer from getenv() for lttng create * Ship license text file with the release archive Project website: http://lttng.org/lttng2.0 Download link: http://lttng.org/files/lttng-tools/lttng-tools-2.0.3.tar.bz2 Cheers! David -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBAgAGBQJP5I7xAAoJEELoaioR9I02bewIAIsCrtibTXuNFlJ1L8GsUkOu kkhLUhOXPP9SF2hA/ghoxX2j3cY4Kr4sKZClpWNqcdJA6K6N9WCttUtavilQXUHM q0H7hozp+o3065aMOmWstFLXvKgIRtUnp53rhAzmny9wQsuay3zItvhagk/zQ/ZX 50n8DcmGzp1C5O9GXUsNhstn6CPEi7lKx9Ico21VX4Pu0aFOgue5/qlw9AHzF3T1 Q58L4OTQPHqGTbLpMeYDv7yG3Th3tglkI4ZN9lXyLjpobQM3NaROVgXdF6YJhXyP XOtgMoqAzrjtIRIQfP/fhjKCBJkrjt6Nl7a9M3WpmfY9QbKGaTctjPQGsxp43Xw= =0eqd -----END PGP SIGNATURE----- From josh at joshtriplett.org Fri Jun 22 15:55:48 2012 From: josh at joshtriplett.org (Josh Triplett) Date: Fri, 22 Jun 2012 12:55:48 -0700 Subject: [lttng-dev] [rp] [RFC] Userspace RCU library internal error handling In-Reply-To: <20120622152231.GA10788@Krystal> References: <20120621164113.GA21197@Krystal> <20120621185941.GC26361@leaf> <20120621190306.GB24179@Krystal> <20120621192824.GE26361@leaf> <20120621194838.GB25571@Krystal> <20120621212102.GF26361@leaf> <20120622152231.GA10788@Krystal> Message-ID: <20120622195548.GB18467@leaf> On Fri, Jun 22, 2012 at 11:22:31AM -0400, Mathieu Desnoyers wrote: > * Josh Triplett (josh at joshtriplett.org) wrote: > > #undef NDEBUG > > #include > > > > Or if you don't consider that sufficient for some reason, you could > > define your own assert(), but that seems like an odd thing to not count > > on. Nonetheless, if you define your own assert, I'd still suggest > > making it look as much like assert() as possible, including the call to > > abort(). > > #undef NDEBUG is unwanted, due to its side-effects. We use "assert()" in > other locations of the code, for which we want the assertion check to be > disabled if NDEBUG is defined in production. Ah, fair enough. > I agree with you that calling "abort()" is exactly what we want, and > it's much more standard that sending a signal chosen with a fair roll of > dices. How about the following ? > > [...] > diff --git a/urcu-die.h b/urcu-die.h > new file mode 100644 > index 0000000..227c8dc > --- /dev/null > +++ b/urcu-die.h > @@ -0,0 +1,37 @@ > +#ifndef _URCU_DIE_H > +#define _URCU_DIE_H > + > +/* > + * urcu-die.h > + * > + * Userspace RCU library unrecoverable error handling > + * > + * Copyright (c) 2012 Mathieu Desnoyers > + * > + * This library is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * This library is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with this library; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include > +#include > +#include > + > +#define urcu_die(cause) \ > +do { \ > + fprintf(stderr, "(" __FILE__ ":%s@%u) Unrecoverable error: %s\n", \ > + __func__, __LINE__, strerror(cause)); \ > + abort(); \ > +} while (0) > + > +#endif /* _URCU_DIE_H */ > [...] That looks reasonable; you've effectively recreated assert, but in a form that will always have the same effect regardless of NDEBUG. - Josh Triplett From sindhu4sind at yahoo.com Sat Jun 23 11:23:55 2012 From: sindhu4sind at yahoo.com (sindu sindhu) Date: Sat, 23 Jun 2012 08:23:55 -0700 (PDT) Subject: [lttng-dev] (no subject) Message-ID: <1340465035.65309.YahooMailNeo@web160106.mail.bf1.yahoo.com> http://www.billgoldberg.com/newsite/wp-content/themes/dilapidated/google.html?orqm=gsy.hsm&frf=ar.hkml&gwyj=ixpa -------------- next part -------------- An HTML attachment was scrubbed... URL: From zheng.chang at emc.com Mon Jun 25 03:32:45 2012 From: zheng.chang at emc.com (changz) Date: Mon, 25 Jun 2012 15:32:45 +0800 Subject: [lttng-dev] Some questions about Lttng In-Reply-To: <20120621124532.GA7255@Krystal> References: <20120619163655.GC6743@Krystal> <6539770C71C3814BB0BFC2DBEBD105087FC418@CORPUSMX30B.corp.emc.com> <20120620130338.GA25746@Krystal> <6539770C71C3814BB0BFC2DBEBD105087FC8FC@CORPUSMX30B.corp.emc.com> <20120621124532.GA7255@Krystal> Message-ID: <4FE8141D.3030107@emc.com> On 6/21/2012 20:45 PM, Mathieu Desnoyers wrote: > * Zheng.Chang at emc.com (Zheng.Chang at emc.com) wrote: >>> -----Original Message----- >>> From: Mathieu Desnoyers [mailto:mathieu.desnoyers at efficios.com] > [...] >>>> 8. What does disable-event command of lttng do? With the >>>> example(hello) provided by lttng-ust, I enabled all events with '-a >>>> -u' and then disabled them again. I launched the example with gdb and >>>> dumped the tracepoint's structure and then found its state was active. >>>> It's supposed to be inactive here, right? >>> Can you provide the detail of commands you launched, and the result of >>> gdb printout ? Please run "hello" with LTTNG_UST_DEBUG=1 env. var set. >>> >>>> BTW: I didn't see any trace generated here with view command. >>> That is after the disable, right ? >> Yes >> >>> Also, did you do a lttng start at some point in your test ? >> >> Yes. Following three examples(A/B/C) provide the testing info for question 8: >> >> Example A: None event >> lttng create hello >> lttng list hello >> Tracing session hello: [active] >> Trace path: /root/lttng-traces/hello-20120621-071913 >> >> lttng start >> >> gdb ./hello >> (gdb) file .libs/hello >> (gdb) set env LTTNG_UST_DEBUG=1 >> (gdb) show env >> ... >> LTTNG_UST_DEBUG=1 >> (gdb) br main >> Breakpoint 1 at 0x80489b9: file hello.c, line 84. >> (gdb) r >> Starting program: /root/project/lttng/lttng-ust-2.0.2/tests/hello/.libs/hello >> [Thread debugging using libthread_db enabled] >> liblttng_ust_tracepoint[3341/3341]: just registered a tracepoints section from 0xb7fddd64 and having 1 tracepoints (in tracepoint_register_lib() at tracepoint.c:638) >> liblttng_ust_tracepoint[3341/3341]: registered tracepoint: lttng_ust:metadata (in tracepoint_register_lib() at tracepoint.c:643) >> libust[3341/3341]: just registered probe lttng_ust containing 1 events (in ltt_probe_register() at ltt-probes.c:109) >> libust[3341/3341]: Registered event probe "lttng_ust:metadata" with signature "const char *, str" (in ltt_probe_register() at ltt-probes.c:118) >> libust[3341/3341]: LTT : ltt ring buffer client init >> (in ltt_ring_buffer_metadata_client_init() at ltt-ring-buffer-metadata-client.h:334) >> libust[3341/3341]: LTT : ltt ring buffer client init >> (in ltt_ring_buffer_client_overwrite_init() at ltt-ring-buffer-client.h:584) >> libust[3341/3341]: LTT : ltt ring buffer client init >> (in ltt_ring_buffer_client_discard_init() at ltt-ring-buffer-client.h:584) >> [New Thread 0xb7dddb70 (LWP 3344)] >> [New Thread 0xb75dcb70 (LWP 3345)] >> libust[3341/3344]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) >> libust[3341/3344]: Waiting for local apps sessiond (in wait_for_sessiond() at lttng-ust-comm.c:621) >> libust[3341/3344]: Linux kernels 2.6.33 to 3.0 (with the exception of stable versions) do not support FUTEX_WAKE on read-only memory mappings correctly. Please upgrade your kernel (fix is commit 9ea71503a8ed9184d2d0b8ccc4d269d05f7940ae in Linux kernel mainline). LTTng-UST will use polling mode fallback. (in wait_for_sessiond() at lttng-ust-comm.c:634) >> libust[3341/3344]: Error: futex: Bad address (in wait_for_sessiond() at lttng-ust-comm.c:636) >> libust[3341/3344]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) >> libust[3341/3345]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3341/3345]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3341/3345]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3341/3345]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3341/3341]: just registered probe ust_tests_hello containing 2 events (in ltt_probe_register() at ltt-probes.c:109) >> libust[3341/3341]: Registered event probe "ust_tests_hello:tptest" with signature "int, anint, int, netint, long *, values, char *, text, size_t, textlen, double, doublearg, float, floatarg" (in ltt_probe_register() at ltt-probes.c:118) >> libust[3341/3341]: Registered event probe "ust_tests_hello:tptest_sighandler" with signature "" (in ltt_probe_register() at ltt-probes.c:118) >> liblttng_ust_tracepoint[3341/3341]: just registered a tracepoints section from 0x804b6c4 and having 2 tracepoints (in tracepoint_register_lib() at tracepoint.c:638) >> liblttng_ust_tracepoint[3341/3341]: registered tracepoint: ust_tests_hello:tptest_sighandler (in tracepoint_register_lib() at tracepoint.c:643) >> liblttng_ust_tracepoint[3341/3341]: registered tracepoint: ust_tests_hello:tptest (in tracepoint_register_lib() at tracepoint.c:643) >> >> Breakpoint 1, main (argc=1, argv=0xbffff754) at hello.c:84 >> 84 if (argc == 2) >> Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.47.el6_2.12.i686 libuuid-2.17.2-12.4.el6.i686 >> (gdb) p __tracepoint_ust_tests_hello___tptest >> $1 = {name = 0x804a380 "ust_tests_hello:tptest", state = 0, probes = 0x0, tracepoint_provider_ref = 0x804b724, >> signature = 0x8049240 "int, anint, int, netint, long *, values, char *, text, size_t, textlen, double, doublearg, float, floatarg", padding = '\000' } >> >> Notice here the __tracepoint_ust_tests_hello___tptest.state and __tracepoint_ust_tests_hello___tptest.probes both are 0, which means no register and API tracepoint doesn't invoke probe. >> >> Example B: enable event ust_tests_hello:tptest >> lttng create hello >> lttng enable-event ust_tests_hello:tptest -u >> lttng list hello >> Tracing session hello: [inactive] >> Trace path: /root/lttng-traces/hello-20120621-073315 >> >> === Domain: UST global === >> >> Channels: >> ------------- >> - channel0: [enabled] >> >> Attributes: >> overwrite mode: 0 >> subbufers size: 4096 >> number of subbufers: 4 >> switch timer interval: 0 >> read timer interval: 200 >> output: mmap() >> >> Events: >> ust_tests_hello:tptest (type: tracepoint) [enabled] >> lttng start >> gdb ./hello >> (gdb) file .libs/hello >> (gdb) set env LTTNG_UST_DEBUG=1 >> (gdb) show env >> ... >> LTTNG_UST_DEBUG=1 >> (gdb) br main >> Breakpoint 1 at 0x80489b9: file hello.c, line 84. >> (gdb) r >> Starting program: /root/project/lttng/lttng-ust-2.0.2/tests/hello/.libs/hello >> [Thread debugging using libthread_db enabled] >> liblttng_ust_tracepoint[3365/3365]: just registered a tracepoints section from 0xb7fddd64 and having 1 tracepoints (in tracepoint_register_lib() at tracepoint.c:638) >> liblttng_ust_tracepoint[3365/3365]: registered tracepoint: lttng_ust:metadata (in tracepoint_register_lib() at tracepoint.c:643) >> libust[3365/3365]: just registered probe lttng_ust containing 1 events (in ltt_probe_register() at ltt-probes.c:109) >> libust[3365/3365]: Registered event probe "lttng_ust:metadata" with signature "const char *, str" (in ltt_probe_register() at ltt-probes.c:118) >> libust[3365/3365]: LTT : ltt ring buffer client init >> (in ltt_ring_buffer_metadata_client_init() at ltt-ring-buffer-metadata-client.h:334) >> libust[3365/3365]: LTT : ltt ring buffer client init >> (in ltt_ring_buffer_client_overwrite_init() at ltt-ring-buffer-client.h:584) >> libust[3365/3365]: LTT : ltt ring buffer client init >> (in ltt_ring_buffer_client_discard_init() at ltt-ring-buffer-client.h:584) >> [New Thread 0xb7dddb70 (LWP 3368)] >> libust[3365/3368]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) >> [New Thread 0xb75dcb70 (LWP 3369)] >> libust[3365/3368]: Waiting for local apps sessiond (in wait_for_sessiond() at lttng-ust-comm.c:621) >> libust[3365/3368]: Linux kernels 2.6.33 to 3.0 (with the exception of stable versions) do not support FUTEX_WAKE on read-only memory mappings correctly. Please upgrade your kernel (fix is commit 9ea71503a8ed9184d2d0b8ccc4d269d05f7940ae in Linux kernel mainline). LTTng-UST will use polling mode fallback. (in wait_for_sessiond() at lttng-ust-comm.c:634) >> libust[3365/3368]: Error: futex: Bad address (in wait_for_sessiond() at lttng-ust-comm.c:636) >> libust[3365/3368]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) >> libust[3365/3369]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3365/3369]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3365/3369]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3365/3369]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3365/3369]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> liblttng_ust_tracepoint[3365/3369]: Registering probe to tracepoint lttng_ust:metadata (in __tracepoint_probe_register() at tracepoint.c:426) >> libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3365/3369]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3365/3369]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3365/3369]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3365/3369]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3365/3369]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3365/3369]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3365/3369]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3365/3369]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3365/3365]: just registered probe ust_tests_hello containing 2 events (in ltt_probe_register() at ltt-probes.c:109) >> libust[3365/3365]: Registered event probe "ust_tests_hello:tptest" with signature "int, anint, int, netint, long *, values, char *, text, size_t, textlen, double, doublearg, float, floatarg" (in ltt_probe_register() at ltt-probes.c:118) >> liblttng_ust_tracepoint[3365/3365]: Registering probe to tracepoint ust_tests_hello:tptest (in __tracepoint_probe_register() at tracepoint.c:426) >> libust[3365/3365]: Registered event probe "ust_tests_hello:tptest_sighandler" with signature "" (in ltt_probe_register() at ltt-probes.c:118) >> liblttng_ust_tracepoint[3365/3365]: just registered a tracepoints section from 0x804b6c4 and having 2 tracepoints (in tracepoint_register_lib() at tracepoint.c:638) >> liblttng_ust_tracepoint[3365/3365]: registered tracepoint: ust_tests_hello:tptest_sighandler (in tracepoint_register_lib() at tracepoint.c:643) >> liblttng_ust_tracepoint[3365/3365]: registered tracepoint: ust_tests_hello:tptest (in tracepoint_register_lib() at tracepoint.c:643) >> >> Breakpoint 1, main (argc=1, argv=0xbffff754) at hello.c:84 >> 84 if (argc == 2) >> Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.47.el6_2.12.i686 libuuid-2.17.2-12.4.el6.i686 >> (gdb) p __tracepoint_ust_tests_hello___tptest >> $1 = {name = 0x804a380 "ust_tests_hello:tptest", state = 1, probes = 0x804c258, tracepoint_provider_ref = 0x804b724, >> signature = 0x8049240 "int, anint, int, netint, long *, values, char *, text, size_t, textlen, double, doublearg, float, floatarg", padding = '\000' } >> >> We can see __tracepoint_ust_tests_hello___tptest.state and __tracepoint_ust_tests_hello___tptest.probes have been set, which means API tracepoint can invoke probe callback function and generate trace info. >> >> Example C: enable event and then disable it >> lttng create hello >> Session hello created. >> Traces will be written in /root/lttng-traces/hello-20120621-074221 >> lttng enable-event ust_tests_hello:tptest -u >> UST event ust_tests_hello:tptest created in channel channel0 >> lttng disable-event ust_tests_hello:tptest -u >> UST event ust_tests_hello:tptest disabled in channel channel0 for session hello >> lttng list hello >> Tracing session hello: [inactive] >> Trace path: /root/lttng-traces/hello-20120621-074221 >> >> === Domain: UST global === >> >> Channels: >> ------------- >> - channel0: [enabled] >> >> Attributes: >> overwrite mode: 0 >> subbufers size: 4096 >> number of subbufers: 4 >> switch timer interval: 0 >> read timer interval: 200 >> output: mmap() >> >> Events: >> ust_tests_hello:tptest (type: tracepoint) [disabled] >> >> lttng start >> gdb ./hello >> (gdb) file .libs/hello >> (gdb) set env LTTNG_UST_DEBUG=1 >> (gdb) show env >> ... >> LTTNG_UST_DEBUG=1 >> (gdb) br main >> Breakpoint 1 at 0x80489b9: file hello.c, line 84. >> (gdb) r >> Starting program: /root/project/lttng/lttng-ust-2.0.2/tests/hello/.libs/hello >> [Thread debugging using libthread_db enabled] >> liblttng_ust_tracepoint[3384/3384]: just registered a tracepoints section from 0xb7fddd64 and having 1 tracepoints (in tracepoint_register_lib() at tracepoint.c:638) >> liblttng_ust_tracepoint[3384/3384]: registered tracepoint: lttng_ust:metadata (in tracepoint_register_lib() at tracepoint.c:643) >> libust[3384/3384]: just registered probe lttng_ust containing 1 events (in ltt_probe_register() at ltt-probes.c:109) >> libust[3384/3384]: Registered event probe "lttng_ust:metadata" with signature "const char *, str" (in ltt_probe_register() at ltt-probes.c:118) >> libust[3384/3384]: LTT : ltt ring buffer client init >> (in ltt_ring_buffer_metadata_client_init() at ltt-ring-buffer-metadata-client.h:334) >> libust[3384/3384]: LTT : ltt ring buffer client init >> (in ltt_ring_buffer_client_overwrite_init() at ltt-ring-buffer-client.h:584) >> libust[3384/3384]: LTT : ltt ring buffer client init >> (in ltt_ring_buffer_client_discard_init() at ltt-ring-buffer-client.h:584) >> [New Thread 0xb7dddb70 (LWP 3387)] >> [New Thread 0xb75dcb70 (LWP 3388)] >> libust[3384/3387]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) >> libust[3384/3387]: Waiting for local apps sessiond (in wait_for_sessiond() at lttng-ust-comm.c:621) >> libust[3384/3387]: Linux kernels 2.6.33 to 3.0 (with the exception of stable versions) do not support FUTEX_WAKE on read-only memory mappings correctly. Please upgrade your kernel (fix is commit 9ea71503a8ed9184d2d0b8ccc4d269d05f7940ae in Linux kernel mainline). LTTng-UST will use polling mode fallback. (in wait_for_sessiond() at lttng-ust-comm.c:634) >> libust[3384/3387]: Error: futex: Bad address (in wait_for_sessiond() at lttng-ust-comm.c:636) >> libust[3384/3387]: Info: sessiond not accepting connections to local apps socket (in ust_listener_thread() at lttng-ust-comm.c:699) >> libust[3384/3388]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3384/3388]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3384/3388]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3384/3388]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3384/3388]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3384/3388]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> liblttng_ust_tracepoint[3384/3388]: Registering probe to tracepoint lttng_ust:metadata (in __tracepoint_probe_register() at tracepoint.c:426) >> libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3384/3388]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3384/3388]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3384/3388]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3384/3388]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3384/3388]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3384/3388]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3384/3388]: message received >> (in ust_listener_thread() at lttng-ust-comm.c:766) >> libust[3384/3388]: message successfully sent (in send_reply() at lttng-ust-comm.c:202) >> libust[3384/3384]: just registered probe ust_tests_hello containing 2 events (in ltt_probe_register() at ltt-probes.c:109) >> libust[3384/3384]: Registered event probe "ust_tests_hello:tptest" with signature "int, anint, int, netint, long *, values, char *, text, size_t, textlen, double, doublearg, float, floatarg" (in ltt_probe_register() at ltt-probes.c:118) >> liblttng_ust_tracepoint[3384/3384]: Registering probe to tracepoint ust_tests_hello:tptest (in __tracepoint_probe_register() at tracepoint.c:426) >> libust[3384/3384]: Registered event probe "ust_tests_hello:tptest_sighandler" with signature "" (in ltt_probe_register() at ltt-probes.c:118) >> liblttng_ust_tracepoint[3384/3384]: just registered a tracepoints section from 0x804b6c4 and having 2 tracepoints (in tracepoint_register_lib() at tracepoint.c:638) >> liblttng_ust_tracepoint[3384/3384]: registered tracepoint: ust_tests_hello:tptest_sighandler (in tracepoint_register_lib() at tracepoint.c:643) >> liblttng_ust_tracepoint[3384/3384]: registered tracepoint: ust_tests_hello:tptest (in tracepoint_register_lib() at tracepoint.c:643) >> >> Breakpoint 1, main (argc=1, argv=0xbffff754) at hello.c:84 >> 84 if (argc == 2) >> Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.47.el6_2.12.i686 libuuid-2.17.2-12.4.el6.i686 >> (gdb) p __tracepoint_ust_tests_hello___tptest >> $1 = {name = 0x804a380 "ust_tests_hello:tptest", state = 1, probes = 0x804c258, tracepoint_provider_ref = 0x804b724, >> signature = 0x8049240 "int, anint, int, netint, long *, values, char *, text, size_t, textlen, double, doublearg, float, floatarg", padding = '\000' } >> >> Here __tracepoint_ust_tests_hello___tptest.state and __tracepoint_ust_tests_hello___tptest.probes have been set, which means probe callback function wiil be called by API tracepoint. But the event has been disabled, is this necessary to call into the probe again? If this behavior is normal, what's suppoed to do with disable-event command? >> >> Any misunderstanding please fix me. Thanks > disabling an event ends up calling, in the program: > > lttng-ust-abi.c:lttng_event_cmd() with "LTTNG_UST_DISABLE", which calls > ltt_event_disable(). Looking at this function: > > int ltt_event_disable(struct ltt_event *event) > { > int old; > > if (event->chan == event->chan->session->metadata) > return -EPERM; > old = uatomic_xchg(&event->enabled, 0); > if (!old) > return -EEXIST; > return 0; > } > > shows us that disabling an event sets the "enabled" flag within the > event (struct ltt_event) to 0. This flag is tested in > include/lttng/ust-tracepoint-event.h: > > #undef TRACEPOINT_EVENT_CLASS > #define TRACEPOINT_EVENT_CLASS(_provider, _name, _args, _fields) \ > static void __event_probe__##_provider##___##_name(_TP_ARGS_DATA_PROTO(_args))\ > [...] > if (caa_unlikely(!CMM_ACCESS_ONCE(__chan->session->active))) \ > return; \ > if (caa_unlikely(!CMM_ACCESS_ONCE(__chan->enabled))) \ > return; \ > if (caa_unlikely(!CMM_ACCESS_ONCE(__event->enabled))) \ > return; > > What you are looking at with the debugger are the tracepoint call sites, > which still have a handler connected to them when you disable the event. > So yes, in theory, we could lessen the overhead of the enable followed > by disable case and ensure that we disconnect the probe from the call > site, but diminishing the performance impact of this rare use-case has > not been high on my priority list so far. > > Hoping that my explanation helps clarifying things, Mathieu, thank you so much. May I know if following two features have been in your schedule list? If yes, what's the target release? a. trace_printf API b. dynamic tracing for lttng-ust Best regards Zheng > Thanks, > > Mathieu > >> Best Regards >> Zheng >> >> >>> Thanks, >>> Mathieu >>> >>> Thanks all for your useful info! >>> >>> Best regards >>> Zheng >>> >>>>> _______________________________________________ >>>>> lttng-dev mailing list >>>>> lttng-dev at lists.lttng.org >>>>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev >>> >>>> -- >>>> Mathieu Desnoyers >>>> Operating System Efficiency R&D Consultant >>>> EfficiOS Inc. >>>> http://www.efficios.com >>>> _______________________________________________ >>>> lttng-dev mailing list >>>> lttng-dev at lists.lttng.org >>>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev >>> -- >>> Mathieu Desnoyers >>> Operating System Efficiency R&D Consultant >>> EfficiOS Inc. >>> http://www.efficios.com From henrik at fiktivkod.org Mon Jun 25 07:22:50 2012 From: henrik at fiktivkod.org (Henrik Hautakoski) Date: Mon, 25 Jun 2012 13:22:50 +0200 Subject: [lttng-dev] [PATCH 1/2] liblttng-ust/lttng-ust-comm.c: fixing typo. Message-ID: <1340623371-22228-1-git-send-email-henrik@fiktivkod.org> Signed-off-by: Henrik Hautakoski --- liblttng-ust/lttng-ust-comm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/liblttng-ust/lttng-ust-comm.c b/liblttng-ust/lttng-ust-comm.c index 4104a3f..deca7cb 100644 --- a/liblttng-ust/lttng-ust-comm.c +++ b/liblttng-ust/lttng-ust-comm.c @@ -681,7 +681,7 @@ error: * This thread does not allocate any resource, except within * handle_message, within mutex protection. This mutex protects against * fork and exit. - * The other moment it allocates resources is at socket connexion, which + * The other moment it allocates resources is at socket connection, which * is also protected by the mutex. */ static -- 1.7.9.5 From henrik at fiktivkod.org Mon Jun 25 07:22:51 2012 From: henrik at fiktivkod.org (Henrik Hautakoski) Date: Mon, 25 Jun 2012 13:22:51 +0200 Subject: [lttng-dev] [PATCH 2/2] liblttng-ust-comm/lttng-ust-com.c: remove unnecessary goto in ustcomm_accept_unix_sock() In-Reply-To: <1340623371-22228-1-git-send-email-henrik@fiktivkod.org> References: <1340623371-22228-1-git-send-email-henrik@fiktivkod.org> Message-ID: <1340623371-22228-2-git-send-email-henrik@fiktivkod.org> Signed-off-by: Henrik Hautakoski --- liblttng-ust-comm/lttng-ust-comm.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/liblttng-ust-comm/lttng-ust-comm.c b/liblttng-ust-comm/lttng-ust-comm.c index 078b56a..1e2fd1a 100644 --- a/liblttng-ust-comm/lttng-ust-comm.c +++ b/liblttng-ust-comm/lttng-ust-comm.c @@ -172,13 +172,9 @@ int ustcomm_accept_unix_sock(int sock) new_fd = accept(sock, (struct sockaddr *) &sun, &len); if (new_fd < 0) { perror("accept"); - goto error; + return -1; } - return new_fd; - -error: - return -1; } /* -- 1.7.9.5 From alexmonthy at voxpopuli.im Mon Jun 25 19:42:40 2012 From: alexmonthy at voxpopuli.im (Alexandre Montplaisir) Date: Mon, 25 Jun 2012 19:42:40 -0400 Subject: [lttng-dev] [UST PATCH] Add a multi-thread version of the 'hello' test program Message-ID: <1340667760-28686-1-git-send-email-alexmonthy@voxpopuli.im> Signed-off-by: Alexandre Montplaisir --- .gitignore | 1 + configure.ac | 1 + tests/Makefile.am | 2 +- tests/hello-mt/Makefile.am | 13 +++++ tests/hello-mt/Makefile.example.bsd | 8 +++ tests/hello-mt/Makefile.example.linux | 8 +++ tests/hello-mt/README | 11 ++++ tests/hello-mt/hello.c | 103 +++++++++++++++++++++++++++++++++ tests/hello-mt/tp.c | 18 ++++++ tests/hello-mt/ust_tests_hello.h | 67 +++++++++++++++++++++ 10 files changed, 231 insertions(+), 1 deletion(-) create mode 100644 tests/hello-mt/Makefile.am create mode 100644 tests/hello-mt/Makefile.example.bsd create mode 100644 tests/hello-mt/Makefile.example.linux create mode 100644 tests/hello-mt/README create mode 100644 tests/hello-mt/hello.c create mode 100644 tests/hello-mt/tp.c create mode 100644 tests/hello-mt/ust_tests_hello.h diff --git a/.gitignore b/.gitignore index 076cd00..c9b45f1 100644 --- a/.gitignore +++ b/.gitignore @@ -39,6 +39,7 @@ tests/exit-fast/exit-fast tests/fork/fork tests/fork/fork2 tests/hello/hello +tests/hello-mt/hello tests/hello.cxx/hello tests/hello2/hello2 tests/libustctl_function_tests/libustctl_function_tests diff --git a/configure.ac b/configure.ac index 294d457..0a50ed0 100644 --- a/configure.ac +++ b/configure.ac @@ -305,6 +305,7 @@ AC_CONFIG_FILES([ tools/Makefile tests/Makefile tests/hello/Makefile + tests/hello-mt/Makefile tests/hello-static-lib/Makefile tests/hello.cxx/Makefile tests/demo/Makefile diff --git a/tests/Makefile.am b/tests/Makefile.am index e79ab7c..443b407 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -1,4 +1,4 @@ -SUBDIRS = . hello hello-static-lib fork ust-basic-tracing ust-multi-test demo hello.cxx +SUBDIRS = . hello hello-mt hello-static-lib fork ust-basic-tracing ust-multi-test demo hello.cxx #SUBDIRS = . hello2 basic basic_long simple_include snprintf test-nevents test-libustinstr-malloc dlopen same_line_marker trace_event register_test tracepoint libustctl_function_tests exit-fast dist_noinst_SCRIPTS = test_loop runtests trace_matches diff --git a/tests/hello-mt/Makefile.am b/tests/hello-mt/Makefile.am new file mode 100644 index 0000000..4b7e16d --- /dev/null +++ b/tests/hello-mt/Makefile.am @@ -0,0 +1,13 @@ +AM_CPPFLAGS = -I$(top_srcdir)/include -Wsystem-headers +AM_CFLAGS = -fopenmp + +noinst_PROGRAMS = hello +hello_SOURCES = hello.c tp.c ust_tests_hello.h +hello_LDADD = $(top_builddir)/liblttng-ust/liblttng-ust.la + +if LTTNG_UST_BUILD_WITH_LIBDL +hello_LDADD += -ldl +endif +if LTTNG_UST_BUILD_WITH_LIBC_DL +hello_LDADD += -lc +endif diff --git a/tests/hello-mt/Makefile.example.bsd b/tests/hello-mt/Makefile.example.bsd new file mode 100644 index 0000000..a71f478 --- /dev/null +++ b/tests/hello-mt/Makefile.example.bsd @@ -0,0 +1,8 @@ +# Example makefile for build outside of the LTTng-UST tree. + +hello: + ${CC} -O2 -I. -o hello -lc -llttng-ust -fopenmp hello.c tp.c + +.PHONY: clean +clean: + rm -f hello diff --git a/tests/hello-mt/Makefile.example.linux b/tests/hello-mt/Makefile.example.linux new file mode 100644 index 0000000..bc5e58f --- /dev/null +++ b/tests/hello-mt/Makefile.example.linux @@ -0,0 +1,8 @@ +# Example makefile for build outside of the LTTng-UST tree. + +hello: + ${CC} -O2 -I. -Wall -o hello hello.c tp.c -ldl -llttng-ust -fopenmp + +.PHONY: clean +clean: + rm -f hello diff --git a/tests/hello-mt/README b/tests/hello-mt/README new file mode 100644 index 0000000..0584dca --- /dev/null +++ b/tests/hello-mt/README @@ -0,0 +1,11 @@ +This is a multi-threaded version of the "hello" test program. It uses OpenMP for +parallelization, and as such requires at least GCC 4.2. + +You can pass one integer as argument when running the program, this will +indicate how many threads to use. For example + +./hello 10 + +will run the test with 10 threads. If no argument is passed, the default of +1 thread is used, in which case the behaviour should be similar to the original +"hello" program. diff --git a/tests/hello-mt/hello.c b/tests/hello-mt/hello.c new file mode 100644 index 0000000..2ff9869 --- /dev/null +++ b/tests/hello-mt/hello.c @@ -0,0 +1,103 @@ +/* + * Copyright (C) 2009 Pierre-Marc Fournier + * Copyright (C) 2011 Mathieu Desnoyers + * Copyright (C) 2012 Alexandre Montplaisir + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; version 2.1 of + * the License. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define TRACEPOINT_DEFINE +#include "ust_tests_hello.h" + +#define NB_ITERATIONS 1000000 + +void inthandler(int sig) +{ + printf("in SIGUSR1 handler\n"); + tracepoint(ust_tests_hello, tptest_sighandler); +} + +int init_int_handler(void) +{ + int result; + struct sigaction act; + + memset(&act, 0, sizeof(act)); + result = sigemptyset(&act.sa_mask); + if (result == -1) { + perror("sigemptyset"); + return -1; + } + + act.sa_handler = inthandler; + act.sa_flags = SA_RESTART; + + /* Only defer ourselves. Also, try to restart interrupted + * syscalls to disturb the traced program as little as possible. + */ + result = sigaction(SIGUSR1, &act, NULL); + if (result == -1) { + perror("sigaction"); + return -1; + } + + return 0; +} + +int main(int argc, char **argv) +{ + int i, netint; + long values[] = { 1, 2, 3 }; + char text[10] = "test"; + double dbl = 2.0; + float flt = 2222.0; + int nb_threads = 1; + bool mybool = 123; /* should print "1" */ + + init_int_handler(); + + if (argc == 2) { + nb_threads = atoi(argv[1]); + } + + fprintf(stderr, "Running %d iterations with %d threads... ", + NB_ITERATIONS, nb_threads); + + #pragma omp parallel private(i, netint) num_threads(nb_threads) + { + for (i = 0; i < NB_ITERATIONS; i++) { + netint = htonl(i); + tracepoint(ust_tests_hello, tptest, i, netint, values, + text, strlen(text), dbl, flt, mybool); + //usleep(100000); + } + } + fprintf(stderr, " done.\n"); + return 0; +} diff --git a/tests/hello-mt/tp.c b/tests/hello-mt/tp.c new file mode 100644 index 0000000..1806b42 --- /dev/null +++ b/tests/hello-mt/tp.c @@ -0,0 +1,18 @@ +/* + * tp.c + * + * Copyright (c) 2011 Mathieu Desnoyers + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + */ + +#define TRACEPOINT_CREATE_PROBES +#include "ust_tests_hello.h" diff --git a/tests/hello-mt/ust_tests_hello.h b/tests/hello-mt/ust_tests_hello.h new file mode 100644 index 0000000..b06dea0 --- /dev/null +++ b/tests/hello-mt/ust_tests_hello.h @@ -0,0 +1,67 @@ +#undef TRACEPOINT_PROVIDER +#define TRACEPOINT_PROVIDER ust_tests_hello + +#if !defined(_TRACEPOINT_UST_TESTS_HELLO_H) || defined(TRACEPOINT_HEADER_MULTI_READ) +#define _TRACEPOINT_UST_TESTS_HELLO_H + +#ifdef __cplusplus +extern "C" { +#endif + +/* + * Copyright (C) 2011 Mathieu Desnoyers + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + */ + +#include +#include + +TRACEPOINT_EVENT(ust_tests_hello, tptest, + TP_ARGS(int, anint, int, netint, long *, values, + char *, text, size_t, textlen, + double, doublearg, float, floatarg, + bool, boolarg), + TP_FIELDS( + ctf_integer(int, intfield, anint) + ctf_integer_hex(int, intfield2, anint) + ctf_integer(long, longfield, anint) + ctf_integer_network(int, netintfield, netint) + ctf_integer_network_hex(int, netintfieldhex, netint) + ctf_array(long, arrfield1, values, 3) + ctf_array_text(char, arrfield2, text, 10) + ctf_sequence(char, seqfield1, text, + size_t, textlen) + ctf_sequence_text(char, seqfield2, text, + size_t, textlen) + ctf_string(stringfield, text) + ctf_float(float, floatfield, floatarg) + ctf_float(double, doublefield, doublearg) + ctf_integer(bool, boolfield, boolarg) + ) +) + +TRACEPOINT_EVENT(ust_tests_hello, tptest_sighandler, + TP_ARGS(), + TP_FIELDS() +) + +#endif /* _TRACEPOINT_UST_TESTS_HELLO_H */ + +#undef TRACEPOINT_INCLUDE_FILE +#define TRACEPOINT_INCLUDE_FILE ./ust_tests_hello.h + +/* This part must be outside ifdef protection */ +#include + +#ifdef __cplusplus +} +#endif -- 1.7.10.4 From mathieu.desnoyers at efficios.com Tue Jun 26 01:52:27 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 26 Jun 2012 01:52:27 -0400 Subject: [lttng-dev] [PATCH 1/2] liblttng-ust/lttng-ust-comm.c: fixing typo. In-Reply-To: <1340623371-22228-1-git-send-email-henrik@fiktivkod.org> References: <1340623371-22228-1-git-send-email-henrik@fiktivkod.org> Message-ID: <20120626055226.GA11445@Krystal> * Henrik Hautakoski (henrik at fiktivkod.org) wrote: > Signed-off-by: Henrik Hautakoski merged, thanks! Mathieu > --- > liblttng-ust/lttng-ust-comm.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/liblttng-ust/lttng-ust-comm.c b/liblttng-ust/lttng-ust-comm.c > index 4104a3f..deca7cb 100644 > --- a/liblttng-ust/lttng-ust-comm.c > +++ b/liblttng-ust/lttng-ust-comm.c > @@ -681,7 +681,7 @@ error: > * This thread does not allocate any resource, except within > * handle_message, within mutex protection. This mutex protects against > * fork and exit. > - * The other moment it allocates resources is at socket connexion, which > + * The other moment it allocates resources is at socket connection, which > * is also protected by the mutex. > */ > static > -- > 1.7.9.5 > > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Tue Jun 26 02:00:43 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 26 Jun 2012 02:00:43 -0400 Subject: [lttng-dev] [PATCH 2/2] liblttng-ust-comm/lttng-ust-com.c: remove unnecessary goto in ustcomm_accept_unix_sock() In-Reply-To: <1340623371-22228-2-git-send-email-henrik@fiktivkod.org> References: <1340623371-22228-1-git-send-email-henrik@fiktivkod.org> <1340623371-22228-2-git-send-email-henrik@fiktivkod.org> Message-ID: <20120626060043.GB11445@Krystal> * Henrik Hautakoski (henrik at fiktivkod.org) wrote: > Signed-off-by: Henrik Hautakoski In this specific case, this is right that the goto appears to be really useless. Although I would not merge this kind of change in all cases: if in another situation the function appears to be bound to have other calls in the future, using gotos allow to do the teardown of the function in error cases in a symmetric way: a = fct(); if (!a) goto error_a; b = fct2(); if (!b) goto error_b; return 0; /* OK */ error_b: teardown(a); error_a: return -1; So the intend in ustcomm_accept_unix_sock() was to ensure that as the function grows to contain more calls, the teardown can be done symmetrically in this way. But I guess it appears that it won't grow, so I'm taking the cleanup. Thanks, Mathieu > --- > liblttng-ust-comm/lttng-ust-comm.c | 6 +----- > 1 file changed, 1 insertion(+), 5 deletions(-) > > diff --git a/liblttng-ust-comm/lttng-ust-comm.c b/liblttng-ust-comm/lttng-ust-comm.c > index 078b56a..1e2fd1a 100644 > --- a/liblttng-ust-comm/lttng-ust-comm.c > +++ b/liblttng-ust-comm/lttng-ust-comm.c > @@ -172,13 +172,9 @@ int ustcomm_accept_unix_sock(int sock) > new_fd = accept(sock, (struct sockaddr *) &sun, &len); > if (new_fd < 0) { > perror("accept"); > - goto error; > + return -1; > } > - > return new_fd; > - > -error: > - return -1; > } > > /* > -- > 1.7.9.5 > > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Tue Jun 26 02:03:56 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 26 Jun 2012 02:03:56 -0400 Subject: [lttng-dev] [UST PATCH] Add a multi-thread version of the 'hello' test program In-Reply-To: <1340667760-28686-1-git-send-email-alexmonthy@voxpopuli.im> References: <1340667760-28686-1-git-send-email-alexmonthy@voxpopuli.im> Message-ID: <20120626060356.GC11445@Krystal> * Alexandre Montplaisir (alexmonthy at voxpopuli.im) wrote: > Signed-off-by: Alexandre Montplaisir > --- > .gitignore | 1 + > configure.ac | 1 + > tests/Makefile.am | 2 +- > tests/hello-mt/Makefile.am | 13 +++++ > tests/hello-mt/Makefile.example.bsd | 8 +++ > tests/hello-mt/Makefile.example.linux | 8 +++ > tests/hello-mt/README | 11 ++++ > tests/hello-mt/hello.c | 103 +++++++++++++++++++++++++++++++++ > tests/hello-mt/tp.c | 18 ++++++ > tests/hello-mt/ust_tests_hello.h | 67 +++++++++++++++++++++ > 10 files changed, 231 insertions(+), 1 deletion(-) > create mode 100644 tests/hello-mt/Makefile.am > create mode 100644 tests/hello-mt/Makefile.example.bsd > create mode 100644 tests/hello-mt/Makefile.example.linux > create mode 100644 tests/hello-mt/README > create mode 100644 tests/hello-mt/hello.c > create mode 100644 tests/hello-mt/tp.c > create mode 100644 tests/hello-mt/ust_tests_hello.h > > diff --git a/.gitignore b/.gitignore > index 076cd00..c9b45f1 100644 > --- a/.gitignore > +++ b/.gitignore > @@ -39,6 +39,7 @@ tests/exit-fast/exit-fast > tests/fork/fork > tests/fork/fork2 > tests/hello/hello > +tests/hello-mt/hello > tests/hello.cxx/hello > tests/hello2/hello2 > tests/libustctl_function_tests/libustctl_function_tests > diff --git a/configure.ac b/configure.ac > index 294d457..0a50ed0 100644 > --- a/configure.ac > +++ b/configure.ac > @@ -305,6 +305,7 @@ AC_CONFIG_FILES([ > tools/Makefile > tests/Makefile > tests/hello/Makefile > + tests/hello-mt/Makefile > tests/hello-static-lib/Makefile > tests/hello.cxx/Makefile > tests/demo/Makefile > diff --git a/tests/Makefile.am b/tests/Makefile.am > index e79ab7c..443b407 100644 > --- a/tests/Makefile.am > +++ b/tests/Makefile.am > @@ -1,4 +1,4 @@ > -SUBDIRS = . hello hello-static-lib fork ust-basic-tracing ust-multi-test demo hello.cxx > +SUBDIRS = . hello hello-mt hello-static-lib fork ust-basic-tracing ust-multi-test demo hello.cxx > #SUBDIRS = . hello2 basic basic_long simple_include snprintf test-nevents test-libustinstr-malloc dlopen same_line_marker trace_event register_test tracepoint libustctl_function_tests exit-fast > > dist_noinst_SCRIPTS = test_loop runtests trace_matches > diff --git a/tests/hello-mt/Makefile.am b/tests/hello-mt/Makefile.am > new file mode 100644 > index 0000000..4b7e16d > --- /dev/null > +++ b/tests/hello-mt/Makefile.am > @@ -0,0 +1,13 @@ > +AM_CPPFLAGS = -I$(top_srcdir)/include -Wsystem-headers > +AM_CFLAGS = -fopenmp > + > +noinst_PROGRAMS = hello > +hello_SOURCES = hello.c tp.c ust_tests_hello.h > +hello_LDADD = $(top_builddir)/liblttng-ust/liblttng-ust.la > + > +if LTTNG_UST_BUILD_WITH_LIBDL > +hello_LDADD += -ldl > +endif > +if LTTNG_UST_BUILD_WITH_LIBC_DL > +hello_LDADD += -lc > +endif > diff --git a/tests/hello-mt/Makefile.example.bsd b/tests/hello-mt/Makefile.example.bsd > new file mode 100644 > index 0000000..a71f478 > --- /dev/null > +++ b/tests/hello-mt/Makefile.example.bsd > @@ -0,0 +1,8 @@ > +# Example makefile for build outside of the LTTng-UST tree. > + > +hello: > + ${CC} -O2 -I. -o hello -lc -llttng-ust -fopenmp hello.c tp.c > + > +.PHONY: clean > +clean: > + rm -f hello > diff --git a/tests/hello-mt/Makefile.example.linux b/tests/hello-mt/Makefile.example.linux > new file mode 100644 > index 0000000..bc5e58f > --- /dev/null > +++ b/tests/hello-mt/Makefile.example.linux > @@ -0,0 +1,8 @@ > +# Example makefile for build outside of the LTTng-UST tree. > + > +hello: > + ${CC} -O2 -I. -Wall -o hello hello.c tp.c -ldl -llttng-ust -fopenmp > + > +.PHONY: clean > +clean: > + rm -f hello > diff --git a/tests/hello-mt/README b/tests/hello-mt/README > new file mode 100644 > index 0000000..0584dca > --- /dev/null > +++ b/tests/hello-mt/README > @@ -0,0 +1,11 @@ > +This is a multi-threaded version of the "hello" test program. It uses OpenMP for > +parallelization, and as such requires at least GCC 4.2. Hrm. This adds a dependency on gcc 4.2. The entire project will fail to build because of this test on other compiler, older gcc, right ? We should have a configure.ac check that tests openmp availability. So I won't merge this for now. Thanks, Mathieu > + > +You can pass one integer as argument when running the program, this will > +indicate how many threads to use. For example > + > +./hello 10 > + > +will run the test with 10 threads. If no argument is passed, the default of > +1 thread is used, in which case the behaviour should be similar to the original > +"hello" program. > diff --git a/tests/hello-mt/hello.c b/tests/hello-mt/hello.c > new file mode 100644 > index 0000000..2ff9869 > --- /dev/null > +++ b/tests/hello-mt/hello.c > @@ -0,0 +1,103 @@ > +/* > + * Copyright (C) 2009 Pierre-Marc Fournier > + * Copyright (C) 2011 Mathieu Desnoyers > + * Copyright (C) 2012 Alexandre Montplaisir > + * > + * This library is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; version 2.1 of > + * the License. > + * > + * This library is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with this library; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#define TRACEPOINT_DEFINE > +#include "ust_tests_hello.h" > + > +#define NB_ITERATIONS 1000000 > + > +void inthandler(int sig) > +{ > + printf("in SIGUSR1 handler\n"); > + tracepoint(ust_tests_hello, tptest_sighandler); > +} > + > +int init_int_handler(void) > +{ > + int result; > + struct sigaction act; > + > + memset(&act, 0, sizeof(act)); > + result = sigemptyset(&act.sa_mask); > + if (result == -1) { > + perror("sigemptyset"); > + return -1; > + } > + > + act.sa_handler = inthandler; > + act.sa_flags = SA_RESTART; > + > + /* Only defer ourselves. Also, try to restart interrupted > + * syscalls to disturb the traced program as little as possible. > + */ > + result = sigaction(SIGUSR1, &act, NULL); > + if (result == -1) { > + perror("sigaction"); > + return -1; > + } > + > + return 0; > +} > + > +int main(int argc, char **argv) > +{ > + int i, netint; > + long values[] = { 1, 2, 3 }; > + char text[10] = "test"; > + double dbl = 2.0; > + float flt = 2222.0; > + int nb_threads = 1; > + bool mybool = 123; /* should print "1" */ > + > + init_int_handler(); > + > + if (argc == 2) { > + nb_threads = atoi(argv[1]); > + } > + > + fprintf(stderr, "Running %d iterations with %d threads... ", > + NB_ITERATIONS, nb_threads); > + > + #pragma omp parallel private(i, netint) num_threads(nb_threads) > + { > + for (i = 0; i < NB_ITERATIONS; i++) { > + netint = htonl(i); > + tracepoint(ust_tests_hello, tptest, i, netint, values, > + text, strlen(text), dbl, flt, mybool); > + //usleep(100000); > + } > + } > + fprintf(stderr, " done.\n"); > + return 0; > +} > diff --git a/tests/hello-mt/tp.c b/tests/hello-mt/tp.c > new file mode 100644 > index 0000000..1806b42 > --- /dev/null > +++ b/tests/hello-mt/tp.c > @@ -0,0 +1,18 @@ > +/* > + * tp.c > + * > + * Copyright (c) 2011 Mathieu Desnoyers > + * > + * Permission is hereby granted, free of charge, to any person obtaining a copy > + * of this software and associated documentation files (the "Software"), to deal > + * in the Software without restriction, including without limitation the rights > + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell > + * copies of the Software, and to permit persons to whom the Software is > + * furnished to do so, subject to the following conditions: > + * > + * The above copyright notice and this permission notice shall be included in > + * all copies or substantial portions of the Software. > + */ > + > +#define TRACEPOINT_CREATE_PROBES > +#include "ust_tests_hello.h" > diff --git a/tests/hello-mt/ust_tests_hello.h b/tests/hello-mt/ust_tests_hello.h > new file mode 100644 > index 0000000..b06dea0 > --- /dev/null > +++ b/tests/hello-mt/ust_tests_hello.h > @@ -0,0 +1,67 @@ > +#undef TRACEPOINT_PROVIDER > +#define TRACEPOINT_PROVIDER ust_tests_hello > + > +#if !defined(_TRACEPOINT_UST_TESTS_HELLO_H) || defined(TRACEPOINT_HEADER_MULTI_READ) > +#define _TRACEPOINT_UST_TESTS_HELLO_H > + > +#ifdef __cplusplus > +extern "C" { > +#endif > + > +/* > + * Copyright (C) 2011 Mathieu Desnoyers > + * > + * Permission is hereby granted, free of charge, to any person obtaining a copy > + * of this software and associated documentation files (the "Software"), to deal > + * in the Software without restriction, including without limitation the rights > + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell > + * copies of the Software, and to permit persons to whom the Software is > + * furnished to do so, subject to the following conditions: > + * > + * The above copyright notice and this permission notice shall be included in > + * all copies or substantial portions of the Software. > + */ > + > +#include > +#include > + > +TRACEPOINT_EVENT(ust_tests_hello, tptest, > + TP_ARGS(int, anint, int, netint, long *, values, > + char *, text, size_t, textlen, > + double, doublearg, float, floatarg, > + bool, boolarg), > + TP_FIELDS( > + ctf_integer(int, intfield, anint) > + ctf_integer_hex(int, intfield2, anint) > + ctf_integer(long, longfield, anint) > + ctf_integer_network(int, netintfield, netint) > + ctf_integer_network_hex(int, netintfieldhex, netint) > + ctf_array(long, arrfield1, values, 3) > + ctf_array_text(char, arrfield2, text, 10) > + ctf_sequence(char, seqfield1, text, > + size_t, textlen) > + ctf_sequence_text(char, seqfield2, text, > + size_t, textlen) > + ctf_string(stringfield, text) > + ctf_float(float, floatfield, floatarg) > + ctf_float(double, doublefield, doublearg) > + ctf_integer(bool, boolfield, boolarg) > + ) > +) > + > +TRACEPOINT_EVENT(ust_tests_hello, tptest_sighandler, > + TP_ARGS(), > + TP_FIELDS() > +) > + > +#endif /* _TRACEPOINT_UST_TESTS_HELLO_H */ > + > +#undef TRACEPOINT_INCLUDE_FILE > +#define TRACEPOINT_INCLUDE_FILE ./ust_tests_hello.h > + > +/* This part must be outside ifdef protection */ > +#include > + > +#ifdef __cplusplus > +} > +#endif > -- > 1.7.10.4 > > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Tue Jun 26 02:06:09 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 26 Jun 2012 02:06:09 -0400 Subject: [lttng-dev] Some questions about Lttng In-Reply-To: <4FE8141D.3030107@emc.com> References: <20120619163655.GC6743@Krystal> <6539770C71C3814BB0BFC2DBEBD105087FC418@CORPUSMX30B.corp.emc.com> <20120620130338.GA25746@Krystal> <6539770C71C3814BB0BFC2DBEBD105087FC8FC@CORPUSMX30B.corp.emc.com> <20120621124532.GA7255@Krystal> <4FE8141D.3030107@emc.com> Message-ID: <20120626060609.GD11445@Krystal> * changz (zheng.chang at emc.com) wrote: [...] > Mathieu, thank you so much. > May I know if following two features have been in your schedule list? If > yes, what's the target release? > a. trace_printf API In schedule list, by the end of 2012. > b. dynamic tracing for lttng-ust No ETA at the moment for this feature. It's a "wishlist" feature, but it is not planned for any specific point in time. Thanks, Mathieu -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Tue Jun 26 02:40:16 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 26 Jun 2012 02:40:16 -0400 Subject: [lttng-dev] [PATCH] Expose kernel tracer to user-space In-Reply-To: <1338938654-16515-1-git-send-email-francis.giraldeau@gmail.com> References: <1338938654-16515-1-git-send-email-francis.giraldeau@gmail.com> Message-ID: <20120626064016.GD11952@Krystal> * Francis Giraldeau (francis.giraldeau at gmail.com) wrote: > By writing to the file /proc/lttng, a user-space application creates a > kernel event. The event's payload is by default UTF-8 text, but any data > can be written, up to 1024 bytes. Null-character is optional and is not > enforced. The event uses sequence for space efficiency and to store any > data as payload. > > Update: split the probe code into it's own module and make it an optional > feature of lttng-abi. The feature is enabled when the module is loaded. The > lttng-abi module exports a register function and includes a wrapper for > lttng_fops write. This is required since struct file_operations must be const. > Since the module dependency is reversed, unloading the lttng-uevent module is > done only when it's not used anymore. This is done with rwlock synchronisation. > The synchronisation doesn't prevent starvation, but this situation is unlikely > and can be prevented by stop active tracing sessions. > > Signed-off-by: Francis Giraldeau > --- > instrumentation/events/lttng-module/uevent.h | 33 +++++++++++++++ > lttng-abi.c | 42 +++++++++++++++++++ > lttng-abi.h | 3 ++ > probes/Makefile | 2 + > probes/lttng-probe-uevent.c | 36 ++++++++++++++++ > probes/lttng-uevent.c | 58 ++++++++++++++++++++++++++ > 6 files changed, 174 insertions(+) > create mode 100644 instrumentation/events/lttng-module/uevent.h > create mode 100644 probes/lttng-probe-uevent.c > create mode 100644 probes/lttng-uevent.c > > diff --git a/instrumentation/events/lttng-module/uevent.h b/instrumentation/events/lttng-module/uevent.h > new file mode 100644 > index 0000000..f67d901 > --- /dev/null > +++ b/instrumentation/events/lttng-module/uevent.h > @@ -0,0 +1,33 @@ > +#undef TRACE_SYSTEM > +#define TRACE_SYSTEM uevent > + > +#if !defined(UEVENT_H_) || defined(TRACE_HEADER_MULTI_READ) > +#define UEVENT_H_ > + > +#include > + > +TRACE_EVENT(lttng_uevent, > + > + TP_PROTO(const char *str, size_t len), > + > + TP_ARGS(str, len), > + > + /* > + * Uses sequence to hold variable size data, by default considered > + * as text. Null-terminal character is optional and is not enforced. > + */ > + TP_STRUCT__entry( > + __dynamic_array_text(char, text, len) > + ), > + > + TP_fast_assign( > + tp_memcpy_dyn_from_user(text, str) > + ), > + > + TP_printk("") > +) > + > +#endif /* UEVENT_H_ */ > + > +/* This part must be outside protection */ > +#include "../../../probes/define_trace.h" > diff --git a/lttng-abi.c b/lttng-abi.c > index 26a02ed..b8e6b57 100644 > --- a/lttng-abi.c > +++ b/lttng-abi.c > @@ -51,6 +51,12 @@ > #include "lttng-tracer.h" > > /* > + * Required data structures to support lttng-probe-uevent > + */ > +DEFINE_RWLOCK(uevent_rwlock); Not required. > +write_ops_t lttng_uevent_handler; this should be static. > + > +/* > * This is LTTng's own personal way to create a system call as an external > * module. We use ioctl() on /proc/lttng. > */ > @@ -252,9 +258,45 @@ long lttng_ioctl(struct file *file, unsigned int cmd, unsigned long arg) > } > } > > +/* > + * lttng_uevent_set_handler - set handler functions for uevent > + * > + * Access to handler code is protected with rwlock in order to > + * prevent the optional module to be removed while in use. > + */ > + > +void lttng_uevent_set_handler(write_ops_t handler) > +{ > + write_lock(&uevent_rwlock); write lock not necessary. if (!lttng_uevent_set_handler) release refcount in prior handler's module. then: take a refcount on the module that contains the handler address. (explicit) > + lttng_uevent_handler = handler; Just declare the lttng_uevent_handler as "volatile". > + write_unlock(&uevent_rwlock); > +} > +EXPORT_SYMBOL_GPL(lttng_uevent_set_handler); > + > +/* > + * lttng_write_uevent - expose kernel tracer to user-space > + */ > + > +static > +ssize_t lttng_write_uevent(struct file *file, const char __user *ubuf, > + size_t count, loff_t *fpos) > +{ > + int ret; > + > + read_lock(&uevent_rwlock); > + if (unlikely(lttng_uevent_handler == NULL)) { > + read_unlock(&uevent_rwlock); > + return -ENOSYS; instead of read_lock, please do: write_ops_t uev_handler; uev_handler = ACCESS_ONCE(lttng_uevent_handler); if (!uev_handler) return -ENOSYS; return uev_handler(file, ubuf, count, fpos); Thanks, Mathieu > + } > + ret = (*lttng_uevent_handler)(file, ubuf, count, fpos); > + read_unlock(&uevent_rwlock); > + return ret; > +} > + > static const struct file_operations lttng_fops = { > .owner = THIS_MODULE, The THIS_MODULE owner protects the module that contains lttng-abi.c from unloading when the handling is running. (keeps a refcount) However, it does not protect the handler probe module from unloading. This is why we need a refcount. > .unlocked_ioctl = lttng_ioctl, > + .write = lttng_write_uevent, > #ifdef CONFIG_COMPAT > .compat_ioctl = lttng_ioctl, > #endif > diff --git a/lttng-abi.h b/lttng-abi.h > index dc230d8..f4a8c0c 100644 > --- a/lttng-abi.h > +++ b/lttng-abi.h > @@ -27,6 +27,9 @@ > > #define LTTNG_KERNEL_SYM_NAME_LEN 256 > > +typedef ssize_t (*write_ops_t) (struct file *, const char __user *, size_t, loff_t *); > +void lttng_uevent_set_handler(write_ops_t handler); > + > enum lttng_kernel_instrumentation { > LTTNG_KERNEL_TRACEPOINT = 0, > LTTNG_KERNEL_KPROBE = 1, > diff --git a/probes/Makefile b/probes/Makefile > index 698a9c9..a895e60 100644 > --- a/probes/Makefile > +++ b/probes/Makefile > @@ -14,6 +14,8 @@ obj-m += lttng-probe-sched.o > obj-m += lttng-probe-irq.o > obj-m += lttng-probe-signal.o > obj-m += lttng-probe-timer.o > +obj-m += lttng-probe-uevent.o > +obj-m += lttng-uevent.o > > obj-m += lttng-probe-statedump.o > > diff --git a/probes/lttng-probe-uevent.c b/probes/lttng-probe-uevent.c > new file mode 100644 > index 0000000..90abb5e > --- /dev/null > +++ b/probes/lttng-probe-uevent.c > @@ -0,0 +1,36 @@ > +/* > + * probes/lttng-probe-uevent.c > + * > + * Expose kernel tracer to user-space through /proc/lttng > + * > + * Copyright (C) 2009-2012 Mathieu Desnoyers > + * > + * This library is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; only > + * version 2.1 of the License. > + * > + * This library is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with this library; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include > + > +/* > + * Create lttng_uevent tracepoint probes. > + */ > +#define LTTNG_PACKAGE_BUILD > +#define CREATE_TRACE_POINTS > +#define TRACE_INCLUDE_PATH ../instrumentation/events/lttng-module > + > +#include "../instrumentation/events/lttng-module/uevent.h" > + > +MODULE_LICENSE("GPL and additional rights"); > +MODULE_AUTHOR("Mathieu Desnoyers "); > +MODULE_DESCRIPTION("LTTng uevent probes"); > diff --git a/probes/lttng-uevent.c b/probes/lttng-uevent.c > new file mode 100644 > index 0000000..7b4bffc > --- /dev/null > +++ b/probes/lttng-uevent.c > @@ -0,0 +1,58 @@ > +/* > + * probes/lttng-uevent.c > + * > + * Expose kernel tracer to user-space through /proc/lttng > + * > + * Copyright (C) 2012 Mathieu Desnoyers > + * > + * This library is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; only > + * version 2.1 of the License. > + * > + * This library is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with this library; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include > +#include "../lttng-abi.h" > + > +/* include our own uevent tracepoint */ > +#include "../instrumentation/events/lttng-module/uevent.h" > +DEFINE_TRACE(lttng_uevent); > + > +#define LTTNG_UEVENT_SIZE 1024 > + > +ssize_t uevent_write_handler(struct file *file, const char __user *ubuf, > + size_t count, loff_t *fpos) > +{ > + if (count > LTTNG_UEVENT_SIZE) > + count = LTTNG_UEVENT_SIZE; > + > + trace_lttng_uevent(ubuf, count); > + return count; > +} > + > +static int __init lttng_probe_uevent_init(void) > +{ > + lttng_uevent_set_handler(uevent_write_handler); > + return 0; > +} > + > +static void __exit lttng_probe_uevent_exit(void) > +{ > + lttng_uevent_set_handler(NULL); > +} > + > +module_init(lttng_probe_uevent_init); > +module_exit(lttng_probe_uevent_exit); > + > +MODULE_LICENSE("GPL and additional rights"); > +MODULE_AUTHOR("Mathieu Desnoyers "); > +MODULE_DESCRIPTION("LTTng kernel event from user-space"); > -- > 1.7.9.5 > > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From alexmonthy at voxpopuli.im Tue Jun 26 03:11:30 2012 From: alexmonthy at voxpopuli.im (Alexandre Montplaisir) Date: Tue, 26 Jun 2012 03:11:30 -0400 Subject: [lttng-dev] [UST PATCH] Add a multi-thread version of the 'hello' test program In-Reply-To: <20120626060356.GC11445@Krystal> References: <1340667760-28686-1-git-send-email-alexmonthy@voxpopuli.im> <20120626060356.GC11445@Krystal> Message-ID: <4FE960A2.7080109@voxpopuli.im> On 12-06-26 02:03 AM, Mathieu Desnoyers wrote: > * Alexandre Montplaisir (alexmonthy at voxpopuli.im) wrote: >> [...] >> diff --git a/tests/hello-mt/README b/tests/hello-mt/README >> new file mode 100644 >> index 0000000..0584dca >> --- /dev/null >> +++ b/tests/hello-mt/README >> @@ -0,0 +1,11 @@ >> +This is a multi-threaded version of the "hello" test program. It uses OpenMP for >> +parallelization, and as such requires at least GCC 4.2. > Hrm. This adds a dependency on gcc 4.2. The entire project will fail to > build because of this test on other compiler, older gcc, right ? We > should have a configure.ac check that tests openmp availability. So I > won't merge this for now. Staying up late, aren't we? ;) Oh ok, I thought we said the other day that GCC 4.2 was old enough to not be worth testing for. In any case, here's a patch to check for OpenMP at configure time (goes on top of the previous one). Two caveats however: - Since the main part of the project doesn't use OpenMP, I don't think we want to turn the "-fopenmp" CFLAG on for the whole thing. So I just hard-coded "-fopenmp" for this test only. I don't know if any other compiler uses some other flag to turn on the OpenMP support, but we only support GCC anyway, don't we? - The autoconf macro requires Autoconf 2.62. Current Debian stable has 2.67, I don't know if it's a big problem. Good night, Alex > > Thanks, > > Mathieu > > -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Only-build-the-hello-mt-test-if-OpenMP-is-available.patch Type: text/x-patch Size: 2182 bytes Desc: not available URL: From zheng.chang at emc.com Tue Jun 26 04:59:55 2012 From: zheng.chang at emc.com (changz) Date: Tue, 26 Jun 2012 16:59:55 +0800 Subject: [lttng-dev] Some questions about Lttng In-Reply-To: <20120626060609.GD11445@Krystal> References: <20120619163655.GC6743@Krystal> <6539770C71C3814BB0BFC2DBEBD105087FC418@CORPUSMX30B.corp.emc.com> <20120620130338.GA25746@Krystal> <6539770C71C3814BB0BFC2DBEBD105087FC8FC@CORPUSMX30B.corp.emc.com> <20120621124532.GA7255@Krystal> <4FE8141D.3030107@emc.com> <20120626060609.GD11445@Krystal> Message-ID: <4FE97A0B.4070608@emc.com> On 6/26/2012 14:06 PM, Mathieu Desnoyers wrote: > * changz (zheng.chang at emc.com) wrote: > [...] >> Mathieu, thank you so much. >> May I know if following two features have been in your schedule list? If >> yes, what's the target release? >> a. trace_printf API > In schedule list, by the end of 2012. > >> b. dynamic tracing for lttng-ust > No ETA at the moment for this feature. It's a "wishlist" feature, but > it is not planned for any specific point in time. I see. Thanks for your info. Best Regards Zheng > Thanks, > > Mathieu > From mathieu.desnoyers at efficios.com Tue Jun 26 08:02:23 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 26 Jun 2012 08:02:23 -0400 Subject: [lttng-dev] [UST PATCH] Add a multi-thread version of the 'hello' test program In-Reply-To: <4FE960A2.7080109@voxpopuli.im> References: <1340667760-28686-1-git-send-email-alexmonthy@voxpopuli.im> <20120626060356.GC11445@Krystal> <4FE960A2.7080109@voxpopuli.im> Message-ID: <20120626120222.GA17072@Krystal> * Alexandre Montplaisir (alexmonthy at voxpopuli.im) wrote: > On 12-06-26 02:03 AM, Mathieu Desnoyers wrote: > > * Alexandre Montplaisir (alexmonthy at voxpopuli.im) wrote: > >> [...] > >> diff --git a/tests/hello-mt/README b/tests/hello-mt/README > >> new file mode 100644 > >> index 0000000..0584dca > >> --- /dev/null > >> +++ b/tests/hello-mt/README > >> @@ -0,0 +1,11 @@ > >> +This is a multi-threaded version of the "hello" test program. It uses OpenMP for > >> +parallelization, and as such requires at least GCC 4.2. > > Hrm. This adds a dependency on gcc 4.2. The entire project will fail to > > build because of this test on other compiler, older gcc, right ? We > > should have a configure.ac check that tests openmp availability. So I > > won't merge this for now. > > Staying up late, aren't we? ;) Nah, I'm in Sweden at the moment, I was doing my morning email catchup ;) > > Oh ok, I thought we said the other day that GCC 4.2 was old enough to > not be worth testing for. In any case, here's a patch to check for > OpenMP at configure time (goes on top of the previous one). > > Two caveats however: > - Since the main part of the project doesn't use OpenMP, I don't think > we want to turn the "-fopenmp" CFLAG on for the whole thing. So I just > hard-coded "-fopenmp" for this test only. I don't know if any other > compiler uses some other flag to turn on the OpenMP support, but we only > support GCC anyway, don't we? Specific to the hello-mt directory makes sense. gcc-specific makes sense too I guess, since we detect its availability with autoconf. > > - The autoconf macro requires Autoconf 2.62. Current Debian stable has > 2.67, I don't know if it's a big problem. Can you respin this patch after looking at: https://bugs.launchpad.net/inkscape/+bug/984836 they check if the AC_OPENMP macro is available, and mark openmp as unavailable if it is not. This would allow us to still only depend on autotools 2.50. Thanks, Mathieu > > > Good night, > Alex > > > > > Thanks, > > > > Mathieu > > > > > > From 496a74a33dc47b61c37b14e0fb0ad5bdeacaf03b Mon Sep 17 00:00:00 2001 > From: Alexandre Montplaisir > Date: Tue, 26 Jun 2012 02:56:19 -0400 > Subject: [UST PATCH] Only build the hello-mt test if OpenMP is available > > Since this test is the only part in the whole tree that uses > OpenMP, we'll simply set the CFLAG manually for this test > rather than using it project-wide. > > Signed-off-by: Alexandre Montplaisir > --- > README | 2 +- > configure.ac | 10 ++++++++++ > tests/hello-mt/Makefile.am | 4 ++++ > 3 files changed, 15 insertions(+), 1 deletion(-) > > diff --git a/README b/README > index 52aebd8..b9df1e6 100644 > --- a/README > +++ b/README > @@ -30,7 +30,7 @@ This source tree is based on the autotools suite from GNU to simplify > portability. Here are some things you should have on your system in order to > compile the git repository tree : > > -- GNU autotools (automake >=1.10, autoconf >=2.50, autoheader >=2.50) > +- GNU autotools (automake >=1.10, autoconf >=2.62, autoheader >=2.62) > (make sure your system wide "automake" points to a recent version!) > - GNU Libtool >=2.2 > (for more information, go to http://www.gnu.org/software/autoconf/) > diff --git a/configure.ac b/configure.ac > index 0a50ed0..1924d0b 100644 > --- a/configure.ac > +++ b/configure.ac > @@ -271,6 +271,16 @@ AS_IF([test "x$with_sdt" = "xyes"],[ > ]) > ]) > > +#Check for OpenMP, if available we will build the test that uses it > +AC_OPENMP() > +AS_IF([ test "x$ac_cv_prog_c_openmp" != "xunsupported" && test "x$ac_cv_prog_c_openmp" != "x"],[ > + openmp_available=yes > +],[ > + openmp_available=no > +]) > +AM_CONDITIONAL([BUILD_OPENMP_TEST], [test "x$openmp_available" = "xyes"]) > + > + > #currently disabled. > #tests/hello2/Makefile > #tests/basic/Makefile > diff --git a/tests/hello-mt/Makefile.am b/tests/hello-mt/Makefile.am > index 4b7e16d..98ecc6f 100644 > --- a/tests/hello-mt/Makefile.am > +++ b/tests/hello-mt/Makefile.am > @@ -1,3 +1,5 @@ > +if BUILD_OPENMP_TEST > + > AM_CPPFLAGS = -I$(top_srcdir)/include -Wsystem-headers > AM_CFLAGS = -fopenmp > > @@ -11,3 +13,5 @@ endif > if LTTNG_UST_BUILD_WITH_LIBC_DL > hello_LDADD += -lc > endif > + > +endif > -- > 1.7.10.4 > -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From yannick.brosseau at gmail.com Tue Jun 26 09:45:54 2012 From: yannick.brosseau at gmail.com (Brosseau, Yannick) Date: Tue, 26 Jun 2012 09:45:54 -0400 Subject: [lttng-dev] [UST PATCH] Add a multi-thread version of the 'hello' test program In-Reply-To: <20120626120222.GA17072@Krystal> References: <1340667760-28686-1-git-send-email-alexmonthy@voxpopuli.im> <20120626060356.GC11445@Krystal> <4FE960A2.7080109@voxpopuli.im> <20120626120222.GA17072@Krystal> Message-ID: On Tue, Jun 26, 2012 at 8:02 AM, Mathieu Desnoyers wrote: > * Alexandre Montplaisir (alexmonthy at voxpopuli.im) wrote: >> On 12-06-26 02:03 AM, Mathieu Desnoyers wrote: >> > * Alexandre Montplaisir (alexmonthy at voxpopuli.im) wrote: >> >> [...] >> >> diff --git a/tests/hello-mt/README b/tests/hello-mt/README >> >> new file mode 100644 >> >> index 0000000..0584dca >> >> --- /dev/null >> >> +++ b/tests/hello-mt/README I was wondering if naming the directement hello-openmp would be more appropriate? (The test is kinda specific tot he use of openmp) Yannick From mathieu.desnoyers at efficios.com Tue Jun 26 09:52:07 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 26 Jun 2012 09:52:07 -0400 Subject: [lttng-dev] [UST PATCH] Add a multi-thread version of the 'hello' test program In-Reply-To: References: <1340667760-28686-1-git-send-email-alexmonthy@voxpopuli.im> <20120626060356.GC11445@Krystal> <4FE960A2.7080109@voxpopuli.im> <20120626120222.GA17072@Krystal> Message-ID: <20120626135207.GA18658@Krystal> * Brosseau, Yannick (yannick.brosseau at gmail.com) wrote: > On Tue, Jun 26, 2012 at 8:02 AM, Mathieu Desnoyers > wrote: > > * Alexandre Montplaisir (alexmonthy at voxpopuli.im) wrote: > >> On 12-06-26 02:03 AM, Mathieu Desnoyers wrote: > >> > * Alexandre Montplaisir (alexmonthy at voxpopuli.im) wrote: > >> >> [...] > >> >> diff --git a/tests/hello-mt/README b/tests/hello-mt/README > >> >> new file mode 100644 > >> >> index 0000000..0584dca > >> >> --- /dev/null > >> >> +++ b/tests/hello-mt/README > > I was wondering if naming the directement hello-openmp would be more > appropriate? (The test is kinda specific tot he use of openmp) good point, makes sense ! Alexandre, can you respin your hello-mp patch and change it to hello-openmp ? Thanks, Mathieu -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From griscom at suitable.com Tue Jun 26 10:06:17 2012 From: griscom at suitable.com (Daniel Griscom) Date: Tue, 26 Jun 2012 10:06:17 -0400 Subject: [lttng-dev] Version of LTTng for kernel 2.6.25? Message-ID: I'd like to use LTTng, but I'm using an old kernel (and can't upgrade): >$ cat /proc/version >Linux version 2.6.25.10-prof (root at atmdev) (gcc version 3.4.6 >(Gentoo Hardened 3.4.6-r1, ssp-3.4.5-1.0, pie-8.7.9)) #5 PREEMPT Sat >Mar 3 16:51:19 EST 2012 I understand that the latest kernel modules require kernel 2.6.38 or later. What is the latest version of the kernel tools that I could run? Are the user-space tools kernel-independent (and LTTng-modules-independent)? Thanks, Dan -- Daniel T. Griscom griscom at suitable.com Suitable Systems http://www.suitable.com/ 1 Centre Street, Suite 204 (781) 665-0053 Wakefield, MA 01880-2400 From francis.giraldeau at gmail.com Tue Jun 26 10:52:01 2012 From: francis.giraldeau at gmail.com (Francis Giraldeau) Date: Tue, 26 Jun 2012 16:52:01 +0200 Subject: [lttng-dev] [PATCH] Expose kernel tracer to user-space In-Reply-To: <20120626064016.GD11952@Krystal> References: <1338938654-16515-1-git-send-email-francis.giraldeau@gmail.com> <20120626064016.GD11952@Krystal> Message-ID: <4FE9CC91.5030301@gmail.com> Le 2012-06-26 08:40, Mathieu Desnoyers a ?crit : >> +/* >> + * lttng_uevent_set_handler - set handler functions for uevent >> + * >> + * Access to handler code is protected with rwlock in order to >> + * prevent the optional module to be removed while in use. >> + */ >> + >> +void lttng_uevent_set_handler(write_ops_t handler) >> +{ >> + write_lock(&uevent_rwlock); > > write lock not necessary. > > if (!lttng_uevent_set_handler) > release refcount in prior handler's module. > > then: > take a refcount on the module that contains the handler address. > (explicit) The function now looks like this: void lttng_uevent_set_handler(struct file_operations *fops) { if (lttng_uevent_handler) { module_put(lttng_uevent_handler->owner); } if (fops && try_module_get(fops->owner)) { lttng_uevent_handler = fops; return; } lttng_uevent_handler = NULL; return; } The problem with this approach is that lttng_uevent can't be unloaded anymore: rmmod lttng_uevent ERROR: Module lttng_uevent is in use The module lttng_uevent can't be unloaded without unloading the module lttng_tracer. The function module_exit of lttng_uevent is never called, such that there is a chicken-and-egg situation. This problem was solved by using rwlock to access the handler pointer. In the rare event when unload module would occur while a trace event is recorded, the write lock was preventing to change the handler pointer while it's in use. But maybe there is another way to do it? Thanks! Francis Giraldeau -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4476 bytes Desc: Signature cryptographique S/MIME URL: From mathieu.desnoyers at efficios.com Tue Jun 26 12:49:57 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 26 Jun 2012 12:49:57 -0400 Subject: [lttng-dev] Version of LTTng for kernel 2.6.25? In-Reply-To: References: Message-ID: <20120626164957.GB23255@Krystal> * Daniel Griscom (griscom at suitable.com) wrote: > I'd like to use LTTng, but I'm using an old kernel (and can't upgrade): > >> $ cat /proc/version >> Linux version 2.6.25.10-prof (root at atmdev) (gcc version 3.4.6 (Gentoo >> Hardened 3.4.6-r1, ssp-3.4.5-1.0, pie-8.7.9)) #5 PREEMPT Sat Mar 3 >> 16:51:19 EST 2012 > > I understand that the latest kernel modules require kernel 2.6.38 or > later. What is the latest version of the kernel tools that I could run? You'll have to dig the old LTTng 0.x kernel tracer series (see legacy on the website). Good luck with that, as it is not supported anymore. > Are the user-space tools kernel-independent (and > LTTng-modules-independent)? Yes, the user-space tracing part is kernel-independent and lttng-modules-independent. However, you will not be able to control the old lttng 0.x kernel tracer with the lttng 2.0 control tools. Good luck! Mathieu > > > Thanks, > Dan > > -- > Daniel T. Griscom griscom at suitable.com > Suitable Systems http://www.suitable.com/ > 1 Centre Street, Suite 204 (781) 665-0053 > Wakefield, MA 01880-2400 > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Tue Jun 26 13:01:09 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 26 Jun 2012 13:01:09 -0400 Subject: [lttng-dev] [PATCH] Expose kernel tracer to user-space In-Reply-To: <4FE9CC91.5030301@gmail.com> References: <1338938654-16515-1-git-send-email-francis.giraldeau@gmail.com> <20120626064016.GD11952@Krystal> <4FE9CC91.5030301@gmail.com> Message-ID: <20120626170109.GC23255@Krystal> * Francis Giraldeau (francis.giraldeau at gmail.com) wrote: > Le 2012-06-26 08:40, Mathieu Desnoyers a ?crit : > >> +/* > >> + * lttng_uevent_set_handler - set handler functions for uevent > >> + * > >> + * Access to handler code is protected with rwlock in order to > >> + * prevent the optional module to be removed while in use. > >> + */ > >> + > >> +void lttng_uevent_set_handler(write_ops_t handler) > >> +{ > >> + write_lock(&uevent_rwlock); > > > > write lock not necessary. > > > > if (!lttng_uevent_set_handler) > > release refcount in prior handler's module. > > > > then: > > take a refcount on the module that contains the handler address. > > (explicit) > > > The function now looks like this: > > void lttng_uevent_set_handler(struct file_operations *fops) > { > if (lttng_uevent_handler) { > module_put(lttng_uevent_handler->owner); > } > if (fops && try_module_get(fops->owner)) { > lttng_uevent_handler = fops; > return; > } > lttng_uevent_handler = NULL; > return; > } > > The problem with this approach is that lttng_uevent can't be unloaded > anymore: > > rmmod lttng_uevent > ERROR: Module lttng_uevent is in use > > The module lttng_uevent can't be unloaded without unloading the module > lttng_tracer. The function module_exit of lttng_uevent is never called, > such that there is a chicken-and-egg situation. This problem was solved > by using rwlock to access the handler pointer. In the rare event when > unload module would occur while a trace event is recorded, the write > lock was preventing to change the handler pointer while it's in use. But > maybe there is another way to do it? Good point, circular dependency on the destructor is not good, because there is no way to unload the module. Another way to do it is to use RCU. module unload contains a synchronize_rcu() call, so you can protect your handler with: /* Using RCU to protect against module removal */ rcu_read_lock(); handler = rcu_dereference(lttng_uevent_handler); ret = handler(...))); rcu_read_unlock; return ret; Thanks, Mathieu > > Thanks! > > Francis Giraldeau > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From yannick.brosseau at gmail.com Tue Jun 26 13:39:45 2012 From: yannick.brosseau at gmail.com (Yannick Brosseau) Date: Tue, 26 Jun 2012 13:39:45 -0400 Subject: [lttng-dev] Update on fedora packaging Message-ID: <4FE9F3E1.5070906@gmail.com> Hi all, A small progress update on the packaging of LTTng for Fedora. Latest Userspace RCU package is in the testing-update queue for Fedora 17 (and in available in rawhide) (https://admin.fedoraproject.org/updates/FEDORA-2012-9686/userspace-rcu-0.7.3-1.fc17) LTTng-UST is now in rawhide. I will push in F17 when URCU get out of the testing queue. LTTng-tools is pending a Review Request (https://bugzilla.redhat.com/show_bug.cgi?id=834481) My plan is to package babeltrace when it reaches 1.0 and lttv when the initial LTTng 2.0 support is finished. Yannicl From mathieu.desnoyers at efficios.com Wed Jun 27 02:47:16 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Wed, 27 Jun 2012 02:47:16 -0400 Subject: [lttng-dev] [RELEASE] LTTng modules 2.0.4 Message-ID: <20120627064716.GA6207@Krystal> The LTTng modules provide Linux kernel tracing capability to the LTTng 2.0 tracer toolset. Changelog: 2012-06-27 LTTng modules 2.0.4 * LTTng: probe-statedump: add #include * fix: signal_generate event should print utf8 for comm field Project website: http://lttng.org Download link: http://lttng.org/download (please refer to the README files for installation instructions and lttng-tools doc/quickstart.txt for usage information) -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Wed Jun 27 09:35:24 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Wed, 27 Jun 2012 09:35:24 -0400 Subject: [lttng-dev] [RELEASE] Userspace RCU 0.7.3 In-Reply-To: References: <20120601180533.GA15522@Krystal> <20120604155143.GA24551@Krystal> Message-ID: <20120627133523.GA13831@Krystal> * Gerhard Mack (gmack at innerfire.net) wrote: > > Just a heads up, I had to add the following to get it working with code > compiled with the gcc flag "-std=c99" > > #ifndef asm > #define asm __asm > #endif Normally, in userspace rcu git master HEAD, you have this commit already: commit e51500edbd9919cee53bc85cbb4b22cd4786fc42 Author: Mathieu Desnoyers Date: Tue Jun 12 11:24:31 2012 -0400 Fix c99 compatibility: use __asm__ and __volatile__ in public headers Signed-off-by: Mathieu Desnoyers Does it fix it for you ? Thanks, Mathieu > > Gerhard > > On Mon, 4 Jun 2012, Mathieu Desnoyers wrote: > > > Date: Mon, 4 Jun 2012 11:51:43 -0400 > > From: Mathieu Desnoyers > > To: Gerhard Mack , > > Alexandre Montplaisir > > Cc: lttng-dev at lists.lttng.org > > Subject: Re: [RELEASE] Userspace RCU 0.7.3 > > > > * Gerhard Mack (gmack at innerfire.net) wrote: > > > > > > Are there any online examples of how to use this library? I can't seem to > > > find any. > > > > The perfbook from Paul McKenney now uses userspace RCU in its examples > > (http://kernel.org/pub/linux/kernel/people/paulmck/perfbook/perfbook.html) > > > > Also, you will find various small programs in the source tree of the > > userspace-rcu packages under tests/ that act as test programs, and also > > show how to use the library. (in the git tree: > > http://git.lttng.org/?p=userspace-rcu.git;a=tree;f=tests;hb=HEAD) > > > > I guess setting up a tutorial HTML page from the test content would be > > valuable, we just have not had the time to do it at this point. Maybe > > setting up links to that documentation on the lttng.org/urcu web page > > would be a good start though. > > > > Alexandre, when you find a minute, can you look into this ? > > > > Thanks! > > > > Mathieu > > > > > > > > > > Gerhard > > > > > > > > > > > > On Fri, 1 Jun 2012, Mathieu Desnoyers wrote: > > > > > > > Date: Fri, 1 Jun 2012 14:05:33 -0400 > > > > From: Mathieu Desnoyers > > > > To: linux-kernel at vger.kernel.org, lttng-dev at lists.lttng.org, > > > > rp at svcs.cs.pdx.edu > > > > Cc: Paul E. McKenney , > > > > Lai Jiangshan , > > > > Stephen Hemminger > > > > Subject: [RELEASE] Userspace RCU 0.7.3 > > > > > > > > liburcu is a LGPLv2.1 userspace RCU (read-copy-update) library. This > > > > data synchronization library provides read-side access which scales > > > > linearly with the number of cores. It does so by allowing multiples > > > > copies of a given data structure to live at the same time, and by > > > > monitoring the data structure accesses to detect grace periods after > > > > which memory reclamation is possible. > > > > > > > > liburcu-cds provides efficient data structures based on RCU and > > > > lock-free algorithms. Those structures include hash tables, queues, > > > > stacks, and doubly-linked lists. > > > > > > > > This is a minor compatibility-related release, fixing build issues with > > > > FreeBSD and NetBSD. On Linux, only the test_perthreadlock fix could > > > > change the result of make check (which could previously fail due to > > > > non-initialized mutexes), but it does not impact anything installed on > > > > the system. > > > > > > > > Changelog: > > > > 2012-06-01 Userspace RCU 0.7.3 > > > > * Fix tests: make dist lib dependency > > > > * Update README for OS supported, tests dependency > > > > * Add CodingStyle to tarball > > > > * Add coding style document > > > > * Test fix: test_perthreadlock uninitialized mutex > > > > * tests: support FreeBSD short "time" args > > > > * freebsd 8.2 fix: define MAP_ANONYMOUS for compatibility > > > > > > > > Project website: http://lttng.org/urcu > > > > Download link: http://lttng.org/files/urcu/ > > > > > > > > > > > > > > -- > > > Gerhard Mack > > > > > > gmack at innerfire.net > > > > > > <>< As a computer, I find your faith in technology amusing. > > > > > > -- > Gerhard Mack > > gmack at innerfire.net > > <>< As a computer, I find your faith in technology amusing. -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From Paul_Woegerer at mentor.com Wed Jun 27 10:08:14 2012 From: Paul_Woegerer at mentor.com (Woegerer, Paul) Date: Wed, 27 Jun 2012 16:08:14 +0200 Subject: [lttng-dev] [PATCH] tracepoint event exclusion Message-ID: <4FEB13CE.1010206@mentor.com> Hi, In order to support situations where a user wants generally all tracepoints enabled but specific tracepoints to be suppressed during a trace session, I created a small patch that allows us to do the following: ... lttng enable-event -u -c met_tools -a lttng enable-event -u -c met_tools --tracepoint '!met_func:*' lttng enable-event -u -c met_tools --tracepoint '!met_call:*' ... This has the effect that all tracepoints get recorded except for tracepoints that match met_func:* or met_call:*. If you think this feature is useful for others we would be happy to see this patch merged into trunk. Thanks, Paul -- Paul Woegerer | SW Development Engineer Mentor Embedded(tm) | Prinz Eugen Stra?e 72/2/4, Vienna, 1040 Austria P 43.1.535991320 Nucleus? | Linux? | Android(tm) | Services | UI | Multi-OS Android is a trademark of Google Inc. Use of this trademark is subject to Google Permissions. Linux is the registered trademark of Linus Torvalds in the U.S. and other countries. -------------- next part -------------- A non-text attachment was scrubbed... Name: event_exclusion.patch Type: text/x-patch Size: 2157 bytes Desc: not available URL: From gmack at innerfire.net Wed Jun 27 10:36:21 2012 From: gmack at innerfire.net (Gerhard Mack) Date: Wed, 27 Jun 2012 10:36:21 -0400 (EDT) Subject: [lttng-dev] [RELEASE] Userspace RCU 0.7.3 In-Reply-To: <20120627133523.GA13831@Krystal> References: <20120601180533.GA15522@Krystal> <20120604155143.GA24551@Krystal> <20120627133523.GA13831@Krystal> Message-ID: Yes, it looks like it does. Sorry for the noise it looks like it picked up debian''s 6.7.2 rather than the newer version. Gerhard On Wed, 27 Jun 2012, Mathieu Desnoyers wrote: > Date: Wed, 27 Jun 2012 09:35:24 -0400 > From: Mathieu Desnoyers > To: Gerhard Mack > Cc: Alexandre Montplaisir , > lttng-dev at lists.lttng.org > Subject: Re: [RELEASE] Userspace RCU 0.7.3 > > * Gerhard Mack (gmack at innerfire.net) wrote: > > > > Just a heads up, I had to add the following to get it working with code > > compiled with the gcc flag "-std=c99" > > > > #ifndef asm > > #define asm __asm > > #endif > > Normally, in userspace rcu git master HEAD, you have this commit > already: > > commit e51500edbd9919cee53bc85cbb4b22cd4786fc42 > Author: Mathieu Desnoyers > Date: Tue Jun 12 11:24:31 2012 -0400 > > Fix c99 compatibility: use __asm__ and __volatile__ in public headers > > Signed-off-by: Mathieu Desnoyers > > Does it fix it for you ? > > Thanks, > > Mathieu > > > > > Gerhard > > > > On Mon, 4 Jun 2012, Mathieu Desnoyers wrote: > > > > > Date: Mon, 4 Jun 2012 11:51:43 -0400 > > > From: Mathieu Desnoyers > > > To: Gerhard Mack , > > > Alexandre Montplaisir > > > Cc: lttng-dev at lists.lttng.org > > > Subject: Re: [RELEASE] Userspace RCU 0.7.3 > > > > > > * Gerhard Mack (gmack at innerfire.net) wrote: > > > > > > > > Are there any online examples of how to use this library? I can't seem to > > > > find any. > > > > > > The perfbook from Paul McKenney now uses userspace RCU in its examples > > > (http://kernel.org/pub/linux/kernel/people/paulmck/perfbook/perfbook.html) > > > > > > Also, you will find various small programs in the source tree of the > > > userspace-rcu packages under tests/ that act as test programs, and also > > > show how to use the library. (in the git tree: > > > http://git.lttng.org/?p=userspace-rcu.git;a=tree;f=tests;hb=HEAD) > > > > > > I guess setting up a tutorial HTML page from the test content would be > > > valuable, we just have not had the time to do it at this point. Maybe > > > setting up links to that documentation on the lttng.org/urcu web page > > > would be a good start though. > > > > > > Alexandre, when you find a minute, can you look into this ? > > > > > > Thanks! > > > > > > Mathieu > > > > > > > > > > > > > > Gerhard > > > > > > > > > > > > > > > > On Fri, 1 Jun 2012, Mathieu Desnoyers wrote: > > > > > > > > > Date: Fri, 1 Jun 2012 14:05:33 -0400 > > > > > From: Mathieu Desnoyers > > > > > To: linux-kernel at vger.kernel.org, lttng-dev at lists.lttng.org, > > > > > rp at svcs.cs.pdx.edu > > > > > Cc: Paul E. McKenney , > > > > > Lai Jiangshan , > > > > > Stephen Hemminger > > > > > Subject: [RELEASE] Userspace RCU 0.7.3 > > > > > > > > > > liburcu is a LGPLv2.1 userspace RCU (read-copy-update) library. This > > > > > data synchronization library provides read-side access which scales > > > > > linearly with the number of cores. It does so by allowing multiples > > > > > copies of a given data structure to live at the same time, and by > > > > > monitoring the data structure accesses to detect grace periods after > > > > > which memory reclamation is possible. > > > > > > > > > > liburcu-cds provides efficient data structures based on RCU and > > > > > lock-free algorithms. Those structures include hash tables, queues, > > > > > stacks, and doubly-linked lists. > > > > > > > > > > This is a minor compatibility-related release, fixing build issues with > > > > > FreeBSD and NetBSD. On Linux, only the test_perthreadlock fix could > > > > > change the result of make check (which could previously fail due to > > > > > non-initialized mutexes), but it does not impact anything installed on > > > > > the system. > > > > > > > > > > Changelog: > > > > > 2012-06-01 Userspace RCU 0.7.3 > > > > > * Fix tests: make dist lib dependency > > > > > * Update README for OS supported, tests dependency > > > > > * Add CodingStyle to tarball > > > > > * Add coding style document > > > > > * Test fix: test_perthreadlock uninitialized mutex > > > > > * tests: support FreeBSD short "time" args > > > > > * freebsd 8.2 fix: define MAP_ANONYMOUS for compatibility > > > > > > > > > > Project website: http://lttng.org/urcu > > > > > Download link: http://lttng.org/files/urcu/ > > > > > > > > > > > > > > > > > > -- > > > > Gerhard Mack > > > > > > > > gmack at innerfire.net > > > > > > > > <>< As a computer, I find your faith in technology amusing. > > > > > > > > > > -- > > Gerhard Mack > > > > gmack at innerfire.net > > > > <>< As a computer, I find your faith in technology amusing. > > -- Gerhard Mack gmack at innerfire.net <>< As a computer, I find your faith in technology amusing. From mathieu.desnoyers at efficios.com Wed Jun 27 12:02:25 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Wed, 27 Jun 2012 12:02:25 -0400 Subject: [lttng-dev] [RELEASE] Userspace RCU 0.7.3 In-Reply-To: References: <20120601180533.GA15522@Krystal> <20120604155143.GA24551@Krystal> <20120627133523.GA13831@Krystal> Message-ID: <20120627160225.GA17076@Krystal> well this fix is in queue for the upcoming 0.7.4, which is not released yet. So yes, it is not present in the last stable release. Thanks, Mathieu * Gerhard Mack (gmack at innerfire.net) wrote: > > > Yes, it looks like it does. Sorry for the noise it looks like it picked > up debian''s 6.7.2 rather than the newer version. > > Gerhard > > > On Wed, 27 Jun 2012, Mathieu Desnoyers wrote: > > > Date: Wed, 27 Jun 2012 09:35:24 -0400 > > From: Mathieu Desnoyers > > To: Gerhard Mack > > Cc: Alexandre Montplaisir , > > lttng-dev at lists.lttng.org > > Subject: Re: [RELEASE] Userspace RCU 0.7.3 > > > > * Gerhard Mack (gmack at innerfire.net) wrote: > > > > > > Just a heads up, I had to add the following to get it working with code > > > compiled with the gcc flag "-std=c99" > > > > > > #ifndef asm > > > #define asm __asm > > > #endif > > > > Normally, in userspace rcu git master HEAD, you have this commit > > already: > > > > commit e51500edbd9919cee53bc85cbb4b22cd4786fc42 > > Author: Mathieu Desnoyers > > Date: Tue Jun 12 11:24:31 2012 -0400 > > > > Fix c99 compatibility: use __asm__ and __volatile__ in public headers > > > > Signed-off-by: Mathieu Desnoyers > > > > Does it fix it for you ? > > > > Thanks, > > > > Mathieu > > > > > > > > Gerhard > > > > > > On Mon, 4 Jun 2012, Mathieu Desnoyers wrote: > > > > > > > Date: Mon, 4 Jun 2012 11:51:43 -0400 > > > > From: Mathieu Desnoyers > > > > To: Gerhard Mack , > > > > Alexandre Montplaisir > > > > Cc: lttng-dev at lists.lttng.org > > > > Subject: Re: [RELEASE] Userspace RCU 0.7.3 > > > > > > > > * Gerhard Mack (gmack at innerfire.net) wrote: > > > > > > > > > > Are there any online examples of how to use this library? I can't seem to > > > > > find any. > > > > > > > > The perfbook from Paul McKenney now uses userspace RCU in its examples > > > > (http://kernel.org/pub/linux/kernel/people/paulmck/perfbook/perfbook.html) > > > > > > > > Also, you will find various small programs in the source tree of the > > > > userspace-rcu packages under tests/ that act as test programs, and also > > > > show how to use the library. (in the git tree: > > > > http://git.lttng.org/?p=userspace-rcu.git;a=tree;f=tests;hb=HEAD) > > > > > > > > I guess setting up a tutorial HTML page from the test content would be > > > > valuable, we just have not had the time to do it at this point. Maybe > > > > setting up links to that documentation on the lttng.org/urcu web page > > > > would be a good start though. > > > > > > > > Alexandre, when you find a minute, can you look into this ? > > > > > > > > Thanks! > > > > > > > > Mathieu > > > > > > > > > > > > > > > > > > Gerhard > > > > > > > > > > > > > > > > > > > > On Fri, 1 Jun 2012, Mathieu Desnoyers wrote: > > > > > > > > > > > Date: Fri, 1 Jun 2012 14:05:33 -0400 > > > > > > From: Mathieu Desnoyers > > > > > > To: linux-kernel at vger.kernel.org, lttng-dev at lists.lttng.org, > > > > > > rp at svcs.cs.pdx.edu > > > > > > Cc: Paul E. McKenney , > > > > > > Lai Jiangshan , > > > > > > Stephen Hemminger > > > > > > Subject: [RELEASE] Userspace RCU 0.7.3 > > > > > > > > > > > > liburcu is a LGPLv2.1 userspace RCU (read-copy-update) library. This > > > > > > data synchronization library provides read-side access which scales > > > > > > linearly with the number of cores. It does so by allowing multiples > > > > > > copies of a given data structure to live at the same time, and by > > > > > > monitoring the data structure accesses to detect grace periods after > > > > > > which memory reclamation is possible. > > > > > > > > > > > > liburcu-cds provides efficient data structures based on RCU and > > > > > > lock-free algorithms. Those structures include hash tables, queues, > > > > > > stacks, and doubly-linked lists. > > > > > > > > > > > > This is a minor compatibility-related release, fixing build issues with > > > > > > FreeBSD and NetBSD. On Linux, only the test_perthreadlock fix could > > > > > > change the result of make check (which could previously fail due to > > > > > > non-initialized mutexes), but it does not impact anything installed on > > > > > > the system. > > > > > > > > > > > > Changelog: > > > > > > 2012-06-01 Userspace RCU 0.7.3 > > > > > > * Fix tests: make dist lib dependency > > > > > > * Update README for OS supported, tests dependency > > > > > > * Add CodingStyle to tarball > > > > > > * Add coding style document > > > > > > * Test fix: test_perthreadlock uninitialized mutex > > > > > > * tests: support FreeBSD short "time" args > > > > > > * freebsd 8.2 fix: define MAP_ANONYMOUS for compatibility > > > > > > > > > > > > Project website: http://lttng.org/urcu > > > > > > Download link: http://lttng.org/files/urcu/ > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Gerhard Mack > > > > > > > > > > gmack at innerfire.net > > > > > > > > > > <>< As a computer, I find your faith in technology amusing. > > > > > > > > > > > > > > -- > > > Gerhard Mack > > > > > > gmack at innerfire.net > > > > > > <>< As a computer, I find your faith in technology amusing. > > > > > > -- > Gerhard Mack > > gmack at innerfire.net > > <>< As a computer, I find your faith in technology amusing. -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Wed Jun 27 14:33:08 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Wed, 27 Jun 2012 14:33:08 -0400 Subject: [lttng-dev] [PATCH] tracepoint event exclusion In-Reply-To: <4FEB13CE.1010206@mentor.com> References: <4FEB13CE.1010206@mentor.com> Message-ID: <20120627183308.GA19735@Krystal> * Woegerer, Paul (Paul_Woegerer at mentor.com) wrote: > Hi, > > In order to support situations where a user wants generally all > tracepoints enabled but specific tracepoints to be suppressed during a > trace session, I created a small patch that allows us to do the > following: > > ... > lttng enable-event -u -c met_tools -a > lttng enable-event -u -c met_tools --tracepoint '!met_func:*' > lttng enable-event -u -c met_tools --tracepoint '!met_call:*' hrm, although possibly interesting, I don't think this semantic follows the lttng UI event enabling semantic. There are a few use-cases to cover: 1) enable tracepoints, start tracing, run app. 2) enable some tracepoints, start tracing, run app, enable more tracepoints while app is running. 3) start tracing, run app, enable all tracepoints while app is running. AFAIU, your patch and proposed semantic covers use-case #1, but breaks the other 2. Basically, we have to apply a logical "or" between each enable-event command (rather than an "and" as your proposed semantic suggests), because they can be applied independently. We could however try to come up with an event attribute that gets applied on a specific event, e.g.: lttng enable-event -u -a --exclude 'met_func:*' --exclude 'met_call:*' thoughts ? Thanks, Mathieu > ... > > This has the effect that all tracepoints get recorded except for > tracepoints that match met_func:* or met_call:*. > > If you think this feature is useful for others we would be happy to see > this patch merged into trunk. > > Thanks, > Paul > > -- > Paul Woegerer | SW Development Engineer > Mentor Embedded(tm) | Prinz Eugen Stra?e 72/2/4, Vienna, 1040 Austria > P 43.1.535991320 > Nucleus? | Linux? | Android(tm) | Services | UI | Multi-OS > > Android is a trademark of Google Inc. Use of this trademark is subject to Google Permissions. > Linux is the registered trademark of Linus Torvalds in the U.S. and other countries. > > From 74602f6f4ea4b1d7faa01f76542f62b496b94091 Mon Sep 17 00:00:00 2001 > From: Paul Woegerer > Date: Wed, 27 Jun 2012 15:21:30 +0200 > Subject: [PATCH] Added excluding/filtering of events with !NAME > > --- > liblttng-ust/ltt-events.c | 33 +++++++++++++++++++++++++++++++-- > 1 file changed, 31 insertions(+), 2 deletions(-) > > diff --git a/liblttng-ust/ltt-events.c b/liblttng-ust/ltt-events.c > index 0fdfd2f..b663312 100644 > --- a/liblttng-ust/ltt-events.c > +++ b/liblttng-ust/ltt-events.c > @@ -150,11 +150,32 @@ struct wildcard_entry *match_wildcard(const struct lttng_event_desc *desc) > struct wildcard_entry *e; > > cds_list_for_each_entry(e, &wildcard_list, list) { > + size_t len = strlen(e->name); > + if (len < 1 || e->name[0] != '!') > + continue; > + > + DBG("Filter %s against wildcard_entry %s\n", desc->name, e->name); > + > + /* Compare exclusion wildcards excluding initial '!' and final '*' */ > + if (!strncmp(desc->name, e->name + 1, len - 2)) > + { > + DBG("Exclude %s due to exclusion wildcard_entry %s\n", desc->name, e->name); > + return NULL; /* exclusion wildcard matched */ > + } > + } > + > + cds_list_for_each_entry(e, &wildcard_list, list) { > + size_t len = strlen(e->name); > + if (len > 0 && e->name[0] == '!') > + continue; > + > + DBG("Match %s against wildcard_entry %s\n", desc->name, e->name); > + > /* If only contain '*' */ > - if (strlen(e->name) == 1) > + if (len == 1) > goto possible_match; > /* Compare excluding final '*' */ > - if (!strncmp(desc->name, e->name, strlen(e->name) - 1)) > + if (!strncmp(desc->name, e->name, len - 1)) > goto possible_match; > continue; /* goto next, no match */ > possible_match: > @@ -507,6 +528,13 @@ int ltt_event_create(struct ltt_channel *chan, > struct ltt_event *event; > int ret = 0; > > + if (strlen(event_param->name) > 0 && event_param->name[0] == '!') > + { > + struct session_wildcard *sw = NULL; > + ltt_wildcard_create(chan, event_param, &sw ); > + goto filter; > + } > + > if (chan->used_event_id == -1U) { > ret = -ENOMEM; > goto full; > @@ -607,6 +635,7 @@ cache_error: > no_loglevel_match: > exist: > full: > +filter: > return ret; > } > > -- > 1.7.10.4 > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From michel.dagenais at polymtl.ca Wed Jun 27 16:24:59 2012 From: michel.dagenais at polymtl.ca (Michel Dagenais) Date: Wed, 27 Jun 2012 16:24:59 -0400 (EDT) Subject: [lttng-dev] Positions to work on LTTng In-Reply-To: <1171102504.7948111340828605975.JavaMail.root@srv11.zimbra.polymtl.ca> Message-ID: <1818367622.7948201340828699100.JavaMail.root@srv11.zimbra.polymtl.ca> Hi, we are looking for a research associate and a postdoctoral fellow to work on developing further LTTng for "Integrated tracing, profiling and debugging for tuning large heterogeneous clusters". These positions require interacting with the team and are thus based in Montreal. If you could be interested, please contact me! -------------- next part -------------- An HTML attachment was scrubbed... URL: From Fredrik_Oestman at mentor.com Thu Jun 28 04:12:47 2012 From: Fredrik_Oestman at mentor.com (Oestman, Fredrik) Date: Thu, 28 Jun 2012 08:12:47 +0000 Subject: [lttng-dev] Support for large files Message-ID: <524C960C5DFC794E82BE548D825F05CF28348826@EU-MBX-01.mgc.mentorg.com> Hi, We've run into a problem when attempting to produce large traces on a 32-bit machine. This is the console output: Session met-2012-06-15_16-09-25 created. Traces will be written in /home/nasir/lttng-traces/met-2012-06-15_16-09-25-20120615-160925 UST channel met_tools enabled for session met-2012-06-15_16-09-25 UST event met_func:* created in channel met_tools UST event met_call:* created in channel met_tools Tracing started for session met-2012-06-15_16-09-25 Ackermann: Jun 15 2012 16:01:06 Error: Error writing to tracefile Tracing stopped for session met-2012-06-15_16-09-25 Session met-2012-06-15_16-09-25 destroyed This is the trace directory (from another, identical run): -rwxrwxrwx 1 nasir nasir 4096 2012-06-18 18:38 metadata* -rwxrwxrwx 1 nasir nasir 0 2012-06-18 18:38 met_tools_0* -rwxrwxrwx 1 nasir nasir 0 2012-06-18 18:38 met_tools_1* -rwxrwxrwx 1 nasir nasir 2147483647 2012-06-18 18:40 met_tools_2* -rwxrwxrwx 1 nasir nasir 0 2012-06-18 18:38 met_tools_3* That the application runs on one core only is expected. The file size is 2 GiB - 1, which is the limit for 32-bit file offset variables. On 64-bit machines, the problem doesn't occur. Is this problem known? We are using lttng-ust 2.0.1 and userspace-rcu 0.6.7. Cheers, Fredrik ?stman From Paul_Woegerer at mentor.com Thu Jun 28 05:26:45 2012 From: Paul_Woegerer at mentor.com (Woegerer, Paul) Date: Thu, 28 Jun 2012 11:26:45 +0200 Subject: [lttng-dev] [PATCH] tracepoint event exclusion In-Reply-To: <20120627183308.GA19735@Krystal> References: <4FEB13CE.1010206@mentor.com> <20120627183308.GA19735@Krystal> Message-ID: <4FEC2355.2040104@mentor.com> On 06/27/2012 08:33 PM, Mathieu Desnoyers wrote: > > hrm, although possibly interesting, I don't think this semantic follows > the lttng UI event enabling semantic. There are a few use-cases to > cover: > > 1) enable tracepoints, start tracing, run app. > > 2) enable some tracepoints, start tracing, run app, enable more > tracepoints while app is running. > > 3) start tracing, run app, enable all tracepoints while app is running. > > AFAIU, your patch and proposed semantic covers use-case #1, but breaks > the other 2. Basically, we have to apply a logical "or" between each > enable-event command (rather than an "and" as your proposed semantic > suggests), because they can be applied independently. You are right, I only had use-case #1 in mind when I wrote this patch. I intentionally kept my modifications small not to break any of the the existing functionality if "!" is not used. Let me provide a more elaborate description of what this patch is doing: Original behavior: match_wildcard(): Check if a wildcard_entry matches the given lttng_event_desc name. If a mach is found and the loglevel also matches return the wildcard_entry. Otherwise return 0. ltt_event_create(): If an event with the given name does not already exist create one, pass a pointer to it (via _event) and return 0. If an event with the given name already exists return -EEXIST. Behavior after patch: match_wildcard(): First check if there is a wildcard_entry that starts with "!" that matches the given lttng_event_desc name. If a filter matches return 0 (no match). If not proceed as before. ltt_event_create(): If an event should be created that starts with "!" do not create it but instead add a new wildcard_entry (if not already existing in the wildcard_list). Otherwise as before. Note that the change of ltt_event_create() was necessary to make sure that also things like: lttng enable-event -u -c met_tools --tracepoint '!met_func:enter' work (i.e. filters that do not have a '*' at the end). > > We could however try to come up with an event attribute that gets > applied on a specific event, e.g.: > > lttng enable-event -u -a --exclude 'met_func:*' --exclude 'met_call:*' I would prefer your "--exclude" over my solution but I doubt that I could implement it properly. Are you planning to add "--exclude" yourself or could you provide me with detailed instructions how to implement it (which functions and structures need to be adapted). -- Thanks, Paul -- Paul Woegerer | SW Development Engineer Mentor Embedded(tm) | Prinz Eugen Stra?e 72/2/4, Vienna, 1040 Austria P 43.1.535991320 Nucleus? | Linux? | Android(tm) | Services | UI | Multi-OS Android is a trademark of Google Inc. Use of this trademark is subject to Google Permissions. Linux is the registered trademark of Linus Torvalds in the U.S. and other countries. From jbernard at debian.org Thu Jun 28 21:58:15 2012 From: jbernard at debian.org (Jon Bernard) Date: Thu, 28 Jun 2012 21:58:15 -0400 Subject: [lttng-dev] Query about LTTng 2.0 Debian packages In-Reply-To: <20120621191452.GA18195@quintessa> References: <20120621012622.GA1387@Krystal> <20120621191452.GA18195@quintessa> Message-ID: <20120629015815.GA22800@quintessa> * Jon Bernard wrote: > * Mathieu Desnoyers wrote: > > Hi Jon, > > > > I wanted to get in touch with you regarding LTTng 2.0 Debian packages. > > Is there anything we can do to help you getting those packages into > > Debian ? They have been out for a while now, and even Ubuntu picked them > > up. I understand that you are probably busy, hence my offer for help. > > I've just uploaded another package I've been working on and LTTng 2.0 packages > are next on my list. Give me a few days to get things sorted, I'll post an > update next week with my status. Is the 2.0 series indented to replace the existing UST, or should they be able to exist side-by-side? I'm inclined to think the new packages should replace the old ones, and thus prevent a user from having both installed at the same time, but wanted to check first. URCU is updated and I'm looking at UST/CTL now. I'll file ITPs for babeltrace and modules in the morning. -- Jon From yannick.brosseau at gmail.com Thu Jun 28 22:51:31 2012 From: yannick.brosseau at gmail.com (Brosseau, Yannick) Date: Thu, 28 Jun 2012 22:51:31 -0400 Subject: [lttng-dev] Query about LTTng 2.0 Debian packages In-Reply-To: <20120629015815.GA22800@quintessa> References: <20120621012622.GA1387@Krystal> <20120621191452.GA18195@quintessa> <20120629015815.GA22800@quintessa> Message-ID: On Thu, Jun 28, 2012 at 9:58 PM, Jon Bernard wrote: > * Jon Bernard wrote: >> * Mathieu Desnoyers wrote: >> > Hi Jon, >> > >> > I wanted to get in touch with you regarding LTTng 2.0 Debian packages. >> > Is there anything we can do to help you getting those packages into >> > Debian ? They have been out for a while now, and even Ubuntu picked them >> > up. I understand that you are probably busy, hence my offer for help. >> >> I've just uploaded another package I've been working on and LTTng 2.0 packages >> are next on my list. Give me a few days to get things sorted, I'll post an >> update next week with my status. > > Is the 2.0 series indented to replace the existing UST, or should they be able > to exist side-by-side? ?I'm inclined to think the new packages should replace > the old ones, and thus prevent a user from having both installed at the same > time, but wanted to check first. Yes, the 2.0 should replace the old 0.x ones. The new lttng-ust replace the old ust. Also you can replace the old lttctl with the new lttng-tools. > URCU is updated and I'm looking at UST/CTL now. I'll file ITPs for babeltrace > and modules in the morning. > > -- > Jon > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Yannick Brosseau yannickbrosseau.com From Paul_Woegerer at mentor.com Fri Jun 29 04:28:08 2012 From: Paul_Woegerer at mentor.com (Woegerer, Paul) Date: Fri, 29 Jun 2012 10:28:08 +0200 Subject: [lttng-dev] notrace missing in lttng-ust Message-ID: <4FED6718.1000205@mentor.com> Hi Mathieu, libust 0.x uses notrace (__attribute__((no_instrument_function)) for all functions that are involved in event emitting. For example: ./libust/marker.c:122:notrace void __ust_marker_empty_function(const struct ust_marker *mdata, ./libust/marker.c:137:notrace void ust_marker_probe_cb(const struct ust_marker *mdata, ./libust/marker.c:197:static notrace void ust_marker_probe_cb_noarg(const struct ust_marker *mdata, I just realized that this is no more the case for lttng-ust. Was it a conscious decision not to use notrace for LTTng 2.0 lttng-ust, or is it just missing ? I would like to have notrace also in lttng-ust, at least for the _DECLARE_TRACEPOINT macro in tracepoint.h. Like this for example: diff --git a/include/lttng/tracepoint.h b/include/lttng/tracepoint.h index 5bab476..71665dc 100644 --- a/include/lttng/tracepoint.h +++ b/include/lttng/tracepoint.h @@ -137,6 +137,7 @@ extern "C" { #define _DECLARE_TRACEPOINT(_provider, _name, ...) \ extern struct tracepoint __tracepoint_##_provider##___##_name; \ +static inline void __tracepoint_cb_##_provider##___##_name(_TP_ARGS_PROTO(__VA_ARGS__)) __attribute__ ((no_instrument_function)); \ static inline void __tracepoint_cb_##_provider##___##_name(_TP_ARGS_PROTO(__VA_ARGS__)) \ { \ struct tracepoint_probe *__tp_probe; \ @@ -158,12 +159,16 @@ end: \ tp_rcu_read_unlock_bp(); \ } \ static inline void __tracepoint_register_##_provider##___##_name(char *name, \ + void (*func)(void), void *data) __attribute__ ((no_instrument_function)); \ +static inline void __tracepoint_register_##_provider##___##_name(char *name, \ void (*func)(void), void *data) \ { \ __tracepoint_probe_register(name, func, data, \ __tracepoint_##_provider##___##_name.signature); \ } \ static inline void __tracepoint_unregister_##_provider##___##_name(char *name, \ + void (*func)(void), void *data) __attribute__ ((no_instrument_function)); \ +static inline void __tracepoint_unregister_##_provider##___##_name(char *name, \ void (*func)(void), void *data) \ { \ __tracepoint_probe_unregister(name, func, data); \ Greetings, Paul -- Paul Woegerer | SW Development Engineer Mentor Embedded(tm) | Prinz Eugen Stra?e 72/2/4, Vienna, 1040 Austria P 43.1.535991320 Nucleus? | Linux? | Android(tm) | Services | UI | Multi-OS Android is a trademark of Google Inc. Use of this trademark is subject to Google Permissions. Linux is the registered trademark of Linus Torvalds in the U.S. and other countries. From mathieu.desnoyers at efficios.com Sat Jun 30 14:13:13 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Sat, 30 Jun 2012 14:13:13 -0400 Subject: [lttng-dev] Support for large files In-Reply-To: <524C960C5DFC794E82BE548D825F05CF28348826@EU-MBX-01.mgc.mentorg.com> References: <524C960C5DFC794E82BE548D825F05CF28348826@EU-MBX-01.mgc.mentorg.com> Message-ID: <20120630181313.GA30747@Krystal> * Oestman, Fredrik (Fredrik_Oestman at mentor.com) wrote: > Hi, > > We've run into a problem when attempting to produce large traces on a 32-bit machine. > > This is the console output: > > Session met-2012-06-15_16-09-25 created. > Traces will be written in /home/nasir/lttng-traces/met-2012-06-15_16-09-25-20120615-160925 > UST channel met_tools enabled for session met-2012-06-15_16-09-25 > UST event met_func:* created in channel met_tools > UST event met_call:* created in channel met_tools > Tracing started for session met-2012-06-15_16-09-25 > Ackermann: Jun 15 2012 16:01:06 > Error: Error writing to tracefile > Tracing stopped for session met-2012-06-15_16-09-25 > Session met-2012-06-15_16-09-25 destroyed > > This is the trace directory (from another, identical run): > > -rwxrwxrwx 1 nasir nasir 4096 2012-06-18 18:38 metadata* > -rwxrwxrwx 1 nasir nasir 0 2012-06-18 18:38 met_tools_0* > -rwxrwxrwx 1 nasir nasir 0 2012-06-18 18:38 met_tools_1* > -rwxrwxrwx 1 nasir nasir 2147483647 2012-06-18 18:40 met_tools_2* > -rwxrwxrwx 1 nasir nasir 0 2012-06-18 18:38 met_tools_3* > > That the application runs on one core only is expected. The file size is 2 GiB - 1, which > is the limit for 32-bit file offset variables. > > On 64-bit machines, the problem doesn't occur. > > Is this problem known? Good catch. I guess we did not test enough on 32-bit. It's fixed on the lttng-tools master branch by commit: commit c72b7d965eb65a9f49a5615cac731cec3082aa7f Author: Mathieu Desnoyers Date: Fri Jun 29 12:40:30 2012 +0200 Fix: support large files on 32-bit systems Signed-off-by: Mathieu Desnoyers it should make its way into stable-2.0 branch soon. By the way, babeltrace has the same issue, but we need to switch from fts.h back to ftw.h to fix that, since fts.h does not support LFS. Thanks, Mathieu > > We are using lttng-ust 2.0.1 and userspace-rcu 0.6.7. > > > Cheers, > > Fredrik ?stman > > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Sat Jun 30 14:16:00 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Sat, 30 Jun 2012 14:16:00 -0400 Subject: [lttng-dev] notrace missing in lttng-ust In-Reply-To: <4FED6718.1000205@mentor.com> References: <4FED6718.1000205@mentor.com> Message-ID: <20120630181600.GB30747@Krystal> * Woegerer, Paul (Paul_Woegerer at mentor.com) wrote: > Hi Mathieu, > > libust 0.x uses notrace (__attribute__((no_instrument_function)) for all > functions that are involved in event emitting. > > For example: > ./libust/marker.c:122:notrace void __ust_marker_empty_function(const > struct ust_marker *mdata, > ./libust/marker.c:137:notrace void ust_marker_probe_cb(const struct > ust_marker *mdata, > ./libust/marker.c:197:static notrace void > ust_marker_probe_cb_noarg(const struct ust_marker *mdata, > > I just realized that this is no more the case for lttng-ust. > Was it a conscious decision not to use notrace for LTTng 2.0 lttng-ust, > or is it just missing ? just missing. only needed for -finstrument-functions. > > I would like to have notrace also in lttng-ust, at least for the > _DECLARE_TRACEPOINT macro in tracepoint.h. I guess the probes generated by ust-tracepoint-event.h would need that too. A patch that adds those noinline everywhere where it's needed (hint: all functions called on the tracing patch within public headers of lttng-ust 2.0 would be a good start). Please test the patch by tracing a program automatically instrumented by -finstrument-functions and see if things blow up. Thanks, Mathieu > Like this for example: > > diff --git a/include/lttng/tracepoint.h b/include/lttng/tracepoint.h > index 5bab476..71665dc 100644 > --- a/include/lttng/tracepoint.h > +++ b/include/lttng/tracepoint.h > @@ -137,6 +137,7 @@ extern "C" { > > #define _DECLARE_TRACEPOINT(_provider, _name, ...) \ > extern struct tracepoint __tracepoint_##_provider##___##_name; > \ > +static inline void > __tracepoint_cb_##_provider##___##_name(_TP_ARGS_PROTO(__VA_ARGS__)) > __attribute__ ((no_instrument_function)); \ > static inline void > __tracepoint_cb_##_provider##___##_name(_TP_ARGS_PROTO(__VA_ARGS__)) \ > { \ > struct tracepoint_probe *__tp_probe; \ > @@ -158,12 +159,16 @@ end: \ > tp_rcu_read_unlock_bp(); \ > } \ > static inline void __tracepoint_register_##_provider##___##_name(char > *name, \ > + void (*func)(void), void *data) __attribute__ > ((no_instrument_function)); \ > +static inline void __tracepoint_register_##_provider##___##_name(char > *name, \ > void (*func)(void), void *data) \ > { \ > __tracepoint_probe_register(name, func, data, \ > __tracepoint_##_provider##___##_name.signature); \ > } \ > static inline void __tracepoint_unregister_##_provider##___##_name(char > *name, \ > + void (*func)(void), void *data) __attribute__ > ((no_instrument_function)); \ > +static inline void __tracepoint_unregister_##_provider##___##_name(char > *name, \ > void (*func)(void), void *data) \ > { \ > __tracepoint_probe_unregister(name, func, data); \ > > Greetings, > Paul > > -- > Paul Woegerer | SW Development Engineer > Mentor Embedded(tm) | Prinz Eugen Stra?e 72/2/4, Vienna, 1040 Austria > P 43.1.535991320 > Nucleus? | Linux? | Android(tm) | Services | UI | Multi-OS > > Android is a trademark of Google Inc. Use of this trademark is subject to Google Permissions. > Linux is the registered trademark of Linus Torvalds in the U.S. and other countries. > > > _______________________________________________ > lttng-dev mailing list > lttng-dev at lists.lttng.org > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From mathieu.desnoyers at efficios.com Sat Jun 30 14:53:35 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Sat, 30 Jun 2012 14:53:35 -0400 Subject: [lttng-dev] [babeltrace] FTS does not support large files on 32-bit Message-ID: <20120630185335.GA31235@Krystal> Hi Yannick, The babeltrace commit: commit 6cba487f031260536d6a77acde888c8b1a876fcf Author: Mathieu Desnoyers Date: Fri Feb 10 12:01:01 2012 -0500 babeltrace lib cleanup, folded with open/remove trace functions Folded patch from Yannick Brosseau , along with various updates and cleanups, related to babeltrace lib. Original changelog from Yannick Brosseau: Move the trace_collection into its own file. Port the converter to uses the new functions Signed-off-by: Mathieu Desnoyers Introduces fts(3) as a replacement for ftw(3). However, it seems that FTS does not support large files (LFS), which I need to turn on in babeltrace. fts.h shows why: /* The fts interface is incompatible with the LFS interface which transparently uses the 64-bit file access functions. */ #ifdef __USE_FILE_OFFSET64 # error " cannot be used with -D_FILE_OFFSET_BITS==64" #endif We should therefore go back to FTW to support large files on 32-bit. Is there any reason why we moved from ftw to fts ? Thanks, Mathieu -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com