[lttng-dev] LTTNG on Android ICS

[Pierre-Luc St-Charles]

Good question: I don't suppose it's fully working right now, as we've
barely touched/tested it yet (we worked mostly locally or via Eclipse). It
should be amongst the next improvement batch though; could you provide the
commands you used to trigger that exact crash? I'll try to replicate it
later this week.


On Tue, May 28, 2013 at 8:18 AM, Ionut D. <ionut5001 at gmail.com> wrote:

> lttng-relayd is functional? Because I want to start it and fails with:
>
> DEBUG3 [3661/3661]: URI string: tcp://0.0.0.0:5342 (in uri_parse() at
> uri.c:250)
> DEBUG3 [3661/3661]: URI dtype: 1, proto: 1, host: 0.0.0.0, subdir: , ctrl:
> 5342, data: 0 (in uri_parse() at uri.c:464)
> DEBUG3 [3661/3661]: URI string: tcp://0.0.0.0:5343 (in uri_parse() at
> uri.c:250)
> DEBUG3 [3661/3661]: URI dtype: 1, proto: 1, host: 0.0.0.0, subdir: , ctrl:
> 5343, data: 0 (in uri_parse() at uri.c:464)
> DEBUG1 [3661/3661]: Signal handler set for SIGTERM, SIGPIPE and SIGINT (in
> set_signal_handler() at main.c:346)
> DEBUG1 [3661/3661]: epoll set max size is 76382 (in
> compat_epoll_set_max_size() at compat-epoll.c:270)
> DEBUG1 [3661/3662]: [thread] Relay dispatcher started (in
> relay_thread_dispatcher() at main.c:646)
> DEBUG1 [3661/3662]: Futex n to 1 prepare done (in futex_nto1_prepare() at
> futex.c:75)
> DEBUG1 [3661/3662]: Woken up but nothing in the relay command queue (in
> relay_thread_dispatcher() at main.c:656)
> DEBUG1 [3661/3664]: [thread] Relay listener started (in
> relay_thread_listener() at main.c:484)
> DEBUG1 [3661/3663]: [thread] Relay worker started (in
> relay_thread_worker() at main.c:1772)
> DEBUG1 [3661/3664]: Listening on sock 7 (in relay_init_sock() at
> main.c:430)
> assertion "URCU_TLS(rcu_reader).need_mb == 0" failed: file "urcu.c", line
> 435, function "rcu_register_thread_mb"
> DEBUG1 [3661/3664]: Listening on sock 9 (in relay_init_sock() at
> main.c:430)
> DEBUG1 [3661/3664]: Listener accepting connections (in
> relay_thread_listener() at main.c:517)
> Segmentation fault
>
>
>
> On Fri, May 24, 2013 at 11:36 PM, Pierre-Luc St-Charles <
> pierre-luc.st-charles at polymtl.ca> wrote:
>
>> Good news: you (most likely) won't need to recompile the whole platform
>> to fix the problem!
>>
>> I think we bumped in a similar bitter-sweet-awful error a while back:
>> after some hefty searches, we found it might have something to do with the
>> Android build system globally 'fortifying' the sources against unsafe
>> functions when compiling some of the most recent versions of the platform.
>> I believe adding -D_FORTIFY_SOURCE=0 to the CPPFLAGS when building the
>> tools solved it for us.
>>
>> -PL
>>
>>
>> On Fri, May 24, 2013 at 3:04 PM, Francis Giraldeau <
>> francis.giraldeau at gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I got some problem compiling required libraries for lttng-tools. I did a
>>> full android build within repo:
>>>
>>> $ source build/envsetup.sh
>>> $ lunch # with full_maguro-eng
>>> $ m
>>>
>>> Build is fine, but when trying to run configure in lttng-tools, it can't
>>> find liboprofile_popt. Here is the error message in config.log:
>>>
>>> /home/francis/android/android-ndk-r8e/toolchains/arm-linux-androideabi-4.6/prebuilt/linux-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld:
>>> /home/francis/android/repo/out/target/product/maguro/obj/STATIC_LIBRARIES/liboprofile_popt_intermediates/liboprofile_popt.a(popthelp.o):
>>> in function singleOptionHelp.isra.4:bionic/libc/include/stdio.h:483: error:
>>> undefined reference to '__sprintf_chk'
>>>
>>> It seems that only the static library of liboprofile_popt is compiled.
>>>
>>> The symbol is defined in libc.a and libc.so:
>>>
>>> $ nm
>>> ./out/target/product/maguro/obj/SHARED_LIBRARIES/libc_intermediates/LINKED/libc.so
>>> | grep -E "__strlen_chk|__sprintf_chk"
>>> 0001395d T __sprintf_chk
>>> 00012df1 T __strlen_chk
>>>
>>> I tried to compile the configtest.c file by hand and it worked, so it
>>> really seems to be some kind of environment variable that make it fail.
>>>
>>> For the people who managed to do the configure, what arguments should be
>>> supplied to configure? Or maybe it's my android build that is incomplete?
>>> Should I build sub-directories in addition to the build in the root android
>>> repo sources?
>>>
>>> Cheers!
>>>
>>> Francis
>>>
>>>
>>> 2013/5/24 Pierre-Luc St-Charles <pierre-luc.st-charles at polymtl.ca>
>>>
>>>> The required kernel configuration options for the modules are listed in
>>>> the lttng-module README file, located in its project root directory. Some
>>>> of the 'optional' options also listed there are simply not supported by
>>>> 3.0.x ARM kernels (such as HAVE_SYSCALL_TRACEPOINTS), so you might be
>>>> missing some modules compared to what you would get on a (x86 arch w/
>>>> recent kernel) platform.
>>>>
>>>> You can find more information on syscall tracing on ARM here:
>>>> http://comments.gmane.org/gmane.linux.kernel.tracing.lttng.devel/5086
>>>>
>>>>
>>>> On Fri, May 24, 2013 at 6:44 AM, Ionut D. <ionut5001 at gmail.com> wrote:
>>>>
>>>>> I managed to fix it. It was not finding `lttng-consumerd`, I fixed it
>>>>> by configuring the package with the right path. But, about the kernel
>>>>> modules the question is still pertinent. What options should I check
>>>>> in the kernel configuration? I saw that in my case not all the lttng kernel
>>>>> modules have been compiled.
>>>>>
>>>>>
>>>>> On Fri, May 24, 2013 at 11:07 AM, Ionut D. <ionut5001 at gmail.com>wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> Thank you for the slides. I manged to build the kernel modules and I
>>>>>> insmod them manually. On the device I copied in /data/lttng following
>>>>>> binaries `lttng`, `lttng-relayd` and `lttng-sessiond`. If I execute
>>>>>> `./lttng list -k` it shows me a list of available kernel events. I create a
>>>>>> session called mysession with `./lttng create mysession` and I enable
>>>>>> tracing on all the event with `./lttng enable-event -a -k`. But here starts
>>>>>> my problem.
>>>>>>
>>>>>> The output of the last command is the following:
>>>>>>
>>>>>> root at android:/data/lttng # ./lttng enable-event -a -k
>>>>>> PERROR [3681/3681]: kernel start consumer exec: (null) (in
>>>>>> spawn_consumerd() at main.c:2007)
>>>>>> Error: Condition timed out. The consumer thread was never ready.
>>>>>> Killing it
>>>>>> Warning: Using pthread_kill as pthread_cancel is missing on the
>>>>>> current platform
>>>>>> PERROR [3608/3658]: write poll pipe: (null) (in notify_thread_pipe()
>>>>>> at utils.c:43)
>>>>>> Error: Events: Fatal error of the session daemon (channel channel0,
>>>>>> session mysession)
>>>>>>
>>>>>> Do you have any idea about what could be wrong? What options should I
>>>>>> check in the kernel configuration? (I checked all that is under Tracing)
>>>>>>
>>>>>> --Ionut
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, May 23, 2013 at 6:09 PM, Charles Brière <
>>>>>> charles.briere at polymtl.ca> wrote:
>>>>>>
>>>>>>> Module location doesn't really matter, we used to push them to
>>>>>>> /data/... or /sdcard/... .
>>>>>>>
>>>>>>>  Modprobe doesn't exists on Android so you'll need to use insmod. We
>>>>>>> used this script<https://docs.google.com/file/d/0B56W2viIJ4oqOU1qTHJZY0V2MzQ/edit>in order to compile, push to device and load modules.
>>>>>>>
>>>>>>> We also made a presentation you might be interested in that can be
>>>>>>> viewed on
>>>>>>> https://docs.google.com/presentation/d/14lte2AchEAyxMk3WSQ53SdbBm2tfmABBFQojuUhxg7Y/pub?start=false&loop=false&delayms=3000
>>>>>>>
>>>>>>>
>>>>>>> On Thu, May 23, 2013 at 10:40 AM, Ionut D. <ionut5001 at gmail.com>wrote:
>>>>>>>
>>>>>>>> About the kernel modules. Where should they be located on the
>>>>>>>> target? Should I have modprobe or insmod it is enough?
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, May 23, 2013 at 2:19 PM, Ionut D. <ionut5001 at gmail.com>wrote:
>>>>>>>>
>>>>>>>>> Sorry for my late response. At the end after different some adds
>>>>>>>>> (defines in libc) I managed to compile them for ICS with NDK r7c. I was a
>>>>>>>>> problem of environment.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, May 21, 2013 at 9:26 PM, Pierre-Luc St-Charles <
>>>>>>>>> pierre-luc.st-charles at polymtl.ca> wrote:
>>>>>>>>>
>>>>>>>>>> Well, I cannot get my hands on a R7 NDK this very moment, but I'm
>>>>>>>>>> surprised there are no grep hits in the ./platforms/android-XX/arch-arm
>>>>>>>>>> directory; can you verify that the NDK you have does include an ARM
>>>>>>>>>> architecture sysroot? That's the the one you should be compiling with, and
>>>>>>>>>> I believe you'll find the asm/unistd.h header there (as there is no 32/64
>>>>>>>>>> bit difference on Android+ARM yet).
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, May 21, 2013 at 11:29 AM, Ionut D. <ionut5001 at gmail.com>wrote:
>>>>>>>>>>
>>>>>>>>>>> The problem is that I am trying to build it on ICS with NDK r7
>>>>>>>>>>> and I tried to build it in the both ways with the ndk and also by creating
>>>>>>>>>>> Android.mk files and it looks like is not working. I searched with grep
>>>>>>>>>>> firstly through NDK and second through bionic and these are the results:
>>>>>>>>>>>
>>>>>>>>>>> NDK:
>>>>>>>>>>> #grep -r __NR_splice .
>>>>>>>>>>> ./platforms/android-9/arch-x86/usr/include/asm/unistd_32.h:#define
>>>>>>>>>>> __NR_splice 313
>>>>>>>>>>> ./platforms/android-14/arch-x86/usr/include/asm/unistd_32.h:#define
>>>>>>>>>>> __NR_splice 313
>>>>>>>>>>>
>>>>>>>>>>> bionic:
>>>>>>>>>>> grep -r __NR_splice bionic
>>>>>>>>>>> bionic/libc/kernel/arch-sh/asm/unistd_32.h:#define __NR_splice
>>>>>>>>>>> 313
>>>>>>>>>>> bionic/libc/kernel/arch-sh/asm/unistd_64.h:#define __NR_splice
>>>>>>>>>>> 341
>>>>>>>>>>> bionic/libc/kernel/arch-x86/asm/unistd_32.h:#define __NR_splice
>>>>>>>>>>> 313
>>>>>>>>>>>
>>>>>>>>>>> Looking through results I see that '__NR_splice' it not in
>>>>>>>>>>> unistd.h and I know this is important.
>>>>>>>>>>>
>>>>>>>>>>> Ionut
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, May 21, 2013 at 5:15 PM, Pierre-Luc St-Charles <
>>>>>>>>>>> pierre-luc.st-charles at polymtl.ca> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> The first error refers to __NR_splice, which should be declared
>>>>>>>>>>>> once asm/unistd.h is included; in this case, it is included right above the
>>>>>>>>>>>> problematic line (src/common/compat/fcntl.h @ L35). Could you verify that
>>>>>>>>>>>> __NR_splice is defined somewhere in that file? If not, it might be a little
>>>>>>>>>>>> more problematic.
>>>>>>>>>>>>
>>>>>>>>>>>> As for the second error, I believe it is also caused by a
>>>>>>>>>>>> missing symbol that should be included in asm/signal.h, which is in turn
>>>>>>>>>>>> usually included via signal.h itself.
>>>>>>>>>>>>
>>>>>>>>>>>> Compiling with the NDK (v8, platform 14) is successful here,
>>>>>>>>>>>> and I can find all these missing symbols in the default system includes
>>>>>>>>>>>> there. Are you sure you specified the target sysroot when compiling?
>>>>>>>>>>>>
>>>>>>>>>>>> -PL
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, May 21, 2013 at 6:26 AM, Ionut D. <ionut5001 at gmail.com>wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I tried to compile lttng-tools and at compile time exits with
>>>>>>>>>>>>> the bellow error. I looked in the sources and it looks like for arm this is
>>>>>>>>>>>>> not implemented. Do you have any idea about how could this be fixed?
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> In file included from
>>>>>>>>>>>>> external/lttng-tools/src/common/compat/compat-fcntl.c:19:
>>>>>>>>>>>>> external/lttng-tools/src/common/compat/../../../src/common/compat/fcntl.h:
>>>>>>>>>>>>> In function 'splice':
>>>>>>>>>>>>> external/lttng-tools/src/common/compat/../../../src/common/compat/fcntl.h:44:
>>>>>>>>>>>>> error: '__NR_splice' undeclared (first use in this function)
>>>>>>>>>>>>> external/lttng-tools/src/common/compat/../../../src/common/compat/fcntl.h:44:
>>>>>>>>>>>>> error: (Each undeclared identifier is reported only once
>>>>>>>>>>>>> external/lttng-tools/src/common/compat/../../../src/common/compat/fcntl.h:44:
>>>>>>>>>>>>> error: for each function it appears in.)
>>>>>>>>>>>>> In file included from
>>>>>>>>>>>>> external/lttng-tools/src/common/compat/compat-fcntl.c:19:
>>>>>>>>>>>>> external/lttng-tools/src/common/compat/../../../src/common/compat/fcntl.h:48:1:
>>>>>>>>>>>>> warning: "POSIX_FADV_DONTNEED" redefined
>>>>>>>>>>>>> In file included from
>>>>>>>>>>>>> external/lttng-tools/src/common/compat/../../../src/common/compat/fcntl.h:47,
>>>>>>>>>>>>>                  from
>>>>>>>>>>>>> external/lttng-tools/src/common/compat/compat-fcntl.c:19:
>>>>>>>>>>>>> bionic/libc/kernel/common/linux/fadvise.h:24:1: warning: this
>>>>>>>>>>>>> is the location of the previous definition
>>>>>>>>>>>>> In file included from bionic/libc/include/sys/select.h:34,
>>>>>>>>>>>>>                  from bionic/libc/include/unistd.h:34,
>>>>>>>>>>>>>                  from
>>>>>>>>>>>>> external/lttng-tools/src/common/compat/compat-fcntl.c:20:
>>>>>>>>>>>>> external/lttng-tools/src/common/compat/signal.h: At top level:
>>>>>>>>>>>>> external/lttng-tools/src/common/compat/signal.h:27: error:
>>>>>>>>>>>>> expected ';', ',' or ')' before '*' token
>>>>>>>>>>>>> external/lttng-tools/src/common/compat/signal.h:28: error:
>>>>>>>>>>>>> expected ';', ',' or ')' before '*' token
>>>>>>>>>>>>> external/lttng-tools/src/common/compat/signal.h:29: error:
>>>>>>>>>>>>> expected ';', ',' or ')' before '*' token
>>>>>>>>>>>>> In file included from bionic/libc/include/unistd.h:34,
>>>>>>>>>>>>>                  from
>>>>>>>>>>>>> external/lttng-tools/src/common/compat/compat-fcntl.c:20:
>>>>>>>>>>>>> bionic/libc/include/sys/select.h:42: error: expected ';', ','
>>>>>>>>>>>>> or ')' before '*' token
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sat, May 18, 2013 at 10:21 AM, Ionut D. <
>>>>>>>>>>>>> ionut5001 at gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Great informations! I will post back the results.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, May 17, 2013 at 4:47 PM, Pierre-Luc St-Charles <
>>>>>>>>>>>>>> pierre-luc.st-charles at polymtl.ca> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Alright, small update for all following a quick
>>>>>>>>>>>>>>> (non-mailing-list) exchange:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We did successfully complete a kernel-tracing port of LTTng
>>>>>>>>>>>>>>> for Android (somewhat old news), but we are still working on bringing our
>>>>>>>>>>>>>>> patches back up to date with the current version of the different projects
>>>>>>>>>>>>>>> (soon-ish still applies here, we're talking days). So far, we dwelled a
>>>>>>>>>>>>>>> little bit in LTTng-UST, but haven't done much changes, so userspace
>>>>>>>>>>>>>>> tracing is still not functional. Device that ran kernel traces successfully
>>>>>>>>>>>>>>> so far were the Nexus S and the Nexus 7 (example Nexus S trace available
>>>>>>>>>>>>>>> here<https://drive.google.com/folderview?id=0B55Ba7lWTLh4U0hGeTQ4bmRYRjQ&usp=sharing>;
>>>>>>>>>>>>>>> still not perfect).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Basically, we cross-compiled everything via the Android NDK
>>>>>>>>>>>>>>> (no Android makefiles involved here), and pushed all required bins to a
>>>>>>>>>>>>>>> device without having to flash it (unless the default kernel config didn't
>>>>>>>>>>>>>>> allow tracing -- then we'd have to flash). This approach is not compatible
>>>>>>>>>>>>>>> with the official Android build system (meaning we wouldn't be able to add
>>>>>>>>>>>>>>> it to Android like this), but we might eventually get working on the damned
>>>>>>>>>>>>>>> Android makefiles (they're extremely painful to write, especially since
>>>>>>>>>>>>>>> LTTng uses Autotools).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The librairies required by LTTng that are not already
>>>>>>>>>>>>>>> included in a regular Android build (uuid/popt) can actually be
>>>>>>>>>>>>>>> found/compiled from the official repo, in its 'external' directory (see
>>>>>>>>>>>>>>> oprofile and e2fsprog -- from memory). That might not be the easiest way to
>>>>>>>>>>>>>>> get them (full repo download is BIG), but compatibility is assured, and you
>>>>>>>>>>>>>>> can skip a full build by simply making those two individually for your
>>>>>>>>>>>>>>> device (if that's all you need).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> You'll also need your device to be rooted to use LTTng at
>>>>>>>>>>>>>>> all; then, most initializations are done via scripts (we'll also provide
>>>>>>>>>>>>>>> those with our patches). We still had some trouble automating everything
>>>>>>>>>>>>>>> via daemons, so we always ran the sessiond locally (with success).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Major problems we encountered so far were missing posix
>>>>>>>>>>>>>>> functions, and sysv shared memory. Basically, Android relies on Bionic for
>>>>>>>>>>>>>>> its system calls and other low-level functions, and since Bionic is pretty
>>>>>>>>>>>>>>> 'lightweight', they trimmed most of the fat out for performance. We had to
>>>>>>>>>>>>>>> make some more-or-less sketchy replacements here and there, but everything
>>>>>>>>>>>>>>> seems to be working decently now.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> So, simple 'how do I put LTTng on Android' workflow would be
>>>>>>>>>>>>>>> this:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>   - Compile modules (using NDK) against device's kernel,
>>>>>>>>>>>>>>> making sure its config is appropriate, ship them over via ADB
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>   - Compile required external libraries (using repo build
>>>>>>>>>>>>>>> system) for the device, ship them over via ADB
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>   - Patch up LTTng-* (soon-ish! I could send you the
>>>>>>>>>>>>>>> outdated patches right now if you're interested though, they might be a
>>>>>>>>>>>>>>> month behind or so on the current trunks)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>   - Cross-compile LTTng-* for the device (using NDK), ship
>>>>>>>>>>>>>>> binaries over via ADB
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Then, on the device, start up the sessiond manually & start
>>>>>>>>>>>>>>> tracing.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I think I went over everything, but if there's a part that's
>>>>>>>>>>>>>>> still cloudy, don't hesitate.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -PL
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On May 17, 2013 1:32 AM, "Ionut D." <ionut5001 at gmail.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hello all,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Recently I started to integrate LTTNG in Android ICS by
>>>>>>>>>>>>>>>> compiling each package, each dependency, by using the Android build system.
>>>>>>>>>>>>>>>> It is a lot of work because I met different problems and to be sincere, I
>>>>>>>>>>>>>>>> do not know if at the end will work and I do not know how much I will need
>>>>>>>>>>>>>>>> to modify the sources in order to work.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> My questions are about this integration. I mention that I
>>>>>>>>>>>>>>>> read a lot of discussions about this on Internet, on this mailing list, but
>>>>>>>>>>>>>>>> the things are still not very clear for me.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> It is ok? what I am trying to do ? to integrate and build
>>>>>>>>>>>>>>>> by using the Android build system.
>>>>>>>>>>>>>>>> Could you give me some guidelines, indications about how I
>>>>>>>>>>>>>>>> should proceed, maybe a tutorial or maybe some android make files/patches
>>>>>>>>>>>>>>>> that would help me to see how the job must be done and if I am doing it
>>>>>>>>>>>>>>>> right ?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Until now I managed to build lttng-modules, liburcu,
>>>>>>>>>>>>>>>> libuuid from util-linux and configure lttng-ust by using the android
>>>>>>>>>>>>>>>> toolchain for arm. I got a lot of errors but until now I managed to
>>>>>>>>>>>>>>>> configure/make these packages.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I mention that I am trying to build lttng 2.x on ICS 4.0.3.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Would be great if you will give me a tutorial about how to
>>>>>>>>>>>>>>>> do it.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Ionut
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>> lttng-dev mailing list
>>>>>>>>>>>>>>>> lttng-dev at lists.lttng.org
>>>>>>>>>>>>>>>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> lttng-dev mailing list
>>>>>>>> lttng-dev at lists.lttng.org
>>>>>>>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> lttng-dev mailing list
>>>> lttng-dev at lists.lttng.org
>>>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>>>
>>>>
>>>
>>> _______________________________________________
>>> lttng-dev mailing list
>>> lttng-dev at lists.lttng.org
>>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>>
>>>
>>
>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev at lists.lttng.org
>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lttng.org/pipermail/lttng-dev/attachments/20130528/3bab769c/attachment-0001.html>