[lttng-dev] LTTNG on Android ICS

Fri May 24 16:36:54 EDT 2013

Good news: you (most likely) won't need to recompile the whole platform to
fix the problem!

I think we bumped in a similar bitter-sweet-awful error a while back: after
some hefty searches, we found it might have something to do with the
Android build system globally 'fortifying' the sources against unsafe
functions when compiling some of the most recent versions of the platform.
I believe adding -D_FORTIFY_SOURCE=0 to the CPPFLAGS when building the
tools solved it for us.

-PL

On Fri, May 24, 2013 at 3:04 PM, Francis Giraldeau <
francis.giraldeau at gmail.com> wrote:

> Hi,
>
> I got some problem compiling required libraries for lttng-tools. I did a
> full android build within repo:
>
> $ source build/envsetup.sh
> $ lunch # with full_maguro-eng
> $ m
>
> Build is fine, but when trying to run configure in lttng-tools, it can't
> find liboprofile_popt. Here is the error message in config.log:
>
> /home/francis/android/android-ndk-r8e/toolchains/arm-linux-androideabi-4.6/prebuilt/linux-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld:
> /home/francis/android/repo/out/target/product/maguro/obj/STATIC_LIBRARIES/liboprofile_popt_intermediates/liboprofile_popt.a(popthelp.o):
> in function singleOptionHelp.isra.4:bionic/libc/include/stdio.h:483: error:
> undefined reference to '__sprintf_chk'
>
> It seems that only the static library of liboprofile_popt is compiled.
>
> The symbol is defined in libc.a and libc.so:
>
> $ nm
> ./out/target/product/maguro/obj/SHARED_LIBRARIES/libc_intermediates/LINKED/libc.so
> | grep -E "__strlen_chk|__sprintf_chk"
> 0001395d T __sprintf_chk
> 00012df1 T __strlen_chk
>
> I tried to compile the configtest.c file by hand and it worked, so it
> really seems to be some kind of environment variable that make it fail.
>
> For the people who managed to do the configure, what arguments should be
> supplied to configure? Or maybe it's my android build that is incomplete?
> Should I build sub-directories in addition to the build in the root android
> repo sources?
>
> Cheers!
>
> Francis
>
>
> 2013/5/24 Pierre-Luc St-Charles <pierre-luc.st-charles at polymtl.ca>
>
>> The required kernel configuration options for the modules are listed in
>> the lttng-module README file, located in its project root directory. Some
>> of the 'optional' options also listed there are simply not supported by
>> 3.0.x ARM kernels (such as HAVE_SYSCALL_TRACEPOINTS), so you might be
>> missing some modules compared to what you would get on a (x86 arch w/
>> recent kernel) platform.
>>
>> You can find more information on syscall tracing on ARM here:
>> http://comments.gmane.org/gmane.linux.kernel.tracing.lttng.devel/5086
>>
>>
>> On Fri, May 24, 2013 at 6:44 AM, Ionut D. <ionut5001 at gmail.com> wrote:
>>
>>> I managed to fix it. It was not finding `lttng-consumerd`, I fixed it by
>>> configuring the package with the right path. But, about the kernel modules
>>> the question is still pertinent. What options should I check in the
>>> kernel configuration? I saw that in my case not all the lttng kernel
>>> modules have been compiled.
>>>
>>>
>>> On Fri, May 24, 2013 at 11:07 AM, Ionut D. <ionut5001 at gmail.com> wrote:
>>>
>>>> Hello,
>>>>
>>>> Thank you for the slides. I manged to build the kernel modules and I
>>>> insmod them manually. On the device I copied in /data/lttng following
>>>> binaries `lttng`, `lttng-relayd` and `lttng-sessiond`. If I execute
>>>> `./lttng list -k` it shows me a list of available kernel events. I create a
>>>> session called mysession with `./lttng create mysession` and I enable
>>>> tracing on all the event with `./lttng enable-event -a -k`. But here starts
>>>> my problem.
>>>>
>>>> The output of the last command is the following:
>>>>
>>>> root at android:/data/lttng # ./lttng enable-event -a -k
>>>> PERROR [3681/3681]: kernel start consumer exec: (null) (in
>>>> spawn_consumerd() at main.c:2007)
>>>> Error: Condition timed out. The consumer thread was never ready.
>>>> Killing it
>>>> Warning: Using pthread_kill as pthread_cancel is missing on the current
>>>> platform
>>>> PERROR [3608/3658]: write poll pipe: (null) (in notify_thread_pipe() at
>>>> utils.c:43)
>>>> Error: Events: Fatal error of the session daemon (channel channel0,
>>>> session mysession)
>>>>
>>>> Do you have any idea about what could be wrong? What options should I
>>>> check in the kernel configuration? (I checked all that is under Tracing)
>>>>
>>>> --Ionut
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, May 23, 2013 at 6:09 PM, Charles Brière <
>>>> charles.briere at polymtl.ca> wrote:
>>>>
>>>>> Module location doesn't really matter, we used to push them to
>>>>> /data/... or /sdcard/... .
>>>>>
>>>>>  Modprobe doesn't exists on Android so you'll need to use insmod. We
>>>>> used this script<https://docs.google.com/file/d/0B56W2viIJ4oqOU1qTHJZY0V2MzQ/edit>in order to compile, push to device and load modules.
>>>>>
>>>>> We also made a presentation you might be interested in that can be
>>>>> viewed on
>>>>> https://docs.google.com/presentation/d/14lte2AchEAyxMk3WSQ53SdbBm2tfmABBFQojuUhxg7Y/pub?start=false&loop=false&delayms=3000
>>>>>
>>>>>
>>>>> On Thu, May 23, 2013 at 10:40 AM, Ionut D. <ionut5001 at gmail.com>wrote:
>>>>>
>>>>>> About the kernel modules. Where should they be located on the target?
>>>>>> Should I have modprobe or insmod it is enough?
>>>>>>
>>>>>>
>>>>>> On Thu, May 23, 2013 at 2:19 PM, Ionut D. <ionut5001 at gmail.com>wrote:
>>>>>>
>>>>>>> Sorry for my late response. At the end after different some adds
>>>>>>> (defines in libc) I managed to compile them for ICS with NDK r7c. I was a
>>>>>>> problem of environment.
>>>>>>>
>>>>>>>
>>>>>>> On Tue, May 21, 2013 at 9:26 PM, Pierre-Luc St-Charles <
>>>>>>> pierre-luc.st-charles at polymtl.ca> wrote:
>>>>>>>
>>>>>>>> Well, I cannot get my hands on a R7 NDK this very moment, but I'm
>>>>>>>> surprised there are no grep hits in the ./platforms/android-XX/arch-arm
>>>>>>>> directory; can you verify that the NDK you have does include an ARM
>>>>>>>> architecture sysroot? That's the the one you should be compiling with, and
>>>>>>>> I believe you'll find the asm/unistd.h header there (as there is no 32/64
>>>>>>>> bit difference on Android+ARM yet).
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, May 21, 2013 at 11:29 AM, Ionut D. <ionut5001 at gmail.com>wrote:
>>>>>>>>
>>>>>>>>> The problem is that I am trying to build it on ICS with NDK r7 and
>>>>>>>>> I tried to build it in the both ways with the ndk and also by creating
>>>>>>>>> Android.mk files and it looks like is not working. I searched with grep
>>>>>>>>> firstly through NDK and second through bionic and these are the results:
>>>>>>>>>
>>>>>>>>> NDK:
>>>>>>>>> #grep -r __NR_splice .
>>>>>>>>> ./platforms/android-9/arch-x86/usr/include/asm/unistd_32.h:#define
>>>>>>>>> __NR_splice 313
>>>>>>>>> ./platforms/android-14/arch-x86/usr/include/asm/unistd_32.h:#define
>>>>>>>>> __NR_splice 313
>>>>>>>>>
>>>>>>>>> bionic:
>>>>>>>>> grep -r __NR_splice bionic
>>>>>>>>> bionic/libc/kernel/arch-sh/asm/unistd_32.h:#define __NR_splice 313
>>>>>>>>> bionic/libc/kernel/arch-sh/asm/unistd_64.h:#define __NR_splice 341
>>>>>>>>> bionic/libc/kernel/arch-x86/asm/unistd_32.h:#define __NR_splice 313
>>>>>>>>>
>>>>>>>>> Looking through results I see that '__NR_splice' it not in
>>>>>>>>> unistd.h and I know this is important.
>>>>>>>>>
>>>>>>>>> Ionut
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, May 21, 2013 at 5:15 PM, Pierre-Luc St-Charles <
>>>>>>>>> pierre-luc.st-charles at polymtl.ca> wrote:
>>>>>>>>>
>>>>>>>>>> The first error refers to __NR_splice, which should be declared
>>>>>>>>>> once asm/unistd.h is included; in this case, it is included right above the
>>>>>>>>>> problematic line (src/common/compat/fcntl.h @ L35). Could you verify that
>>>>>>>>>> __NR_splice is defined somewhere in that file? If not, it might be a little
>>>>>>>>>> more problematic.
>>>>>>>>>>
>>>>>>>>>> As for the second error, I believe it is also caused by a missing
>>>>>>>>>> symbol that should be included in asm/signal.h, which is in turn usually
>>>>>>>>>> included via signal.h itself.
>>>>>>>>>>
>>>>>>>>>> Compiling with the NDK (v8, platform 14) is successful here, and
>>>>>>>>>> I can find all these missing symbols in the default system includes there.
>>>>>>>>>> Are you sure you specified the target sysroot when compiling?
>>>>>>>>>>
>>>>>>>>>> -PL
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, May 21, 2013 at 6:26 AM, Ionut D. <ionut5001 at gmail.com>wrote:
>>>>>>>>>>
>>>>>>>>>>> I tried to compile lttng-tools and at compile time exits with
>>>>>>>>>>> the bellow error. I looked in the sources and it looks like for arm this is
>>>>>>>>>>> not implemented. Do you have any idea about how could this be fixed?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> In file included from
>>>>>>>>>>> external/lttng-tools/src/common/compat/compat-fcntl.c:19:
>>>>>>>>>>> external/lttng-tools/src/common/compat/../../../src/common/compat/fcntl.h:
>>>>>>>>>>> In function 'splice':
>>>>>>>>>>> external/lttng-tools/src/common/compat/../../../src/common/compat/fcntl.h:44:
>>>>>>>>>>> error: '__NR_splice' undeclared (first use in this function)
>>>>>>>>>>> external/lttng-tools/src/common/compat/../../../src/common/compat/fcntl.h:44:
>>>>>>>>>>> error: (Each undeclared identifier is reported only once
>>>>>>>>>>> external/lttng-tools/src/common/compat/../../../src/common/compat/fcntl.h:44:
>>>>>>>>>>> error: for each function it appears in.)
>>>>>>>>>>> In file included from
>>>>>>>>>>> external/lttng-tools/src/common/compat/compat-fcntl.c:19:
>>>>>>>>>>> external/lttng-tools/src/common/compat/../../../src/common/compat/fcntl.h:48:1:
>>>>>>>>>>> warning: "POSIX_FADV_DONTNEED" redefined
>>>>>>>>>>> In file included from
>>>>>>>>>>> external/lttng-tools/src/common/compat/../../../src/common/compat/fcntl.h:47,
>>>>>>>>>>>                  from
>>>>>>>>>>> external/lttng-tools/src/common/compat/compat-fcntl.c:19:
>>>>>>>>>>> bionic/libc/kernel/common/linux/fadvise.h:24:1: warning: this is
>>>>>>>>>>> the location of the previous definition
>>>>>>>>>>> In file included from bionic/libc/include/sys/select.h:34,
>>>>>>>>>>>                  from bionic/libc/include/unistd.h:34,
>>>>>>>>>>>                  from
>>>>>>>>>>> external/lttng-tools/src/common/compat/compat-fcntl.c:20:
>>>>>>>>>>> external/lttng-tools/src/common/compat/signal.h: At top level:
>>>>>>>>>>> external/lttng-tools/src/common/compat/signal.h:27: error:
>>>>>>>>>>> expected ';', ',' or ')' before '*' token
>>>>>>>>>>> external/lttng-tools/src/common/compat/signal.h:28: error:
>>>>>>>>>>> expected ';', ',' or ')' before '*' token
>>>>>>>>>>> external/lttng-tools/src/common/compat/signal.h:29: error:
>>>>>>>>>>> expected ';', ',' or ')' before '*' token
>>>>>>>>>>> In file included from bionic/libc/include/unistd.h:34,
>>>>>>>>>>>                  from
>>>>>>>>>>> external/lttng-tools/src/common/compat/compat-fcntl.c:20:
>>>>>>>>>>> bionic/libc/include/sys/select.h:42: error: expected ';', ',' or
>>>>>>>>>>> ')' before '*' token
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Sat, May 18, 2013 at 10:21 AM, Ionut D. <ionut5001 at gmail.com>wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Great informations! I will post back the results.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, May 17, 2013 at 4:47 PM, Pierre-Luc St-Charles <
>>>>>>>>>>>> pierre-luc.st-charles at polymtl.ca> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Alright, small update for all following a quick
>>>>>>>>>>>>> (non-mailing-list) exchange:
>>>>>>>>>>>>>
>>>>>>>>>>>>> We did successfully complete a kernel-tracing port of LTTng
>>>>>>>>>>>>> for Android (somewhat old news), but we are still working on bringing our
>>>>>>>>>>>>> patches back up to date with the current version of the different projects
>>>>>>>>>>>>> (soon-ish still applies here, we're talking days). So far, we dwelled a
>>>>>>>>>>>>> little bit in LTTng-UST, but haven't done much changes, so userspace
>>>>>>>>>>>>> tracing is still not functional. Device that ran kernel traces successfully
>>>>>>>>>>>>> so far were the Nexus S and the Nexus 7 (example Nexus S trace available
>>>>>>>>>>>>> here<https://drive.google.com/folderview?id=0B55Ba7lWTLh4U0hGeTQ4bmRYRjQ&usp=sharing>;
>>>>>>>>>>>>> still not perfect).
>>>>>>>>>>>>>
>>>>>>>>>>>>> Basically, we cross-compiled everything via the Android NDK
>>>>>>>>>>>>> (no Android makefiles involved here), and pushed all required bins to a
>>>>>>>>>>>>> device without having to flash it (unless the default kernel config didn't
>>>>>>>>>>>>> allow tracing -- then we'd have to flash). This approach is not compatible
>>>>>>>>>>>>> with the official Android build system (meaning we wouldn't be able to add
>>>>>>>>>>>>> it to Android like this), but we might eventually get working on the damned
>>>>>>>>>>>>> Android makefiles (they're extremely painful to write, especially since
>>>>>>>>>>>>> LTTng uses Autotools).
>>>>>>>>>>>>>
>>>>>>>>>>>>> The librairies required by LTTng that are not already included
>>>>>>>>>>>>> in a regular Android build (uuid/popt) can actually be found/compiled from
>>>>>>>>>>>>> the official repo, in its 'external' directory (see oprofile and e2fsprog
>>>>>>>>>>>>> -- from memory). That might not be the easiest way to get them (full repo
>>>>>>>>>>>>> download is BIG), but compatibility is assured, and you can skip a full
>>>>>>>>>>>>> build by simply making those two individually for your device (if that's
>>>>>>>>>>>>> all you need).
>>>>>>>>>>>>>
>>>>>>>>>>>>> You'll also need your device to be rooted to use LTTng at all;
>>>>>>>>>>>>> then, most initializations are done via scripts (we'll also provide those
>>>>>>>>>>>>> with our patches). We still had some trouble automating everything via
>>>>>>>>>>>>> daemons, so we always ran the sessiond locally (with success).
>>>>>>>>>>>>>
>>>>>>>>>>>>> Major problems we encountered so far were missing posix
>>>>>>>>>>>>> functions, and sysv shared memory. Basically, Android relies on Bionic for
>>>>>>>>>>>>> its system calls and other low-level functions, and since Bionic is pretty
>>>>>>>>>>>>> 'lightweight', they trimmed most of the fat out for performance. We had to
>>>>>>>>>>>>> make some more-or-less sketchy replacements here and there, but everything
>>>>>>>>>>>>> seems to be working decently now.
>>>>>>>>>>>>>
>>>>>>>>>>>>> So, simple 'how do I put LTTng on Android' workflow would be
>>>>>>>>>>>>> this:
>>>>>>>>>>>>>
>>>>>>>>>>>>>   - Compile modules (using NDK) against device's kernel,
>>>>>>>>>>>>> making sure its config is appropriate, ship them over via ADB
>>>>>>>>>>>>>
>>>>>>>>>>>>>   - Compile required external libraries (using repo build
>>>>>>>>>>>>> system) for the device, ship them over via ADB
>>>>>>>>>>>>>
>>>>>>>>>>>>>   - Patch up LTTng-* (soon-ish! I could send you the outdated
>>>>>>>>>>>>> patches right now if you're interested though, they might be a month behind
>>>>>>>>>>>>> or so on the current trunks)
>>>>>>>>>>>>>
>>>>>>>>>>>>>   - Cross-compile LTTng-* for the device (using NDK), ship
>>>>>>>>>>>>> binaries over via ADB
>>>>>>>>>>>>>
>>>>>>>>>>>>> Then, on the device, start up the sessiond manually & start
>>>>>>>>>>>>> tracing.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I think I went over everything, but if there's a part that's
>>>>>>>>>>>>> still cloudy, don't hesitate.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> -PL
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On May 17, 2013 1:32 AM, "Ionut D." <ionut5001 at gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hello all,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Recently I started to integrate LTTNG in Android ICS by
>>>>>>>>>>>>>> compiling each package, each dependency, by using the Android build system.
>>>>>>>>>>>>>> It is a lot of work because I met different problems and to be sincere, I
>>>>>>>>>>>>>> do not know if at the end will work and I do not know how much I will need
>>>>>>>>>>>>>> to modify the sources in order to work.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> My questions are about this integration. I mention that I
>>>>>>>>>>>>>> read a lot of discussions about this on Internet, on this mailing list, but
>>>>>>>>>>>>>> the things are still not very clear for me.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> It is ok? what I am trying to do ? to integrate and build by
>>>>>>>>>>>>>> using the Android build system.
>>>>>>>>>>>>>> Could you give me some guidelines, indications about how I
>>>>>>>>>>>>>> should proceed, maybe a tutorial or maybe some android make files/patches
>>>>>>>>>>>>>> that would help me to see how the job must be done and if I am doing it
>>>>>>>>>>>>>> right ?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Until now I managed to build lttng-modules, liburcu, libuuid
>>>>>>>>>>>>>> from util-linux and configure lttng-ust by using the android toolchain for
>>>>>>>>>>>>>> arm. I got a lot of errors but until now I managed to configure/make these
>>>>>>>>>>>>>> packages.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I mention that I am trying to build lttng 2.x on ICS 4.0.3.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Would be great if you will give me a tutorial about how to do
>>>>>>>>>>>>>> it.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ionut
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> lttng-dev mailing list
>>>>>>>>>>>>>> lttng-dev at lists.lttng.org
>>>>>>>>>>>>>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> lttng-dev mailing list
>>>>>> lttng-dev at lists.lttng.org
>>>>>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev at lists.lttng.org
>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>
>>
>
> _______________________________________________
> lttng-dev mailing list
> lttng-dev at lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lttng.org/pipermail/lttng-dev/attachments/20130524/2503eb45/attachment-0001.html>