[lttng-dev] Fwd: compiling LTTng for android
Pierre-Luc St-Charles
pierre-luc.st-charles at polymtl.ca
Tue Mar 12 18:51:20 EDT 2013
Hey Amit,
Here's a small update on my latest findings regarding the broken *splice* /
bad Bionic impl problem:
Although it didn't really cross my mind earlier, we had seen this exact
problem before, but in another project; we had also investigated a bit at
the time, but to no avail. It seems the issue is known by the
Android/Bionic community, but it's simply not on their immediate todo list
(and hasn't been for at least the past two years -- see
https://code.google.com/p/android/issues/detail?id=11330). The solutions we
are now considering are: adding splice support (and whatever else might
also be needed) to Bionic ourselves (saving BILLIONS of broken lives in the
process, IF the changes are accepted), or simply inserting glibc (and
building against it) in our device images.
In the next few days, we'll be studying how much time the first solution
would actually take (as we're not really experts in that domain at all),
but considering the nature/objective of this project, it might be the
wisest decision (as it would not require too much swerving away from the
default images people have on their devices for LTTng to actually work). If
we deem it 'impossible/out of our league', we'll simply try to get a
working tool asap using glibc, which should not be very complicated itself
(many people have actually successfully done it before).
In any case, I'll keep you updated, and I'll also post any big updates on
the mailing list.
-PL
On Wed, Mar 6, 2013 at 8:04 PM, Pierre-Luc St-Charles <
pierre-luc.st-charles at polymtl.ca> wrote:
> Great findings!
>
> I'll also try to look into this new problem by next monday; I'm falling a
> bit behind on my 'investigations', but I'll keep you updated if anything
> comes up.
>
> -PL
> On Mar 6, 2013 6:09 PM, "Amit Balboul" <amit.balboul at shinesec.com> wrote:
>
>> Hi All,
>>
>> Regarding the last mail about tracing system calls on Android using LTTng,
>> The issue of not receiving enabled event to trace from the kernel can be
>> fixed patching src/bin/lttng-sessiond/kernel.c (patch attached)
>> In the function *kernel_list_events,* fscanf-ing the events exported by
>> the kernel using the fp must not use the syntax *%m* as it is obsolete
>> (in Android),
>> thus the fscanf returns 0 and won't read the kernel events (first
>> condition false).
>> Instead I use a pre-allocated buffer for the string and use the *%[^;]*syntax, and omitting the memory free (attached is the patch).
>>
>> The resulting output is (sessiond is run with verbose debug prints):
>>
>> root at android:/data/lttng/bin # *./lttng list -k*
>> DEBUG1 [10239/10290]: Wait for client response (in
>> thread_manage_clients() at main.c:3242)
>> DEBUG1 [10239/10290]: Receiving data from client ... (in
>> thread_manage_clients() at main.c:3287)
>> DEBUG1 [10239/10290]: Nothing recv() from client... continuing (in
>> thread_manage_clients() at main.c:3291)
>> DEBUG1 [10239/10290]: Clean command context structure (in
>> clean_command_ctx() at main.c:482)
>> DEBUG1 [10239/10290]: Accepting client command ... (in
>> thread_manage_clients() at main.c:3200)
>> DEBUG1 [10239/10290]: Wait for client response (in
>> thread_manage_clients() at main.c:3242)
>> DEBUG1 [10239/10290]: Receiving data from client ... (in
>> thread_manage_clients() at main.c:3287)
>> DEBUG1 [10239/10290]: Processing client command 14 (in
>> process_client_msg() at main.c:2227)
>> DEBUG1 [10239/10290]: Kernel list events done (61 events) (in
>> kernel_list_events() at kernel.c:653)
>> DEBUG1 [10239/10290]: Sending response (size: 35640, retcode: Success)
>> (in thread_manage_clients() at main.c:3338)
>> DEBUG1 [10239/10290]: Clean command context structure (in
>> clean_command_ctx() at main.c:482)
>> DEBUG1 [10239/10290]: Accepting client command ... (in
>> thread_manage_clients() at main.c:3200)
>> Kernel events:
>> -------------
>> block_rq_abort (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> block_rq_requeue (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> block_rq_complete (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> block_rq_insert (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> block_rq_issue (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> block_bio_bounce (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> block_bio_complete (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> block_bio_backmerge (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> block_bio_frontmerge (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> block_bio_queue (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> block_getrq (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> block_sleeprq (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> block_plug (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> block_unplug (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> block_split (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> block_bio_remap (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> block_rq_remap (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> irq_handler_entry (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> irq_handler_exit (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> softirq_entry (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> softirq_exit (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> softirq_raise (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> sched_kthread_stop (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> sched_kthread_stop_ret (loglevel: TRACE_EMERG (0)) (type:
>> tracepoint)
>> sched_wakeup (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> sched_wakeup_new (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> sched_switch (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> sched_migrate_task (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> sched_process_free (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> sched_process_exit (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> sched_wait_task (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> sched_process_wait (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> sched_process_fork (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> sched_stat_wait (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> sched_stat_sleep (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> sched_stat_iowait (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> sched_stat_runtime (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> sched_pi_setprio (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> signal_generate (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> signal_deliver (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> signal_overflow_fail (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> signal_lose_info (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> lttng_statedump_start (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> lttng_statedump_end (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> lttng_statedump_process_state (loglevel: TRACE_EMERG (0)) (type:
>> tracepoint)
>> lttng_statedump_file_descriptor (loglevel: TRACE_EMERG (0)) (type:
>> tracepoint)
>> lttng_statedump_vm_map (loglevel: TRACE_EMERG (0)) (type:
>> tracepoint)
>> lttng_statedump_network_interface (loglevel: TRACE_EMERG (0))
>> (type: tracepoint)
>> lttng_statedump_interrupt (loglevel: TRACE_EMERG (0)) (type:
>> tracepoint)
>> timer_init (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> timer_start (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> timer_expire_entry (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> timer_expire_exit (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> timer_cancel (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> hrtimer_init (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> hrtimer_start (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> hrtimer_expire_entry (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> hrtimer_expire_exit (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> hrtimer_cancel (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> itimer_state (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>> itimer_expire (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>>
>> root at android:/data/lttng/bin #
>>
>> Now I'm facing the issue of starting the traces:
>> debugging the sessiond, consumerd and lttng I concluded that the issue is
>> due to splice system call failure.
>> even using a MMAP output type channel, the metadata channel is SPLICE
>> type (hard-coded).
>> Problem persists also when using relayd to pass the traces through the
>> network.
>> Ignoring the splice failure means closing of the metadata channel
>> (maybe??) but need not affect the traces channel (right ?)
>> Attached is a debug log of each thread of the sessiond, right after
>> invoking "*./lttng start*". (with extra informative my prints)
>>
>> Please keep me in touch with any progress in this issue,
>> Thank you,
>> Amit.
>>
>>
>>
>>
>>
>>
>> ---------- Forwarded message ----------
>> From: Amit Balboul <amit.balboul at shinesec.com>
>> Date: Wed, Feb 27, 2013 at 4:37 PM
>> Subject: Re: [lttng-dev] compiling LTTng for android
>> To: PLSTC <b0mb00z.it at gmail.com>
>>
>>
>> Hey Pierre,
>>
>> Some major progress has been achieved, following is the summary:
>>
>> 1. Modules issue:
>> I built (for ARM/Android) and installed latest stable version of busybox
>> on the device.
>> Seeing the code tries to run (hardcoded) /sbin/modprobe, made a soft link
>> to busybox's modeprobe.
>> Also, created /lib/modules/<version+build> directory and pushed the
>> <version+build> directory from /lib/modules on the machine.
>> Now session daemon modprobes the desired modules successfully, (some are
>> not present though as mentioned before) - no need to use insmod.
>>
>> 2. epoll_create1 issue:
>> epoll_create1 patch was OK, I accidentally defined EPOLL_CLOEXEC as
>> 0x02000000 instead of 02000000 (fixed).
>>
>> The running scenario:
>>
>> 1. Run ./lttng-sessiond -d -vvv
>>
>> Then: (commands are in bold, output is in blue for readability)
>>
>> root at android:/data/lttng/install_sysroot/bin # *./lttng create s1*
>>
>> Session s1 created.
>> Traces will be written in /data/lttng-traces/s1-20130227-161145
>> root at android:/data/lttng/install_sysroot/bin # *./lttng enable-channel
>> ch -k*
>> Kernel channel ch enabled for session s1
>> root at android:/data/lttng/install_sysroot/bin # *./lttng enable-event ev
>> -k -a*
>> All kernel events are enabled in channel channel0
>> root at android:/data/lttng/install_sysroot/bin # *./lttng list s1*
>> Tracing session s1: [inactive]
>> Trace path: /data/lttng-traces/s1-20130227-161145
>>
>> === Domain: Kernel ===
>>
>> Channels:
>> -------------
>> - channel0: [enabled]
>>
>> Attributes:
>> overwrite mode: 0
>> subbufers size: 262144
>> number of subbufers: 4
>> switch timer interval: 0
>> read timer interval: 200
>> output: splice()
>>
>> Events:
>> None
>>
>> - ch: [enabled]
>>
>> Attributes:
>> overwrite mode: 0
>> subbufers size: 262144
>> number of subbufers: 4
>> switch timer interval: 0
>> read timer interval: 200
>> output: splice()
>>
>> Events:
>> None
>>
>> root at android:/data/lttng/install_sysroot/bin # *./lttng list -k *
>>
>> Kernel events:
>> -------------
>>
>> root at android:/data/lttng/install_sysroot/bin # *./lttng start*
>> Tracing started for session s1
>> root at android:/data/lttng/install_sysroot/bin # *./lttng stop *
>>
>> Waiting for data availability
>> Tracing stopped for session s1
>> root at android:/data/lttng/install_sysroot/bin #
>>
>> *Now: (this is not surprising because the kernel events list is empty...)
>> *
>> the folder /data/lttng-traces/s1-20130227-161145/kernel contains:
>> total 0
>> -rwxrwxrwx 1 0 0 0 Feb 27 14:18 ch_0
>> -rwxrwxrwx 1 0 0 0 Feb 27 14:18 ch_1
>> -rwxrwxrwx 1 0 0 0 Feb 27 14:18 channel0_0
>> -rwxrwxrwx 1 0 0 0 Feb 27 14:18 channel0_1
>> -rwxrwxrwx 1 0 0 0 Feb 27 14:18 metadata
>>
>> *By the way:*
>> When running the same scenario but creating the channel with *--output
>> mmap*, still no data but the size of the first two files in the traces
>> directory are 4Kb (I didn't bother to investigate them as I assume it is
>> just the header - not the traces payload).
>>
>> Another issue: at this point, listing the sessions gives:
>> root at android:/data/lttng/install_sysroot/bin # *./lttng list s1 *
>>
>> Tracing session s1: [inactive]
>> Trace path: /data/lttng-traces/s1-20130227-161145
>>
>> === Domain: Kernel ===
>>
>> Error: No kernel consumer detected
>> 99|root at android:/data/lttng/install_sysroot/bin # *./lttng list -k *
>>
>> Error: Unable to list kernel events
>> Error: Command error
>> 1|root at android:/data/lttng/install_sysroot/bin #
>>
>> So currently I'm facing these issues:
>> 1. Kernel events list is empty.
>> 2. Listing the session/ kernel events after creating session, channel,
>> event, start, stop - results in "no kernel consumer detected"
>>
>> If you have some insights please share them with me..
>>
>> Thanks
>>
>> Amit.
>>
>>
>> On Tue, Feb 26, 2013 at 7:35 PM, PLSTC <b0mb00z.it at gmail.com> wrote:
>>
>>> The modules you can't find are the same I am also missing I believe; I
>>> think they are only compiled when certain kernel config defines are met
>>> (and those might simply be unavailable under ARM).
>>>
>>> As for what you pointed in your first mail, I believe modprobe isn't
>>> available on ARM, did you mean that some modules were not found when using
>>> insmod?
>>>
>>> For the epoll_create1 problem, what kind of patch did you apply to add
>>> support for the epoll_create1 function?
>>>
>>>
>>> On Tue, Feb 26, 2013 at 11:29 AM, Amit Balboul <
>>> amit.balboul at shinesec.com> wrote:
>>>
>>>> Ok I got :
>>>> lttng-kretprobes.ko
>>>> lttng-kprobes.ko
>>>> , the other two are missing.
>>>>
>>>>
>>>> ---------- Forwarded message ----------
>>>> From: Amit Balboul <amit.balboul at shinesec.com>
>>>> Date: Tue, Feb 26, 2013 at 5:49 PM
>>>> Subject: Re: [lttng-dev] compiling LTTng for android
>>>> To: PLSTC <b0mb00z.it at gmail.com>
>>>>
>>>>
>>>> One more thing:
>>>>
>>>> The list of modules you gave my is not identical to mine:
>>>> a. I pushed the modules from my machine to my folder in the device (not
>>>> to /system/lib) as you directed me, and insmoded them manually.
>>>> b. I pushed the whole tree under /lib/modules/<KERNEL_VER_BUILD>/ which
>>>> includes the modules.* and the sub directories:
>>>>
>>>> /lib/modules/3.0.31-gd5a18e0/extra$ ll
>>>> total 2464
>>>> drwxr-xr-x 4 root root 4096 Feb 26 15:58 ./
>>>> drwxr-xr-x 3 root root 4096 Feb 26 15:58 ../
>>>> drwxr-xr-x 2 root root 4096 Feb 26 15:58 lib/
>>>> -rw-r--r-- 1 root root 167184 Feb 26 15:58
>>>> lttng-ring-buffer-client-discard.ko
>>>> -rw-r--r-- 1 root root 167206 Feb 26 15:58
>>>> lttng-ring-buffer-client-mmap-discard.ko
>>>> -rw-r--r-- 1 root root 178586 Feb 26 15:58
>>>> lttng-ring-buffer-client-mmap-overwrite.ko
>>>> -rw-r--r-- 1 root root 178564 Feb 26 15:58
>>>> lttng-ring-buffer-client-overwrite.ko
>>>> -rw-r--r-- 1 root root 137179 Feb 26 15:58
>>>> lttng-ring-buffer-metadata-client.ko
>>>> -rw-r--r-- 1 root root 137233 Feb 26 15:58
>>>> lttng-ring-buffer-metadata-mmap-client.ko
>>>> -rw-r--r-- 1 root root 213451 Feb 26 15:58 lttng-statedump.ko
>>>> -rw-r--r-- 1 root root 1314235 Feb 26 15:58 lttng-tracer.ko
>>>> drwxr-xr-x 2 root root 4096 Feb 26 15:58 probes/
>>>>
>>>> /lib/modules/3.0.31-gd5a18e0/extra/lib$ ll
>>>> total 732
>>>> drwxr-xr-x 2 root root 4096 Feb 26 15:58 ./
>>>> drwxr-xr-x 4 root root 4096 Feb 26 15:58 ../
>>>> -rw-r--r-- 1 root root 738964 Feb 26 15:58 lttng-lib-ring-buffer.ko
>>>>
>>>> /lib/modules/3.0.31-gd5a18e0/extra/probes$ ll
>>>> total 2872
>>>> drwxr-xr-x 2 root root 4096 Feb 26 15:58 ./
>>>> drwxr-xr-x 4 root root 4096 Feb 26 15:58 ../
>>>> -rw-r--r-- 1 root root 162872 Feb 26 15:58 lttng-probe-asoc.ko
>>>> -rw-r--r-- 1 root root 171049 Feb 26 15:58 lttng-probe-block.ko
>>>> -rw-r--r-- 1 root root 81165 Feb 26 15:58 lttng-probe-compaction.ko
>>>> -rw-r--r-- 1 root root 333215 Feb 26 15:58 lttng-probe-ext4.ko
>>>> -rw-r--r-- 1 root root 80413 Feb 26 15:58 lttng-probe-gpio.ko
>>>> -rw-r--r-- 1 root root 84256 Feb 26 15:58 lttng-probe-irq.ko
>>>> -rw-r--r-- 1 root root 135453 Feb 26 15:58 lttng-probe-jbd2.ko
>>>> -rw-r--r-- 1 root root 105399 Feb 26 15:58 lttng-probe-kmem.ko
>>>> -rw-r--r-- 1 root root 76240 Feb 26 15:58 lttng-probe-lttng.ko
>>>> -rw-r--r-- 1 root root 88059 Feb 26 15:58 lttng-probe-module.ko
>>>> -rw-r--r-- 1 root root 154032 Feb 26 15:58 lttng-probe-napi.ko
>>>> -rw-r--r-- 1 root root 142038 Feb 26 15:58 lttng-probe-net.ko
>>>> -rw-r--r-- 1 root root 93564 Feb 26 15:58 lttng-probe-power.ko
>>>> -rw-r--r-- 1 root root 85530 Feb 26 15:58 lttng-probe-regulator.ko
>>>> -rw-r--r-- 1 root root 138121 Feb 26 15:58 lttng-probe-sched.ko
>>>> -rw-r--r-- 1 root root 152909 Feb 26 15:58 lttng-probe-scsi.ko
>>>> -rw-r--r-- 1 root root 107491 Feb 26 15:58 lttng-probe-signal.ko
>>>> -rw-r--r-- 1 root root 142711 Feb 26 15:58 lttng-probe-skb.ko
>>>> -rw-r--r-- 1 root root 181983 Feb 26 15:58 lttng-probe-statedump.ko
>>>> -rw-r--r-- 1 root root 126731 Feb 26 15:58 lttng-probe-timer.ko
>>>> -rw-r--r-- 1 root root 86591 Feb 26 15:58 lttng-probe-workqueue.ko
>>>> -rw-r--r-- 1 root root 133886 Feb 26 15:58 lttng-probe-writeback.ko
>>>> -rw-r--r-- 1 root root 30863 Feb 26 15:58 lttng-types.ko
>>>>
>>>> Looking at the files shows I miss:
>>>> lttng-kretprobes.ko
>>>> lttng-kprobes.ko
>>>> lttng-ftrace.ko
>>>> lttng-probe-kvm.ko
>>>>
>>>>
>>>> On Tue, Feb 26, 2013 at 3:56 PM, PLSTC <b0mb00z.it at gmail.com> wrote:
>>>>
>>>>> Back on the last email:
>>>>>
>>>>> Here's the insmod list I ended up building; you might not have
>>>>> compiled all those modules, depending on your kernel config and on the
>>>>> architecture used, so some might simply be absent from your output
>>>>> directory.
>>>>>
>>>>> insmod /system/lib/modules/lttng-types.ko
>>>>> insmod /system/lib/modules/lttng-kretprobes.ko
>>>>> insmod /system/lib/modules/lttng-kprobes.ko
>>>>> insmod /system/lib/modules/lttng-ftrace.ko
>>>>> insmod /system/lib/modules/lttng-lib-ring-buffer.ko
>>>>> insmod /system/lib/modules/lttng-statedump.ko
>>>>> insmod /system/lib/modules/lttng-tracer.ko
>>>>> insmod /system/lib/modules/lttng-probe-timer.ko
>>>>> insmod /system/lib/modules/lttng-probe-statedump.ko
>>>>> insmod /system/lib/modules/lttng-probe-signal.ko
>>>>> insmod /system/lib/modules/lttng-probe-sched.ko
>>>>> insmod /system/lib/modules/lttng-probe-lttng.ko
>>>>> insmod /system/lib/modules/lttng-probe-kvm.ko
>>>>> insmod /system/lib/modules/lttng-probe-irq.ko
>>>>> insmod /system/lib/modules/lttng-probe-block.ko
>>>>> insmod /system/lib/modules/lttng-ring-buffer-metadata-mmap-client.ko
>>>>> insmod /system/lib/modules/lttng-ring-buffer-metadata-client.ko
>>>>> insmod /system/lib/modules/lttng-ring-buffer-client-overwrite.ko
>>>>> insmod /system/lib/modules/lttng-ring-buffer-client-mmap-overwrite.ko
>>>>> insmod /system/lib/modules/lttng-ring-buffer-client-mmap-discard.ko
>>>>> insmod /system/lib/modules/lttng-ring-buffer-client-discard.ko
>>>>>
>>>>>
>>>>> We also haven't had time recently to test if the kprobes/kretprobes
>>>>> worked, but they probably do; we decided to focus a bit more on the
>>>>> 'tracepoints' aspect instead.
>>>>>
>>>>> Anyway, good luck with your exploration, and if you find something
>>>>> interesting, I'd love to know more about it!
>>>>>
>>>>> -PL
>>>>>
>>>>> On Tue, Feb 26, 2013 at 8:06 AM, PLSTC <b0mb00z.it at gmail.com> wrote:
>>>>>
>>>>>> Hey Amit,
>>>>>>
>>>>>> Good news indeed it seems!
>>>>>>
>>>>>> When it comes to pushing the libs/bin, we haven't found any better
>>>>>> way than using adb push yet; of course, if all the projects were included
>>>>>> directly in Android's external, we would simply flash the device after each
>>>>>> build, but currently we're still having trouble generating the correct
>>>>>> Android makefiles.
>>>>>>
>>>>>> As for the modules, I might still have a dep load list somewhere I
>>>>>> could send you in a few hours, I remember fighting to get the order right...
>>>>>>
>>>>>> Also, once everything loads, could you tell me what kind of output
>>>>>> 'lttng list -k' provides you? Here, we're having trouble listing the
>>>>>> 'default' kernel trace points provided by HAVE_TRACEPOINTS, and the
>>>>>> returned list is simply empty (which is a bit strange).
>>>>>>
>>>>>> -PL
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lttng.org/pipermail/lttng-dev/attachments/20130312/ccda299c/attachment-0001.html>
More information about the lttng-dev
mailing list