<div dir="ltr">That's great news !!!<div style>Glad to hear that this is the last hurdle to pass,<br></div><div style>Thanks,</div><div style>Amit.</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Mar 15, 2013 at 5:08 PM, Mathieu Desnoyers <span dir="ltr"><<a href="mailto:mathieu.desnoyers@efficios.com" target="_blank">mathieu.desnoyers@efficios.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">* Pierre-Luc St-Charles (<a href="mailto:pierre-luc.st-charles@polymtl.ca">pierre-luc.st-charles@polymtl.ca</a>) wrote:<br>
> Good news!<br>
><br>
> After peeking yesterday through the different glibc/SysV implementations<br>
> (wrappers...) of splice I could find, I realized that those are not<br>
> actually necessary for what we intend to do: from what I found out, they<br>
> only redefine/wrap/redeclare the sys_splice system call that's implemented<br>
> directly in the kernel (in any architecture). Considering we don't need<br>
> anything more than the default (kernel-space) splice, we can simply rewire<br>
> the 'splice' declaration we use/have to syscall(__NR_splice,...), just like<br>
> what was needed for epoll_create1 earlier.<br>
><br>
> For example : (doing this from memory... I don't have access to the code<br>
> atm)<br>
> > in src/common/compat/fcntl.h, inside the linux ifdef, top of the file:<br>
><br>
> #ifdef __ANDROID__<br>
> #define SPLICE_F_MOVE 0x01<br>
> #define SPLICE_F_NONBLOCK 0x02<br>
> #define SPLICE_F_MORE 0x04<br>
> #define SPLICE_F_GIFT 0x08<br>
> static inline ssize_t splice(int fd_in, loff_t *off_in, int fd_out,<br>
> loff_t *off_out,<br>
> size_t len, unsigned int flags)<br>
> {<br>
> return syscall(__NR_splice,fd_in,loff_in,fd_out,loff_out,len,flags);<br>
> }<br>
> #endif<br>
><br>
><br>
> Result? Boom, traces.<br>
><br>
> The rest of the project seems to work without any other visible problem,<br>
> and we can finally obtain traces when asking the tools nicely. In fact, we<br>
> can obtain traces from any device without even having to modify the<br>
> kernel/system image at all (except to add module and tracepoints support,<br>
> which should already be there), which is pretty surprising, but also very<br>
> useful, considering we can just ship the required binaries anywhere and<br>
> start tracing.<br>
><br>
> Once we confirm that the traces are actually good and we haven't broken<br>
> anything else (we're still not sure liburcu works 100%, we're trying to run<br>
> all the tests), we'll start sending in results for anyone who might be<br>
> interested.<br>
<br>
</div></div>That's great news! So I understand that you got lttng-modules Linux<br>
kernel tracing to work on Android. I look forward to see patches we<br>
could merge into lttng-tools.<br>
<br>
I guess the next step will be to look into LTTng-UST user-space tracing.<br>
<br>
Thanks,<br>
<br>
Mathieu<br>
<div><div class="h5"><br>
><br>
> - PL<br>
><br>
> On Tue, Mar 12, 2013 at 6:51 PM, Pierre-Luc St-Charles <<br>
> <a href="mailto:pierre-luc.st-charles@polymtl.ca">pierre-luc.st-charles@polymtl.ca</a>> wrote:<br>
><br>
> > Hey Amit,<br>
> ><br>
> > Here's a small update on my latest findings regarding the broken *splice*<br>
> > / bad Bionic impl problem:<br>
> ><br>
> > Although it didn't really cross my mind earlier, we had seen this exact<br>
> > problem before, but in another project; we had also investigated a bit at<br>
> > the time, but to no avail. It seems the issue is known by the<br>
> > Android/Bionic community, but it's simply not on their immediate todo list<br>
> > (and hasn't been for at least the past two years -- see<br>
> > <a href="https://code.google.com/p/android/issues/detail?id=11330" target="_blank">https://code.google.com/p/android/issues/detail?id=11330</a>). The solutions<br>
> > we are now considering are: adding splice support (and whatever else might<br>
> > also be needed) to Bionic ourselves (saving BILLIONS of broken lives in the<br>
> > process, IF the changes are accepted), or simply inserting glibc (and<br>
> > building against it) in our device images.<br>
> ><br>
> > In the next few days, we'll be studying how much time the first solution<br>
> > would actually take (as we're not really experts in that domain at all),<br>
> > but considering the nature/objective of this project, it might be the<br>
> > wisest decision (as it would not require too much swerving away from the<br>
> > default images people have on their devices for LTTng to actually work). If<br>
> > we deem it 'impossible/out of our league', we'll simply try to get a<br>
> > working tool asap using glibc, which should not be very complicated itself<br>
> > (many people have actually successfully done it before).<br>
> ><br>
> > In any case, I'll keep you updated, and I'll also post any big updates on<br>
> > the mailing list.<br>
> ><br>
> > -PL<br>
> ><br>
> > On Wed, Mar 6, 2013 at 8:04 PM, Pierre-Luc St-Charles <<br>
> > <a href="mailto:pierre-luc.st-charles@polymtl.ca">pierre-luc.st-charles@polymtl.ca</a>> wrote:<br>
> ><br>
> >> Great findings!<br>
> >><br>
> >> I'll also try to look into this new problem by next monday; I'm falling a<br>
> >> bit behind on my 'investigations', but I'll keep you updated if anything<br>
> >> comes up.<br>
> >><br>
> >> -PL<br>
> >> On Mar 6, 2013 6:09 PM, "Amit Balboul" <<a href="mailto:amit.balboul@shinesec.com">amit.balboul@shinesec.com</a>> wrote:<br>
> >><br>
> >>> Hi All,<br>
> >>><br>
> >>> Regarding the last mail about tracing system calls on Android using<br>
> >>> LTTng,<br>
> >>> The issue of not receiving enabled event to trace from the kernel can be<br>
> >>> fixed patching src/bin/lttng-sessiond/kernel.c (patch attached)<br>
</div></div>> >>> In the function *kernel_list_events,* fscanf-ing the events exported by<br>
> >>> the kernel using the fp must not use the syntax *%m* as it is obsolete<br>
<div class="im">> >>> (in Android),<br>
> >>> thus the fscanf returns 0 and won't read the kernel events (first<br>
> >>> condition false).<br>
</div>> >>> Instead I use a pre-allocated buffer for the string and use the *%[^;]*syntax, and omitting the memory free (attached is the patch).<br>
<div class="im">> >>><br>
> >>> The resulting output is (sessiond is run with verbose debug prints):<br>
> >>><br>
</div>> >>> root@android:/data/lttng/bin # *./lttng list -k*<br>
<div><div class="h5">> >>> DEBUG1 [10239/10290]: Wait for client response (in<br>
> >>> thread_manage_clients() at main.c:3242)<br>
> >>> DEBUG1 [10239/10290]: Receiving data from client ... (in<br>
> >>> thread_manage_clients() at main.c:3287)<br>
> >>> DEBUG1 [10239/10290]: Nothing recv() from client... continuing (in<br>
> >>> thread_manage_clients() at main.c:3291)<br>
> >>> DEBUG1 [10239/10290]: Clean command context structure (in<br>
> >>> clean_command_ctx() at main.c:482)<br>
> >>> DEBUG1 [10239/10290]: Accepting client command ... (in<br>
> >>> thread_manage_clients() at main.c:3200)<br>
> >>> DEBUG1 [10239/10290]: Wait for client response (in<br>
> >>> thread_manage_clients() at main.c:3242)<br>
> >>> DEBUG1 [10239/10290]: Receiving data from client ... (in<br>
> >>> thread_manage_clients() at main.c:3287)<br>
> >>> DEBUG1 [10239/10290]: Processing client command 14 (in<br>
> >>> process_client_msg() at main.c:2227)<br>
> >>> DEBUG1 [10239/10290]: Kernel list events done (61 events) (in<br>
> >>> kernel_list_events() at kernel.c:653)<br>
> >>> DEBUG1 [10239/10290]: Sending response (size: 35640, retcode: Success)<br>
> >>> (in thread_manage_clients() at main.c:3338)<br>
> >>> DEBUG1 [10239/10290]: Clean command context structure (in<br>
> >>> clean_command_ctx() at main.c:482)<br>
> >>> DEBUG1 [10239/10290]: Accepting client command ... (in<br>
> >>> thread_manage_clients() at main.c:3200)<br>
> >>> Kernel events:<br>
> >>> -------------<br>
> >>> block_rq_abort (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> block_rq_requeue (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> block_rq_complete (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> block_rq_insert (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> block_rq_issue (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> block_bio_bounce (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> block_bio_complete (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> block_bio_backmerge (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> block_bio_frontmerge (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> block_bio_queue (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> block_getrq (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> block_sleeprq (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> block_plug (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> block_unplug (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> block_split (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> block_bio_remap (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> block_rq_remap (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> irq_handler_entry (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> irq_handler_exit (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> softirq_entry (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> softirq_exit (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> softirq_raise (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> sched_kthread_stop (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> sched_kthread_stop_ret (loglevel: TRACE_EMERG (0)) (type:<br>
> >>> tracepoint)<br>
> >>> sched_wakeup (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> sched_wakeup_new (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> sched_switch (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> sched_migrate_task (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> sched_process_free (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> sched_process_exit (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> sched_wait_task (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> sched_process_wait (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> sched_process_fork (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> sched_stat_wait (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> sched_stat_sleep (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> sched_stat_iowait (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> sched_stat_runtime (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> sched_pi_setprio (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> signal_generate (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> signal_deliver (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> signal_overflow_fail (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> signal_lose_info (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> lttng_statedump_start (loglevel: TRACE_EMERG (0)) (type:<br>
> >>> tracepoint)<br>
> >>> lttng_statedump_end (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> lttng_statedump_process_state (loglevel: TRACE_EMERG (0)) (type:<br>
> >>> tracepoint)<br>
> >>> lttng_statedump_file_descriptor (loglevel: TRACE_EMERG (0)) (type:<br>
> >>> tracepoint)<br>
> >>> lttng_statedump_vm_map (loglevel: TRACE_EMERG (0)) (type:<br>
> >>> tracepoint)<br>
> >>> lttng_statedump_network_interface (loglevel: TRACE_EMERG (0))<br>
> >>> (type: tracepoint)<br>
> >>> lttng_statedump_interrupt (loglevel: TRACE_EMERG (0)) (type:<br>
> >>> tracepoint)<br>
> >>> timer_init (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> timer_start (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> timer_expire_entry (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> timer_expire_exit (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> timer_cancel (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> hrtimer_init (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> hrtimer_start (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> hrtimer_expire_entry (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> hrtimer_expire_exit (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> hrtimer_cancel (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> itimer_state (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>> itimer_expire (loglevel: TRACE_EMERG (0)) (type: tracepoint)<br>
> >>><br>
> >>> root@android:/data/lttng/bin #<br>
> >>><br>
> >>> Now I'm facing the issue of starting the traces:<br>
> >>> debugging the sessiond, consumerd and lttng I concluded that the issue<br>
> >>> is due to splice system call failure.<br>
> >>> even using a MMAP output type channel, the metadata channel is SPLICE<br>
> >>> type (hard-coded).<br>
> >>> Problem persists also when using relayd to pass the traces through the<br>
> >>> network.<br>
> >>> Ignoring the splice failure means closing of the metadata channel<br>
> >>> (maybe??) but need not affect the traces channel (right ?)<br>
> >>> Attached is a debug log of each thread of the sessiond, right after<br>
</div></div>> >>> invoking "*./lttng start*". (with extra informative my prints)<br>
<div><div class="h5">> >>><br>
> >>> Please keep me in touch with any progress in this issue,<br>
> >>> Thank you,<br>
> >>> Amit.<br>
> >>><br>
> >>><br>
> >>><br>
> >>><br>
> >>><br>
> >>><br>
> >>> ---------- Forwarded message ----------<br>
> >>> From: Amit Balboul <<a href="mailto:amit.balboul@shinesec.com">amit.balboul@shinesec.com</a>><br>
> >>> Date: Wed, Feb 27, 2013 at 4:37 PM<br>
> >>> Subject: Re: [lttng-dev] compiling LTTng for android<br>
> >>> To: PLSTC <<a href="mailto:b0mb00z.it@gmail.com">b0mb00z.it@gmail.com</a>><br>
> >>><br>
> >>><br>
> >>> Hey Pierre,<br>
> >>><br>
> >>> Some major progress has been achieved, following is the summary:<br>
> >>><br>
> >>> 1. Modules issue:<br>
> >>> I built (for ARM/Android) and installed latest stable version of busybox<br>
> >>> on the device.<br>
> >>> Seeing the code tries to run (hardcoded) /sbin/modprobe, made a soft<br>
> >>> link to busybox's modeprobe.<br>
> >>> Also, created /lib/modules/<version+build> directory and pushed the<br>
> >>> <version+build> directory from /lib/modules on the machine.<br>
> >>> Now session daemon modprobes the desired modules successfully, (some are<br>
> >>> not present though as mentioned before) - no need to use insmod.<br>
> >>><br>
> >>> 2. epoll_create1 issue:<br>
> >>> epoll_create1 patch was OK, I accidentally defined EPOLL_CLOEXEC as<br>
> >>> 0x02000000 instead of 02000000 (fixed).<br>
> >>><br>
> >>> The running scenario:<br>
> >>><br>
> >>> 1. Run ./lttng-sessiond -d -vvv<br>
> >>><br>
> >>> Then: (commands are in bold, output is in blue for readability)<br>
> >>><br>
</div></div>> >>> root@android:/data/lttng/install_sysroot/bin # *./lttng create s1*<br>
<div class="im">> >>><br>
> >>> Session s1 created.<br>
> >>> Traces will be written in /data/lttng-traces/s1-20130227-161145<br>
</div>> >>> root@android:/data/lttng/install_sysroot/bin # *./lttng enable-channel<br>
> >>> ch -k*<br>
<div class="im">> >>> Kernel channel ch enabled for session s1<br>
</div>> >>> root@android:/data/lttng/install_sysroot/bin # *./lttng enable-event ev<br>
> >>> -k -a*<br>
<div class="im">> >>> All kernel events are enabled in channel channel0<br>
</div>> >>> root@android:/data/lttng/install_sysroot/bin # *./lttng list s1*<br>
<div><div class="h5">> >>> Tracing session s1: [inactive]<br>
> >>> Trace path: /data/lttng-traces/s1-20130227-161145<br>
> >>><br>
> >>> === Domain: Kernel ===<br>
> >>><br>
> >>> Channels:<br>
> >>> -------------<br>
> >>> - channel0: [enabled]<br>
> >>><br>
> >>> Attributes:<br>
> >>> overwrite mode: 0<br>
> >>> subbufers size: 262144<br>
> >>> number of subbufers: 4<br>
> >>> switch timer interval: 0<br>
> >>> read timer interval: 200<br>
> >>> output: splice()<br>
> >>><br>
> >>> Events:<br>
> >>> None<br>
> >>><br>
> >>> - ch: [enabled]<br>
> >>><br>
> >>> Attributes:<br>
> >>> overwrite mode: 0<br>
> >>> subbufers size: 262144<br>
> >>> number of subbufers: 4<br>
> >>> switch timer interval: 0<br>
> >>> read timer interval: 200<br>
> >>> output: splice()<br>
> >>><br>
> >>> Events:<br>
> >>> None<br>
> >>><br>
</div></div>> >>> root@android:/data/lttng/install_sysroot/bin # *./lttng list -k *<br>
> >>><br>
> >>> Kernel events:<br>
> >>> -------------<br>
> >>><br>
> >>> root@android:/data/lttng/install_sysroot/bin # *./lttng start*<br>
<div class="im">> >>> Tracing started for session s1<br>
</div>> >>> root@android:/data/lttng/install_sysroot/bin # *./lttng stop *<br>
<div class="im">> >>><br>
> >>> Waiting for data availability<br>
> >>> Tracing stopped for session s1<br>
> >>> root@android:/data/lttng/install_sysroot/bin #<br>
> >>><br>
</div>> >>> *Now: (this is not surprising because the kernel events list is<br>
> >>> empty...)*<br>
<div class="im">> >>> the folder /data/lttng-traces/s1-20130227-161145/kernel contains:<br>
> >>> total 0<br>
> >>> -rwxrwxrwx 1 0 0 0 Feb 27 14:18 ch_0<br>
> >>> -rwxrwxrwx 1 0 0 0 Feb 27 14:18 ch_1<br>
> >>> -rwxrwxrwx 1 0 0 0 Feb 27 14:18 channel0_0<br>
> >>> -rwxrwxrwx 1 0 0 0 Feb 27 14:18 channel0_1<br>
> >>> -rwxrwxrwx 1 0 0 0 Feb 27 14:18 metadata<br>
> >>><br>
</div>> >>> *By the way:*<br>
> >>> When running the same scenario but creating the channel with *--output<br>
> >>> mmap*, still no data but the size of the first two files in the traces<br>
<div class="im">> >>> directory are 4Kb (I didn't bother to investigate them as I assume it is<br>
> >>> just the header - not the traces payload).<br>
> >>><br>
> >>> Another issue: at this point, listing the sessions gives:<br>
</div>> >>> root@android:/data/lttng/install_sysroot/bin # *./lttng list s1 *<br>
<div class="im">> >>><br>
> >>> Tracing session s1: [inactive]<br>
> >>> Trace path: /data/lttng-traces/s1-20130227-161145<br>
> >>><br>
> >>> === Domain: Kernel ===<br>
> >>><br>
> >>> Error: No kernel consumer detected<br>
</div>> >>> 99|root@android:/data/lttng/install_sysroot/bin # *./lttng list -k *<br>
<div class="HOEnZb"><div class="h5">> >>><br>
> >>> Error: Unable to list kernel events<br>
> >>> Error: Command error<br>
> >>> 1|root@android:/data/lttng/install_sysroot/bin #<br>
> >>><br>
> >>> So currently I'm facing these issues:<br>
> >>> 1. Kernel events list is empty.<br>
> >>> 2. Listing the session/ kernel events after creating session, channel,<br>
> >>> event, start, stop - results in "no kernel consumer detected"<br>
> >>><br>
> >>> If you have some insights please share them with me..<br>
> >>><br>
> >>> Thanks<br>
> >>><br>
> >>> Amit.<br>
> >>><br>
> >>><br>
> >>> On Tue, Feb 26, 2013 at 7:35 PM, PLSTC <<a href="mailto:b0mb00z.it@gmail.com">b0mb00z.it@gmail.com</a>> wrote:<br>
> >>><br>
> >>>> The modules you can't find are the same I am also missing I believe; I<br>
> >>>> think they are only compiled when certain kernel config defines are met<br>
> >>>> (and those might simply be unavailable under ARM).<br>
> >>>><br>
> >>>> As for what you pointed in your first mail, I believe modprobe isn't<br>
> >>>> available on ARM, did you mean that some modules were not found when using<br>
> >>>> insmod?<br>
> >>>><br>
> >>>> For the epoll_create1 problem, what kind of patch did you apply to add<br>
> >>>> support for the epoll_create1 function?<br>
> >>>><br>
> >>>><br>
> >>>> On Tue, Feb 26, 2013 at 11:29 AM, Amit Balboul <<br>
> >>>> <a href="mailto:amit.balboul@shinesec.com">amit.balboul@shinesec.com</a>> wrote:<br>
> >>>><br>
> >>>>> Ok I got :<br>
> >>>>> lttng-kretprobes.ko<br>
> >>>>> lttng-kprobes.ko<br>
> >>>>> , the other two are missing.<br>
> >>>>><br>
> >>>>><br>
> >>>>> ---------- Forwarded message ----------<br>
> >>>>> From: Amit Balboul <<a href="mailto:amit.balboul@shinesec.com">amit.balboul@shinesec.com</a>><br>
> >>>>> Date: Tue, Feb 26, 2013 at 5:49 PM<br>
> >>>>> Subject: Re: [lttng-dev] compiling LTTng for android<br>
> >>>>> To: PLSTC <<a href="mailto:b0mb00z.it@gmail.com">b0mb00z.it@gmail.com</a>><br>
> >>>>><br>
> >>>>><br>
> >>>>> One more thing:<br>
> >>>>><br>
> >>>>> The list of modules you gave my is not identical to mine:<br>
> >>>>> a. I pushed the modules from my machine to my folder in the device<br>
> >>>>> (not to /system/lib) as you directed me, and insmoded them manually.<br>
> >>>>> b. I pushed the whole tree under /lib/modules/<KERNEL_VER_BUILD>/<br>
> >>>>> which includes the modules.* and the sub directories:<br>
> >>>>><br>
> >>>>> /lib/modules/3.0.31-gd5a18e0/extra$ ll<br>
> >>>>> total 2464<br>
> >>>>> drwxr-xr-x 4 root root 4096 Feb 26 15:58 ./<br>
> >>>>> drwxr-xr-x 3 root root 4096 Feb 26 15:58 ../<br>
> >>>>> drwxr-xr-x 2 root root 4096 Feb 26 15:58 lib/<br>
> >>>>> -rw-r--r-- 1 root root 167184 Feb 26 15:58<br>
> >>>>> lttng-ring-buffer-client-discard.ko<br>
> >>>>> -rw-r--r-- 1 root root 167206 Feb 26 15:58<br>
> >>>>> lttng-ring-buffer-client-mmap-discard.ko<br>
> >>>>> -rw-r--r-- 1 root root 178586 Feb 26 15:58<br>
> >>>>> lttng-ring-buffer-client-mmap-overwrite.ko<br>
> >>>>> -rw-r--r-- 1 root root 178564 Feb 26 15:58<br>
> >>>>> lttng-ring-buffer-client-overwrite.ko<br>
> >>>>> -rw-r--r-- 1 root root 137179 Feb 26 15:58<br>
> >>>>> lttng-ring-buffer-metadata-client.ko<br>
> >>>>> -rw-r--r-- 1 root root 137233 Feb 26 15:58<br>
> >>>>> lttng-ring-buffer-metadata-mmap-client.ko<br>
> >>>>> -rw-r--r-- 1 root root 213451 Feb 26 15:58 lttng-statedump.ko<br>
> >>>>> -rw-r--r-- 1 root root 1314235 Feb 26 15:58 lttng-tracer.ko<br>
> >>>>> drwxr-xr-x 2 root root 4096 Feb 26 15:58 probes/<br>
> >>>>><br>
> >>>>> /lib/modules/3.0.31-gd5a18e0/extra/lib$ ll<br>
> >>>>> total 732<br>
> >>>>> drwxr-xr-x 2 root root 4096 Feb 26 15:58 ./<br>
> >>>>> drwxr-xr-x 4 root root 4096 Feb 26 15:58 ../<br>
> >>>>> -rw-r--r-- 1 root root 738964 Feb 26 15:58 lttng-lib-ring-buffer.ko<br>
> >>>>><br>
> >>>>> /lib/modules/3.0.31-gd5a18e0/extra/probes$ ll<br>
> >>>>> total 2872<br>
> >>>>> drwxr-xr-x 2 root root 4096 Feb 26 15:58 ./<br>
> >>>>> drwxr-xr-x 4 root root 4096 Feb 26 15:58 ../<br>
> >>>>> -rw-r--r-- 1 root root 162872 Feb 26 15:58 lttng-probe-asoc.ko<br>
> >>>>> -rw-r--r-- 1 root root 171049 Feb 26 15:58 lttng-probe-block.ko<br>
> >>>>> -rw-r--r-- 1 root root 81165 Feb 26 15:58 lttng-probe-compaction.ko<br>
> >>>>> -rw-r--r-- 1 root root 333215 Feb 26 15:58 lttng-probe-ext4.ko<br>
> >>>>> -rw-r--r-- 1 root root 80413 Feb 26 15:58 lttng-probe-gpio.ko<br>
> >>>>> -rw-r--r-- 1 root root 84256 Feb 26 15:58 lttng-probe-irq.ko<br>
> >>>>> -rw-r--r-- 1 root root 135453 Feb 26 15:58 lttng-probe-jbd2.ko<br>
> >>>>> -rw-r--r-- 1 root root 105399 Feb 26 15:58 lttng-probe-kmem.ko<br>
> >>>>> -rw-r--r-- 1 root root 76240 Feb 26 15:58 lttng-probe-lttng.ko<br>
> >>>>> -rw-r--r-- 1 root root 88059 Feb 26 15:58 lttng-probe-module.ko<br>
> >>>>> -rw-r--r-- 1 root root 154032 Feb 26 15:58 lttng-probe-napi.ko<br>
> >>>>> -rw-r--r-- 1 root root 142038 Feb 26 15:58 lttng-probe-net.ko<br>
> >>>>> -rw-r--r-- 1 root root 93564 Feb 26 15:58 lttng-probe-power.ko<br>
> >>>>> -rw-r--r-- 1 root root 85530 Feb 26 15:58 lttng-probe-regulator.ko<br>
> >>>>> -rw-r--r-- 1 root root 138121 Feb 26 15:58 lttng-probe-sched.ko<br>
> >>>>> -rw-r--r-- 1 root root 152909 Feb 26 15:58 lttng-probe-scsi.ko<br>
> >>>>> -rw-r--r-- 1 root root 107491 Feb 26 15:58 lttng-probe-signal.ko<br>
> >>>>> -rw-r--r-- 1 root root 142711 Feb 26 15:58 lttng-probe-skb.ko<br>
> >>>>> -rw-r--r-- 1 root root 181983 Feb 26 15:58 lttng-probe-statedump.ko<br>
> >>>>> -rw-r--r-- 1 root root 126731 Feb 26 15:58 lttng-probe-timer.ko<br>
> >>>>> -rw-r--r-- 1 root root 86591 Feb 26 15:58 lttng-probe-workqueue.ko<br>
> >>>>> -rw-r--r-- 1 root root 133886 Feb 26 15:58 lttng-probe-writeback.ko<br>
> >>>>> -rw-r--r-- 1 root root 30863 Feb 26 15:58 lttng-types.ko<br>
> >>>>><br>
> >>>>> Looking at the files shows I miss:<br>
> >>>>> lttng-kretprobes.ko<br>
> >>>>> lttng-kprobes.ko<br>
> >>>>> lttng-ftrace.ko<br>
> >>>>> lttng-probe-kvm.ko<br>
> >>>>><br>
> >>>>><br>
> >>>>> On Tue, Feb 26, 2013 at 3:56 PM, PLSTC <<a href="mailto:b0mb00z.it@gmail.com">b0mb00z.it@gmail.com</a>> wrote:<br>
> >>>>><br>
> >>>>>> Back on the last email:<br>
> >>>>>><br>
> >>>>>> Here's the insmod list I ended up building; you might not have<br>
> >>>>>> compiled all those modules, depending on your kernel config and on the<br>
> >>>>>> architecture used, so some might simply be absent from your output<br>
> >>>>>> directory.<br>
> >>>>>><br>
> >>>>>> insmod /system/lib/modules/lttng-types.ko<br>
> >>>>>> insmod /system/lib/modules/lttng-kretprobes.ko<br>
> >>>>>> insmod /system/lib/modules/lttng-kprobes.ko<br>
> >>>>>> insmod /system/lib/modules/lttng-ftrace.ko<br>
> >>>>>> insmod /system/lib/modules/lttng-lib-ring-buffer.ko<br>
> >>>>>> insmod /system/lib/modules/lttng-statedump.ko<br>
> >>>>>> insmod /system/lib/modules/lttng-tracer.ko<br>
> >>>>>> insmod /system/lib/modules/lttng-probe-timer.ko<br>
> >>>>>> insmod /system/lib/modules/lttng-probe-statedump.ko<br>
> >>>>>> insmod /system/lib/modules/lttng-probe-signal.ko<br>
> >>>>>> insmod /system/lib/modules/lttng-probe-sched.ko<br>
> >>>>>> insmod /system/lib/modules/lttng-probe-lttng.ko<br>
> >>>>>> insmod /system/lib/modules/lttng-probe-kvm.ko<br>
> >>>>>> insmod /system/lib/modules/lttng-probe-irq.ko<br>
> >>>>>> insmod /system/lib/modules/lttng-probe-block.ko<br>
> >>>>>> insmod /system/lib/modules/lttng-ring-buffer-metadata-mmap-client.ko<br>
> >>>>>> insmod /system/lib/modules/lttng-ring-buffer-metadata-client.ko<br>
> >>>>>> insmod /system/lib/modules/lttng-ring-buffer-client-overwrite.ko<br>
> >>>>>> insmod /system/lib/modules/lttng-ring-buffer-client-mmap-overwrite.ko<br>
> >>>>>> insmod /system/lib/modules/lttng-ring-buffer-client-mmap-discard.ko<br>
> >>>>>> insmod /system/lib/modules/lttng-ring-buffer-client-discard.ko<br>
> >>>>>><br>
> >>>>>><br>
> >>>>>> We also haven't had time recently to test if the kprobes/kretprobes<br>
> >>>>>> worked, but they probably do; we decided to focus a bit more on the<br>
> >>>>>> 'tracepoints' aspect instead.<br>
> >>>>>><br>
> >>>>>> Anyway, good luck with your exploration, and if you find something<br>
> >>>>>> interesting, I'd love to know more about it!<br>
> >>>>>><br>
> >>>>>> -PL<br>
> >>>>>><br>
> >>>>>> On Tue, Feb 26, 2013 at 8:06 AM, PLSTC <<a href="mailto:b0mb00z.it@gmail.com">b0mb00z.it@gmail.com</a>> wrote:<br>
> >>>>>><br>
> >>>>>>> Hey Amit,<br>
> >>>>>>><br>
> >>>>>>> Good news indeed it seems!<br>
> >>>>>>><br>
> >>>>>>> When it comes to pushing the libs/bin, we haven't found any better<br>
> >>>>>>> way than using adb push yet; of course, if all the projects were included<br>
> >>>>>>> directly in Android's external, we would simply flash the device after each<br>
> >>>>>>> build, but currently we're still having trouble generating the correct<br>
> >>>>>>> Android makefiles.<br>
> >>>>>>><br>
> >>>>>>> As for the modules, I might still have a dep load list somewhere I<br>
> >>>>>>> could send you in a few hours, I remember fighting to get the order right...<br>
> >>>>>>><br>
> >>>>>>> Also, once everything loads, could you tell me what kind of output<br>
> >>>>>>> 'lttng list -k' provides you? Here, we're having trouble listing the<br>
> >>>>>>> 'default' kernel trace points provided by HAVE_TRACEPOINTS, and the<br>
> >>>>>>> returned list is simply empty (which is a bit strange).<br>
> >>>>>>><br>
> >>>>>>> -PL<br>
> >>>>>>><br>
> >>>>>><br>
> >>>>>><br>
> >>>>><br>
> >>>>><br>
> >>>><br>
> >>><br>
> >>><br>
> ><br>
<br>
</div></div><div class="HOEnZb"><div class="h5">> _______________________________________________<br>
> lttng-dev mailing list<br>
> <a href="mailto:lttng-dev@lists.lttng.org">lttng-dev@lists.lttng.org</a><br>
> <a href="http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev" target="_blank">http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev</a><br>
<br>
<br>
</div></div><span class="HOEnZb"><font color="#888888">--<br>
Mathieu Desnoyers<br>
EfficiOS Inc.<br>
<a href="http://www.efficios.com" target="_blank">http://www.efficios.com</a><br>
</font></span></blockquote></div><br></div>