<p dir="ltr">Great findings!</p>
<p dir="ltr">I'll also try to look into this new problem by next monday; I'm falling a bit behind on my 'investigations', but I'll keep you updated if anything comes up.</p>
<p dir="ltr">-PL</p>
<div class="gmail_quote">On Mar 6, 2013 6:09 PM, "Amit Balboul" <<a href="mailto:amit.balboul@shinesec.com">amit.balboul@shinesec.com</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Hi All,<div><br></div><div>Regarding the last mail about tracing system calls on Android using LTTng,</div><div>The issue of not receiving enabled event to trace from the kernel can be fixed patching <font face="courier new, monospace">src/bin/lttng-sessiond/kernel.c </font><font face="arial, helvetica, sans-serif">(patch attached)</font></div>
<div>In the function <i><font face="courier new, monospace">kernel_list_events,</font></i> fscanf-ing the events exported by the kernel using the fp must not use the syntax <b>%m</b> as it is obsolete (in Android),</div>
<div>thus the fscanf returns 0 and won't read the kernel events (first condition false).</div><div>Instead I use a pre-allocated buffer for the string and use the <b>%[^;]</b> syntax, and omitting the memory free (attached is the patch).</div>
<div><br></div><div>The resulting output is (sessiond is run with verbose debug prints):<br></div><div><br></div><div><div><font face="courier new, monospace">root@android:/data/lttng/bin # <b>./lttng list -k</b></font></div>
<div><font face="courier new, monospace">DEBUG1 [10239/10290]: Wait for client response (in thread_manage_clients() at main.c:3242)</font></div><div><font face="courier new, monospace">DEBUG1 [10239/10290]: Receiving data from client ... (in thread_manage_clients() at main.c:3287)</font></div>
<div><font face="courier new, monospace">DEBUG1 [10239/10290]: Nothing recv() from client... continuing (in thread_manage_clients() at main.c:3291)</font></div><div><font face="courier new, monospace">DEBUG1 [10239/10290]: Clean command context structure (in clean_command_ctx() at main.c:482)</font></div>
<div><font face="courier new, monospace">DEBUG1 [10239/10290]: Accepting client command ... (in thread_manage_clients() at main.c:3200)</font></div><div><font face="courier new, monospace">DEBUG1 [10239/10290]: Wait for client response (in thread_manage_clients() at main.c:3242)</font></div>
<div><font face="courier new, monospace">DEBUG1 [10239/10290]: Receiving data from client ... (in thread_manage_clients() at main.c:3287)</font></div><div><font face="courier new, monospace">DEBUG1 [10239/10290]: Processing client command 14 (in process_client_msg() at main.c:2227)</font></div>
<div><font face="courier new, monospace">DEBUG1 [10239/10290]: Kernel list events done (61 events) (in kernel_list_events() at kernel.c:653)</font></div><div><font face="courier new, monospace">DEBUG1 [10239/10290]: Sending response (size: 35640, retcode: Success) (in thread_manage_clients() at main.c:3338)</font></div>
<div><font face="courier new, monospace">DEBUG1 [10239/10290]: Clean command context structure (in clean_command_ctx() at main.c:482)</font></div><div><font face="courier new, monospace">DEBUG1 [10239/10290]: Accepting client command ... (in thread_manage_clients() at main.c:3200)</font></div>
<div><font face="courier new, monospace">Kernel events:</font></div><div><font face="courier new, monospace">-------------</font></div><div><font face="courier new, monospace"> block_rq_abort (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> block_rq_requeue (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> block_rq_complete (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> block_rq_insert (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> block_rq_issue (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> block_bio_bounce (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> block_bio_complete (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> block_bio_backmerge (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> block_bio_frontmerge (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> block_bio_queue (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> block_getrq (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> block_sleeprq (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> block_plug (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> block_unplug (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> block_split (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> block_bio_remap (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> block_rq_remap (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> irq_handler_entry (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> irq_handler_exit (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> softirq_entry (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> softirq_exit (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> softirq_raise (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> sched_kthread_stop (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> sched_kthread_stop_ret (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> sched_wakeup (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> sched_wakeup_new (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> sched_switch (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> sched_migrate_task (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> sched_process_free (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> sched_process_exit (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> sched_wait_task (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> sched_process_wait (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> sched_process_fork (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> sched_stat_wait (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> sched_stat_sleep (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> sched_stat_iowait (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> sched_stat_runtime (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> sched_pi_setprio (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> signal_generate (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> signal_deliver (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> signal_overflow_fail (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> signal_lose_info (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> lttng_statedump_start (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> lttng_statedump_end (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> lttng_statedump_process_state (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> lttng_statedump_file_descriptor (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> lttng_statedump_vm_map (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> lttng_statedump_network_interface (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> lttng_statedump_interrupt (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> timer_init (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> timer_start (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> timer_expire_entry (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> timer_expire_exit (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> timer_cancel (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> hrtimer_init (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> hrtimer_start (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> hrtimer_expire_entry (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> hrtimer_expire_exit (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> hrtimer_cancel (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"> itimer_state (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div><div><font face="courier new, monospace"> itimer_expire (loglevel: TRACE_EMERG (0)) (type: tracepoint)</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">root@android:/data/lttng/bin #</font></div><div><br></div><div>Now I'm facing the issue of starting the traces:</div>
<div>debugging the sessiond, consumerd and lttng I concluded that the issue is due to <font face="courier new, monospace">splice</font> system call failure.</div><div>even using a MMAP output type channel, the metadata channel is SPLICE type (hard-coded).</div>
<div>Problem persists also when using relayd to pass the traces through the network.</div><div>Ignoring the splice failure means closing of the metadata channel (maybe??) but need not affect the traces channel (right ?)</div>
<div>Attached is a debug log of each thread of the sessiond, right after invoking "<b><font face="courier new, monospace">./lttng start</font></b>". (with extra informative my prints)</div><div><br></div>
<div>Please keep me in touch with any progress in this issue,</div><div>Thank you,</div><div>Amit.</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br>
</div><div><br></div></div><div><div class="gmail_quote">---------- Forwarded message ----------<br>From: <b class="gmail_sendername">Amit Balboul</b> <span dir="ltr"><<a href="mailto:amit.balboul@shinesec.com" target="_blank">amit.balboul@shinesec.com</a>></span><br>
Date: Wed, Feb 27, 2013 at 4:37 PM<br>Subject: Re: [lttng-dev] compiling LTTng for android<br>To: PLSTC <<a href="mailto:b0mb00z.it@gmail.com" target="_blank">b0mb00z.it@gmail.com</a>><br><br><br><div dir="ltr">Hey Pierre,<div>
<br>
</div><div>Some major progress has been achieved, following is the summary:</div><div><br></div><div>1. Modules issue:</div><div>I built (for ARM/Android) and installed latest stable version of busybox on the device.</div>
<div>Seeing the code tries to run (hardcoded) /sbin/modprobe, made a soft link to busybox's modeprobe.</div><div>Also, created /lib/modules/<version+build> directory and pushed the <version+build> directory from /lib/modules on the machine.</div>
<div>Now session daemon modprobes the desired modules successfully, (some are not present though as mentioned before) - no need to use insmod.</div><div><br></div><div>2. epoll_create1 issue:</div><div>
epoll_create1 patch was OK, I accidentally defined EPOLL_CLOEXEC as 0x02000000 instead of 02000000 (fixed).</div><div><br></div><div>The running scenario:</div><div><br></div><div>1. Run <font face="courier new, monospace">./lttng-sessiond -d -vvv</font><br>
</div><div><br></div><div>Then: (commands are in bold, output is in blue for readability)</div><div><br></div><div><div><font face="courier new, monospace">root@android:/data/lttng/install_sysroot/bin # <b>./lttng create s1</b> </font></div>
<div><font face="courier new, monospace" color="#0000ff">Session s1 created.</font></div><div><font face="courier new, monospace" color="#0000ff">Traces will be written in /data/lttng-traces/s1-20130227-161145</font></div>
<div><font face="courier new, monospace">root@android:/data/lttng/install_sysroot/bin # <b>./lttng enable-channel ch -k</b> </font></div><div><font face="courier new, monospace" color="#0000ff">Kernel channel ch enabled for session s1</font></div>
<div><font face="courier new, monospace">root@android:/data/lttng/install_sysroot/bin # <b>./lttng enable-event ev -k -a</b></font></div><div><font face="courier new, monospace" color="#0000ff">All kernel events are enabled in channel channel0</font></div>
<div><font face="courier new, monospace">root@android:/data/lttng/install_sysroot/bin # <b>./lttng list s1</b></font></div><div><font face="courier new, monospace" color="#0000ff">Tracing session s1: [inactive]</font></div>
<div><font face="courier new, monospace" color="#0000ff"> Trace path: /data/lttng-traces/s1-20130227-161145</font></div><div><font face="courier new, monospace" color="#0000ff"><br></font></div><div><font face="courier new, monospace" color="#0000ff">=== Domain: Kernel ===</font></div>
<div><font face="courier new, monospace" color="#0000ff"><br></font></div><div><font face="courier new, monospace" color="#0000ff">Channels:</font></div><div><font face="courier new, monospace" color="#0000ff">-------------</font></div>
<div><font face="courier new, monospace" color="#0000ff">- channel0: [enabled]</font></div><div><font face="courier new, monospace" color="#0000ff"><br></font></div><div><font face="courier new, monospace" color="#0000ff"> Attributes:</font></div>
<div><font face="courier new, monospace" color="#0000ff"> overwrite mode: 0</font></div><div><font face="courier new, monospace" color="#0000ff"> subbufers size: 262144</font></div><div><font face="courier new, monospace" color="#0000ff"> number of subbufers: 4</font></div>
<div><font face="courier new, monospace" color="#0000ff"> switch timer interval: 0</font></div><div><font face="courier new, monospace" color="#0000ff"> read timer interval: 200</font></div><div><font face="courier new, monospace" color="#0000ff"> output: splice()</font></div>
<div><font face="courier new, monospace" color="#0000ff"><br></font></div><div><font face="courier new, monospace" color="#0000ff"> Events:</font></div><div><font face="courier new, monospace" color="#0000ff"> None</font></div>
<div><font face="courier new, monospace" color="#0000ff"><br></font></div><div><font face="courier new, monospace" color="#0000ff">- ch: [enabled]</font></div><div><font face="courier new, monospace" color="#0000ff"><br>
</font></div>
<div><font face="courier new, monospace" color="#0000ff"> Attributes:</font></div><div><font face="courier new, monospace" color="#0000ff"> overwrite mode: 0</font></div><div><font face="courier new, monospace" color="#0000ff"> subbufers size: 262144</font></div>
<div><font face="courier new, monospace" color="#0000ff"> number of subbufers: 4</font></div><div><font face="courier new, monospace" color="#0000ff"> switch timer interval: 0</font></div><div><font face="courier new, monospace" color="#0000ff"> read timer interval: 200</font></div>
<div><font face="courier new, monospace" color="#0000ff"> output: splice()</font></div><div><font face="courier new, monospace" color="#0000ff"><br></font></div><div><font face="courier new, monospace" color="#0000ff"> Events:</font></div>
<div><font face="courier new, monospace" color="#0000ff"> None</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">root@android:/data/lttng/install_sysroot/bin # <b>./lttng list -k </b> </font></div>
<div><font face="courier new, monospace" color="#0000ff">Kernel events:</font></div><div><font face="courier new, monospace" color="#0000ff">-------------</font></div><div><font face="courier new, monospace"><br></font></div>
<div><font face="courier new, monospace">root@android:/data/lttng/install_sysroot/bin # <b>./lttng start</b></font></div><div><font face="courier new, monospace" color="#0000ff">Tracing started for session s1</font></div>
<div><font face="courier new, monospace">root@android:/data/lttng/install_sysroot/bin # <b>./lttng stop </b> </font></div><div><font face="courier new, monospace" color="#0000ff">Waiting for data availability</font></div>
<div><font face="courier new, monospace" color="#0000ff">Tracing stopped for session s1</font></div><div><font face="courier new, monospace">root@android:/data/lttng/install_sysroot/bin # </font></div><div><br></div></div>
<div><u>Now: (this is not surprising because the kernel events list is empty...)</u></div><div>the folder /<font face="courier new, monospace">data/lttng-traces/s1-20130227-161145/kernel</font> contains:</div>
<div><div><font face="courier new, monospace">total 0</font></div><div><font face="courier new, monospace">-rwxrwxrwx 1 0 0 0 Feb 27 14:18 ch_0</font></div><div><font face="courier new, monospace">-rwxrwxrwx 1 0 0 0 Feb 27 14:18 ch_1</font></div>
<div><font face="courier new, monospace">-rwxrwxrwx 1 0 0 0 Feb 27 14:18 channel0_0</font></div><div><font face="courier new, monospace">-rwxrwxrwx 1 0 0 0 Feb 27 14:18 channel0_1</font></div>
<div><font face="courier new, monospace">-rwxrwxrwx 1 0 0 0 Feb 27 14:18 metadata</font></div><div><br></div></div><div><u>By the way:</u></div><div>When running the same scenario but creating the channel with <font face="courier new, monospace"><b>--output mmap</b></font>, still no data but the size of the first two files in the traces directory are 4Kb (I didn't bother to investigate them as I assume it is just the header - not the traces payload).</div>
<div><br></div><div>Another issue: at this point, listing the sessions gives:<br></div><div><div><font face="courier new, monospace">root@android:/data/lttng/install_sysroot/bin # <b>./lttng list s1 </b> </font></div>
<div><font face="courier new, monospace" color="#0000ff">Tracing session s1: [inactive]</font></div><div><font face="courier new, monospace" color="#0000ff"> Trace path: /data/lttng-traces/s1-20130227-161145</font></div>
<div><font face="courier new, monospace" color="#0000ff"><br></font></div><div><font face="courier new, monospace" color="#0000ff">=== Domain: Kernel ===</font></div><div><font face="courier new, monospace" color="#0000ff"><br>
</font></div><div><font face="courier new, monospace" color="#0000ff">Error: No kernel consumer detected</font></div><div><font face="courier new, monospace">99|root@android:/data/lttng/install_sysroot/bin # <b>./lttng list -k </b> </font></div>
<div><font face="courier new, monospace" color="#0000ff">Error: Unable to list kernel events</font></div><div><font face="courier new, monospace" color="#0000ff">Error: Command error</font></div><div><font face="courier new, monospace">1|root@android:/data/lttng/install_sysroot/bin # </font></div>
<div><font face="courier new, monospace"><br></font></div></div><div>So currently I'm facing these issues:</div><div>1. Kernel events list is empty.</div><div>2. Listing the session/ kernel events after creating session, channel, event, start, stop - results in "no kernel consumer detected"</div>
<div><br></div><div>If you have some insights please share them with me..</div><div><br></div><div>Thanks</div><span><font color="#888888"><div><br></div><div>Amit.</div></font></span></div><div><div>
<div class="gmail_extra"><br><br><div class="gmail_quote">
On Tue, Feb 26, 2013 at 7:35 PM, PLSTC <span dir="ltr"><<a href="mailto:b0mb00z.it@gmail.com" target="_blank">b0mb00z.it@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
The modules you can't find are the same I am also missing I believe; I think they are only compiled when certain kernel config defines are met (and those might simply be unavailable under ARM).<div><br></div><div>As for what you pointed in your first mail, I believe modprobe isn't available on ARM, did you mean that some modules were not found when using insmod?</div>
<div><br></div><div>For the epoll_create1 problem, what kind of patch did you apply to add support for the epoll_create1 function?<div><div><br><br><div class="gmail_quote">On Tue, Feb 26, 2013 at 11:29 AM, Amit Balboul <span dir="ltr"><<a href="mailto:amit.balboul@shinesec.com" target="_blank">amit.balboul@shinesec.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div style="font-family:arial,sans-serif;font-size:12.571428298950195px">
Ok I got :</div><div style="font-family:arial,sans-serif;font-size:12.571428298950195px">
lttng-kretprobes.ko</div><div style="font-family:arial,sans-serif;font-size:12.571428298950195px">
lttng-kprobes.ko</div><div style="font-family:arial,sans-serif;font-size:12.571428298950195px">, the other two are missing.</div><div><div><div style="font-family:arial,sans-serif;font-size:12.571428298950195px">
<br></div><br><div class="gmail_quote">
---------- Forwarded message ----------<br>From: <b class="gmail_sendername">Amit Balboul</b> <span dir="ltr"><<a href="mailto:amit.balboul@shinesec.com" target="_blank">amit.balboul@shinesec.com</a>></span><br>Date: Tue, Feb 26, 2013 at 5:49 PM<br>
Subject: Re: [lttng-dev] compiling LTTng for android<br>To: PLSTC <<a href="mailto:b0mb00z.it@gmail.com" target="_blank">b0mb00z.it@gmail.com</a>><br><br><br><div dir="ltr">One more thing:<div><div><br></div><div>The list of modules you gave my is not identical to mine:</div>
<div>a. I pushed the modules from my machine to my folder in the device (not to /system/lib) as you directed me, and insmoded them manually.</div>
<div>b. I pushed the whole tree under /lib/modules/<KERNEL_VER_BUILD>/ which includes the modules.* and the sub directories:</div><div><br></div><div><div><font face="courier new, monospace" size="1">/lib/modules/3.0.31-gd5a18e0/extra$ ll</font></div>
<div><font face="courier new, monospace" size="1">total 2464</font></div><div><font face="courier new, monospace" size="1">drwxr-xr-x 4 root root 4096 Feb 26 15:58 ./</font></div><div><font face="courier new, monospace" size="1">drwxr-xr-x 3 root root 4096 Feb 26 15:58 ../</font></div>
<div><font face="courier new, monospace" size="1">drwxr-xr-x 2 root root 4096 Feb 26 15:58 lib/</font></div><div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 167184 Feb 26 15:58 lttng-ring-buffer-client-discard.ko</font></div>
<div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 167206 Feb 26 15:58 lttng-ring-buffer-client-mmap-discard.ko</font></div><div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 178586 Feb 26 15:58 lttng-ring-buffer-client-mmap-overwrite.ko</font></div>
<div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 178564 Feb 26 15:58 lttng-ring-buffer-client-overwrite.ko</font></div><div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 137179 Feb 26 15:58 lttng-ring-buffer-metadata-client.ko</font></div>
<div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 137233 Feb 26 15:58 lttng-ring-buffer-metadata-mmap-client.ko</font></div><div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 213451 Feb 26 15:58 lttng-statedump.ko</font></div>
<div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 1314235 Feb 26 15:58 lttng-tracer.ko</font></div><div><font face="courier new, monospace" size="1">drwxr-xr-x 2 root root 4096 Feb 26 15:58 probes/</font></div>
<div><font face="courier new, monospace" size="1"><br></font></div><div><div><font face="courier new, monospace" size="1">/lib/modules/3.0.31-gd5a18e0/extra/lib$ ll</font></div><div><font face="courier new, monospace" size="1">total 732</font></div>
<div><font face="courier new, monospace" size="1">drwxr-xr-x 2 root root 4096 Feb 26 15:58 ./</font></div><div><font face="courier new, monospace" size="1">drwxr-xr-x 4 root root 4096 Feb 26 15:58 ../</font></div><div>
<font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 738964 Feb 26 15:58 lttng-lib-ring-buffer.ko</font></div></div><div><font face="courier new, monospace" size="1"><br></font></div><div><div><font face="courier new, monospace" size="1">/lib/modules/3.0.31-gd5a18e0/extra/probes$ ll</font></div>
<div><font face="courier new, monospace" size="1">total 2872</font></div><div><font face="courier new, monospace" size="1">drwxr-xr-x 2 root root 4096 Feb 26 15:58 ./</font></div><div><font face="courier new, monospace" size="1">drwxr-xr-x 4 root root 4096 Feb 26 15:58 ../</font></div>
<div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 162872 Feb 26 15:58 lttng-probe-asoc.ko</font></div><div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 171049 Feb 26 15:58 lttng-probe-block.ko</font></div>
<div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 81165 Feb 26 15:58 lttng-probe-compaction.ko</font></div><div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 333215 Feb 26 15:58 lttng-probe-ext4.ko</font></div>
<div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 80413 Feb 26 15:58 lttng-probe-gpio.ko</font></div><div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 84256 Feb 26 15:58 lttng-probe-irq.ko</font></div>
<div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 135453 Feb 26 15:58 lttng-probe-jbd2.ko</font></div><div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 105399 Feb 26 15:58 lttng-probe-kmem.ko</font></div>
<div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 76240 Feb 26 15:58 lttng-probe-lttng.ko</font></div><div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 88059 Feb 26 15:58 lttng-probe-module.ko</font></div>
<div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 154032 Feb 26 15:58 lttng-probe-napi.ko</font></div><div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 142038 Feb 26 15:58 lttng-probe-net.ko</font></div>
<div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 93564 Feb 26 15:58 lttng-probe-power.ko</font></div><div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 85530 Feb 26 15:58 lttng-probe-regulator.ko</font></div>
<div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 138121 Feb 26 15:58 lttng-probe-sched.ko</font></div><div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 152909 Feb 26 15:58 lttng-probe-scsi.ko</font></div>
<div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 107491 Feb 26 15:58 lttng-probe-signal.ko</font></div><div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 142711 Feb 26 15:58 lttng-probe-skb.ko</font></div>
<div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 181983 Feb 26 15:58 lttng-probe-statedump.ko</font></div><div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 126731 Feb 26 15:58 lttng-probe-timer.ko</font></div>
<div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 86591 Feb 26 15:58 lttng-probe-workqueue.ko</font></div><div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 133886 Feb 26 15:58 lttng-probe-writeback.ko</font></div>
<div><font face="courier new, monospace" size="1">-rw-r--r-- 1 root root 30863 Feb 26 15:58 lttng-types.ko</font></div></div><div><br></div><div>Looking at the files shows I miss:<br></div><div><div>lttng-kretprobes.ko</div>
<div>lttng-kprobes.ko</div><div>lttng-ftrace.ko</div><div>lttng-probe-kvm.ko</div></div></div></div></div><div class="gmail_extra"><br><br><div class="gmail_quote"><div>On Tue, Feb 26, 2013 at 3:56 PM, PLSTC <span dir="ltr"><<a href="mailto:b0mb00z.it@gmail.com" target="_blank">b0mb00z.it@gmail.com</a>></span> wrote:<br>
</div><div><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Back on the last email:<div><br>
</div><div>Here's the insmod list I ended up building; you might not have compiled all those modules, depending on your kernel config and on the architecture used, so some might simply be absent from your output directory.</div>
<div><br></div><div><pre style="margin-right:1em;margin-left:1.6em;padding:2px 2px 2px 0px;background-color:rgb(250,250,250);border:1px solid rgb(218,218,218);width:auto;color:rgb(72,72,72);font-size:12px">insmod /system/lib/modules/lttng-types.ko
insmod /system/lib/modules/lttng-kretprobes.ko
insmod /system/lib/modules/lttng-kprobes.ko
insmod /system/lib/modules/lttng-ftrace.ko
insmod /system/lib/modules/lttng-lib-ring-buffer.ko
insmod /system/lib/modules/lttng-statedump.ko
insmod /system/lib/modules/lttng-tracer.ko
insmod /system/lib/modules/lttng-probe-timer.ko
insmod /system/lib/modules/lttng-probe-statedump.ko
insmod /system/lib/modules/lttng-probe-signal.ko
insmod /system/lib/modules/lttng-probe-sched.ko
insmod /system/lib/modules/lttng-probe-lttng.ko
insmod /system/lib/modules/lttng-probe-kvm.ko
insmod /system/lib/modules/lttng-probe-irq.ko
insmod /system/lib/modules/lttng-probe-block.ko
insmod /system/lib/modules/lttng-ring-buffer-metadata-mmap-client.ko
insmod /system/lib/modules/lttng-ring-buffer-metadata-client.ko
insmod /system/lib/modules/lttng-ring-buffer-client-overwrite.ko
insmod /system/lib/modules/lttng-ring-buffer-client-mmap-overwrite.ko
insmod /system/lib/modules/lttng-ring-buffer-client-mmap-discard.ko
insmod /system/lib/modules/lttng-ring-buffer-client-discard.ko</pre><div><br></div><div>We also haven't had time recently to test if the kprobes/kretprobes worked, but they probably do; we decided to focus a bit more on the 'tracepoints' aspect instead.</div>
<div><br></div><div>Anyway, good luck with your exploration, and if you find something interesting, I'd love to know more about it!</div><div><br></div><div>-PL</div><div><div><br><div class="gmail_quote">
On Tue, Feb 26, 2013 at 8:06 AM, PLSTC <span dir="ltr"><<a href="mailto:b0mb00z.it@gmail.com" target="_blank">b0mb00z.it@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><p dir="ltr">Hey Amit,</p>
<p dir="ltr">Good news indeed it seems! </p>
<p dir="ltr">When it comes to pushing the libs/bin, we haven't found any better way than using adb push yet; of course, if all the projects were included directly in Android's external, we would simply flash the device after each build, but currently we're still having trouble generating the correct Android makefiles.</p>
<p dir="ltr">As for the modules, I might still have a dep load list somewhere I could send you in a few hours, I remember fighting to get the order right...</p>
<p dir="ltr">Also, once everything loads, could you tell me what kind of output 'lttng list -k' provides you? Here, we're having trouble listing the 'default' kernel trace points provided by HAVE_TRACEPOINTS, and the returned list is simply empty (which is a bit strange).</p>
<p dir="ltr">-PL</p>
</blockquote></div><br></div></div></div>
</blockquote></div></div></div><br></div>
</div><br></div></div></div>
</blockquote></div><br></div></div></div>
</blockquote></div><br></div>
</div></div></div><br></div></div>
</blockquote></div>