[lttng-dev] LTTng system call tracing on ARM
Mathieu Desnoyers
mathieu.desnoyers at efficios.com
Wed Apr 10 12:20:12 EDT 2013
* Jan Glauber (jan.glauber at gmail.com) wrote:
> On Wed, Apr 10, 2013 at 09:38:33AM -0400, Pierre-Luc St-Charles wrote:
> > Here are the links I stashed a while back about previous patching attempts
> > to add syscall tracepoints to the ARM kernel :
> >
> > Most promising:
> > http://www.spinics.net/lists/arm-kernel/msg166419.html
> >
> > Earlier draft(s):
> > http://lists.infradead.org/pipermail/linux-arm-kernel/2012-February/086974.html
> > https://lkml.org/lkml/2011/12/1/131
> > http://comments.gmane.org/gmane.linux.ports.arm.kernel/141933
> >
> > I believe the first link shows the most refined patch there is out there,
> > but it might take some minor tinkering to apply it to a different kernel
> > version. I briefly tried to apply it to the 3.0.31 kernel, but it's a bit
> > out of my 'tinkering' range, and I never finished it.
> >
> >
> > Good luck!
>
> Thanks, fortunately I'm targeting 3.4. It wasn't too hard, got the kernel running
> and ftrace already works.
>
> Now I'm facing something hilarious, on loading the lttng-tracer module I get:
>
> Error: could not insert module lttng-tracer.ko: Cannot allocate memory
>
> [15912.259399] alloc_vmap_area: 2 callbacks suppressed
> [15912.264953] vmap allocation for size 15998976 failed: use vmalloc=<size> to increase size.
> [15912.273895] vmalloc: allocation failure: 15991653 bytes
> [15912.279602] insmod: page allocation failure: order:0, mode:0xd0
>
> Never saw vmalloc fail on loading a kernel module before. Hmmm.
> Looking at the module reveals:
> -rw-rw-r-- 1 jang jang 16036281 Apr 10 17:48 lttng-tracer.ko
>
> Wow. 16 megs for a kernel module ?!? readelf says its all in .rodata.
>
> Doing an objdump on that
> Disassembly of section .rodata:
>
> 00000000 <sc_table>:
> ...
> 1714: 00f0a788 rscseq sl, r0, r8, lsl #15
> 1718: 00f0a7a0 rscseq sl, r0, r0, lsr #15
> 171c: 00000004 andeq r0, r0, r4
> ...
> f00054: 00f0a860 rscseq sl, r0, r0, ror #16
> f00058: 00f0a878 rscseq sl, r0, r8, ror r8
> f0005c: 00000002 andeq r0, r0, r2
>
> Now whats that? A big hole in the system call table?
>
> Digging through the various defines and headers of lttng-modules I've found:
>
> instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h:
> ...
> #define OVERRIDE_TABLE_32_sys_set_tls
> TRACE_SYSCALL_TABLE(sys_set_tls, sys_set_tls, 0xf0005, 2)
>
> Weird ARM apparently expanded the system call area above 0xf0000. I don't
> understand why this override logic is needed but removing that define results
> in a kernel module with a much more sane size.
>
> LTTng system call tracing works with that change for me.
>
> Any idea on how the override problem can be solved with the magic system calls?
Thanks for the detailed report !!
Please try with this commit:
commit 11fa478f18241fedee86f7dc7820a91c629c9e7e
Author: Mathieu Desnoyers <mathieu.desnoyers at efficios.com>
Date: Wed Apr 10 12:14:38 2013 -0400
Fix: remove ARM set_tls system call override
We'll need to find a better way to instrument ARM-specific system calls
located at a far offset from the standard systems calls. A 16MB
lttng-modules kernel module is really not acceptable.
Removing this instrumentation for now. sys_set_tls will appear as
sys_unknown.
Fixes #472
CC: Ryan Kyser <Ryan.Kyser at jci.com>
Ref: http://lists.lttng.org/pipermail/lttng-dev/2013-April/019990.html
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers at efficios.com>
diff --git a/instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h b/instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h
index 93b8674..895370f 100644
--- a/instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h
+++ b/instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h
@@ -2,7 +2,6 @@
#define OVERRIDE_TABLE_32_sys_arm_fadvise64_64
#define OVERRIDE_TABLE_32_sys_sync_file_range2
-#define OVERRIDE_TABLE_32_sys_set_tls
#ifndef CREATE_SYSCALL_TABLE
@@ -38,18 +37,6 @@ SC_TRACE_EVENT(sys_sync_file_range2,
TP_printk()
)
-SC_TRACE_EVENT(sys_set_tls,
- TP_PROTO(unsigned int tid, unsigned long tls),
- TP_ARGS(tid, tls),
- TP_STRUCT__entry(
- __field(unsigned int, tid)
- __field_hex(unsigned int, tls)),
- TP_fast_assign(
- tp_assign(tid, tid)
- tp_assign(tls, tls)),
- TP_printk()
-)
-
#else /* CREATE_SYSCALL_TABLE */
#define OVVERRIDE_TABLE_32_sys_mmap
@@ -59,9 +46,6 @@ TRACE_SYSCALL_TABLE(sys_mmap, sys_mmap, 90, 6)
TRACE_SYSCALL_TABLE(sys_arm_fadvise64_64, sys_arm_fadvise64_64, 270, 4)
#define OVERRIDE_TABLE_32_sys_sync_file_range2
TRACE_SYSCALL_TABLE(sys_sync_file_range2, sys_sync_file_range2, 341, 4)
-#define OVERRIDE_TABLE_32_sys_set_tls
-TRACE_SYSCALL_TABLE(sys_set_tls, sys_set_tls, 0xf0005, 2)
-
#endif /* CREATE_SYSCALL_TABLE */
>
> --Jan
>
> > -PL
> >
> >
> >
> > On Wed, Apr 10, 2013 at 9:02 AM, Jan Glauber <jan.glauber at gmail.com> wrote:
> >
> > > On Wed, Apr 10, 2013 at 08:37:18AM -0400, PLSTC wrote:
> > > > Hey Jan,
> > > >
> > > > I cannot speak on behalf of everyone here, but during our attempt to port
> > > > LTTng to Android, we also noticed that the kernels we were using (3.0.x)
> > > > were nowhere near the requirements for syscall tracepoints support. I
> > > > believe such support was added on x86/64 way earlier (early 3.x) than on
> > > > ARM, which is why it was included in LTTng's modules a while ago. Simply
> > > > put, the ARM kernel is late.
> > >
> > > OK, so for ARM kernel version 3.6 is the minimum unless the syscall
> > > tracepoint
> > > support is backported.
> > >
> > > > There are a few actuals ways to 'enable' syscall tracepoints support on
> > > > early ARM kernels, but they all including a bit of kernel
> > > hacking/patching.
> > > > I could send you some links if you're interested in that.
> > >
> > > Yes, sure!
> > >
> > > --Jan
> > >
> > > > -PL
> > > > On Apr 10, 2013 4:40 AM, "Jan Glauber" <jan.glauber at gmail.com> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I want to use LTTng for system call tracing on ARM. Now lttng-modules
> > > seems
> > > > > to support system call tracing on ARM already since
> > > > > "8f4f80e LTTng Modules ARM syscall instrumentation".
> > > > >
> > > > > But I wonder how that worked since lttng-syscalls.c is only build under
> > > > > CONFIG_HAVE_SYSCALL_TRACEPOINTS and that was added to ARM only with
> > > kernel
> > > > > 3.6
> > > > > (much after than the lttng-modules commit).
> > > > >
> > > > > Am I missing something? Is system call tracing working on ARM with the
> > > > > upstream
> > > > > LTTng version?
> > > > >
> > > > > thanks,
> > > > > Jan
> > > > >
> > > > > _______________________________________________
> > > > > lttng-dev mailing list
> > > > > lttng-dev at lists.lttng.org
> > > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> > > > >
> > >
>
> _______________________________________________
> lttng-dev mailing list
> lttng-dev at lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
More information about the lttng-dev
mailing list