[ltt-dev] kernel oops with LTTng 0.178 with linux 2.6.31.6-rt19

Tue Mar 16 12:14:14 EDT 2010

Hello,

I disabled the CONFIG_LTT_STATEDUMP and have been able to run lttctl on my
appln. Thank you !

However, i am now facing another issue when i try to insmod my fpga driver
on the board.

$ insmod fpga.ko
BUG: unable to handle kernel NULL pointer dereference at 00000004
IP: [<c1049791>] marker_update_probe_range+0xa1/0x310
*pde = 00000000
Oops: 0000 [#1] PREEMPT
LTT NESTING LEVEL : 0
last sysfs file:
Modules linked in: fpga(+)
Pid: 799, comm: insmod Not tainted (2.6.31.6-rt19 #4) H6037
EIP: 0060:[<c1049791>] EFLAGS: 00010203 CPU: 0
EIP is at marker_update_probe_range+0xa1/0x310
EAX: 00000000 EBX: 00000000 ECX: cff42aa0 EDX: cf2cf070
ESI: 00000000 EDI: c126bac8 EBP: fdc60fc0 ESP: cf0ddf14
 DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068 preempt:00000000
Process insmod (pid: 799, ti=cf0dc000 task=cf2cf070 task.ti=cf0dc000)
Stack:
 00000000 00000000 cf0ddf7c 00000000 00047fd4 00000001 cff5d000 0000002b
<0> c126baa8 00000000 c126bac8 fffffffe c1049a2a c10335b6 00000001 cff42aa0
<0> c1263ce0 c1263cf8 00000001 00000000 c103388a ffffffff 00000000 cff42aa0
Call Trace:
 [<c1049a2a>] ? marker_module_notify+0x2a/0x30
 [<c10335b6>] ? notifier_call_chain+0x36/0x70
 [<c103388a>] ? __blocking_notifier_call_chain+0x4a/0x70
 [<c10338c7>] ? blocking_notifier_call_chain+0x17/0x20
 [<c10436e6>] ? sys_init_module+0x86/0x220
 [<c1002e44>] ? sysenter_do_call+0x12/0x26
Code: 08 08 0f 85 54 02 00 00 85 ff 0f 84 44 02 00 00 8b 53 28 8b 43 24 e8
0f 1a
 00 00 89 f0 88 43 0c 83 c3 30 39 dd 0f 86 3f 01 00 00 <8b> 53 04 8b 03 e8
b5 f0
 ff ff 85 c0 89 c7 0f 84 db 00 00 00 8b
EIP: [<c1049791>] marker_update_probe_range+0xa1/0x310 SS:ESP 0068:cf0ddf14
CR2: 0000000000000004
---[ end trace 0a7239d58be85cb5 ]---
Killed
$ Clocksource tsc unstable (delta = 130894527 ns)

Is this related to the tracer or a fault my fpga module ?

Thanks in adv
rachana

On Tue, Mar 16, 2010 at 7:22 PM, Mathieu Desnoyers <
compudj at krystal.dyndns.org> wrote:

> * Rachana Rao (rachana.rtuser at gmail.com) wrote:
> > Hello,
> >
> > I am using LTTng version 0.178 on linux 2.6.31.6 patched with rt prempt
> > patch-2.6.31.6-rt19
> > LTT control version is 0.79
> >
> > This is running on Celeron M 370 processor (1.5GHz) board.
>
> It seems to be caused by the ltt-statedump module, which can be disabled
> with CONFIG_LTT_STATEDUMP=n.
>
> Make sure you have no warnings when ltt-statedump compiles.. maybe
> there is a spinlock taken there that happens to become a raw spinlock in
> RT...
>
> Thanks,
>
> Mathieu
>
> >
> > *I am getting the following oops -
>  > *
> > $ lttctl -C -w /home/app1/trace7 trace7
> > Linux Trace Toolkit Trace Control 0.79-01022010
> >
> > Controlling trace : trace7
> >
> > lttctl: Creating trace
> > lttctl: Forking lttd
> > Linux Trace Toolkit Trace Daemon 0.79-01022010
> >
> > ReadingBUG: unable to handle kernel NULL pointer dereference at 00000008
> > IP: [<c10f4f0b>] plist_add+0x1b/0x90
> > *pde = 00000000
> > Oops: 0000 [#1] PREEMPT
> > LTT NESTING LEVEL : 0
> > last sysfs file:
> > Modules linked in:
> >
> > Pid: 839, comm: lttctl Not tainted (2.6.31.6-rt19 #3) H6037
> > EIP: 0060:[<c10f4f0b>] EFLAGS: 00010046 CPU: 0
> > EIP is at plist_add+0x1b/0x90
> > EAX: 00000004 EBX: c1266120 ECX: cf261e74 EDX: c1266120
> > ESI: cf261e7c EDI: cf2ce750 EBP: 00000008 ESP: cf261e18
> >  DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068 preempt:00000002
> > Process lttctl (pid: 839, ti=cf260000 task=cf2ce750 task.ti=cf260000)
> > Stack:
> >  cf261e88 cf261e68 c1265dfc cf2ce750 c1265e0c c103eb41 00000001 00000041
> > <0> cf261e74 cf261e68 c1266120 00000000 00000282 c1265dfc ffffffff
> c11dd97c
> > <0> 00000000 00000282 cec98cfc c1265e04 00000078 c1265dfc c1265dfc
> c1265e04
> > Call Trace:
> >  [<c103eb41>] ? task_blocks_on_rt_mutex+0x141/0x1c0
> >  [<c11dd97c>] ? rt_spin_lock_slowlock+0x18c/0x200
> >  [<c10e4d9d>] ? ltt_statedump_start+0x2fd/0x7a0
> >  [<c104a6ee>] ? marker_probe_cb+0x4e/0xc0
> >  [<c10de3d2>] ? ltt_trace_start+0x122/0x270
> >  [<c10e70e8>] ? enabled_write+0xd8/0x160
> >  [<c10e7010>] ? enabled_write+0x0/0x160
> >  [<c106dfd0>] ? vfs_write+0x90/0xc0
> >  [<c106e173>] ? sys_write+0x53/0xf0
> >  [<c1002eb1>] ? syscall_call+0x7/0xb
> > Code: 04 89 50 10 8b 7c 24 08 83 c4 0c c3 8d 74 26 00 55 57 56 89 c6 53
> 89
> > d3 83
> >  ec 04 8d 40 0c 3b 46 0c 89 04 24 75 67 8b 2b 8d 45 fc <8b> 50 04 0f 18
> 02
> > 90 39
> >  dd 89 e9 74 1f 8b 3e 3b 7d fc 7c 18 90
> > EIP: [<c10f4f0b>] plist_add+0x1b/0x90 SS:ESP 0068:cf261e18
> > CR2: 0000000000000008
> > ---[ end trace 18077621483d0c39 ]---
> > note: lttctl[839] exited with preempt_count 2
> >  from debugfs directory : /mnt/debugfs/ltt/trace7
> > Writing to trace directory : /home/app1/trace7
> >
> > lttctl: Starting trace
> > Killed
> >
> > $ Clocksource tsc unstable (delta = 146049025 ns)
> >
> > *Following are excerpts from the kernel config -
> >
> > *#
> > # GCOV-based kernel profiling
> > #
> > # CONFIG_GCOV_KERNEL is not set
> > # CONFIG_SLOW_WORK is not set
> > CONFIG_LTT=y
> > CONFIG_LTT_FILTER=y
> > CONFIG_HAVE_LTT_DUMP_TABLES=y
> > CONFIG_LTT_RELAY=y
> > CONFIG_LTT_RELAY_LOCKLESS=y
> > # CONFIG_LTT_RELAY_IRQOFF is not set
> > # CONFIG_LTT_RELAY_LOCKED is not set
> > CONFIG_LTT_SERIALIZE=y
> > CONFIG_LTT_FAST_SERIALIZE=y
> > CONFIG_LTT_TRACEPROBES=y
> > CONFIG_LTT_TRACE_CONTROL=y
> > CONFIG_LTT_TRACER=y
> > CONFIG_LTT_ALIGNMENT=y
> > CONFIG_LTT_CHECK_ARCH_EFFICIENT_UNALIGNED_ACCESS=y
> > # CONFIG_LTT_DEBUG_EVENT_SIZE is not set
> > CONFIG_LTT_USERSPACE_EVENT=y
> > CONFIG_LTT_VMCORE=y
> > CONFIG_LTT_STATEDUMP=y
> > # CONFIG_LTT_ASCII is not set
> > CONFIG_HAVE_IMMEDIATE=y
> > CONFIG_IMMEDIATE=y
> > CONFIG_HAVE_GENERIC_DMA_COHERENT=y
> > CONFIG_SLABINFO=y
> > CONFIG_RT_MUTEXES=y
> > CONFIG_BASE_SMALL=0
> > CONFIG_MODULES=y
> > CONFIG_MODULE_FORCE_LOAD=y
> > CONFIG_MODULE_UNLOAD=y
> > # CONFIG_MODULE_FORCE_UNLOAD is not set
> > # CONFIG_MODVERSIONS is not set
> > # CONFIG_MODULE_SRCVERSION_ALL is not set
> > CONFIG_BLOCK=y
> > # CONFIG_LBDAF is not set
> > CONFIG_BLK_DEV_BSG=y
> > # CONFIG_BLK_DEV_INTEGRITY is not set
> >
> > #
> > # RCU Subsystem
> > #
> > # CONFIG_CLASSIC_RCU is not set
> > # CONFIG_TREE_RCU is not set
> > CONFIG_PREEMPT_RCU=y
> > # CONFIG_RCU_TRACE is not set
> > # CONFIG_TREE_RCU_TRACE is not set
> > # CONFIG_PREEMPT_RCU_TRACE is not set
> > # CONFIG_IKCONFIG is not set
> > CONFIG_LOG_BUF_SHIFT=17
> > CONFIG_HAVE_UNSTABLE_SCHED_CLOCK=y
> > CONFIG_HAVE_GET_CYCLES=y
> > CONFIG_HAVE_TRACE_CLOCK=y
> > # CONFIG_HAVE_TRACE_CLOCK_GENERIC is not set
> > # CONFIG_HAVE_TRACE_CLOCK_32_TO_64 is not set
> > # CONFIG_HAVE_UNSYNCHRONIZED_TSC is not set
> > # CONFIG_GROUP_SCHED is not set
> > # CONFIG_CGROUPS is not set
> > # CONFIG_SYSFS_DEPRECATED_V2 is not set
> > # CONFIG_RELAY is not set
> > # CONFIG_NAMESPACES is not set
> > # CONFIG_BLK_DEV_INITRD is not set
> > # CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
> > CONFIG_SYSCTL=y
> > CONFIG_ANON_INODES=y
> > CONFIG_EMBEDDED=y
> > CONFIG_UID16=y
> > CONFIG_SYSCTL_SYSCALL=y
> > CONFIG_KALLSYMS=y
> > # CONFIG_KALLSYMS_EXTRA_PASS is not set
> > CONFIG_HOTPLUG=y
> > CONFIG_PRINTK=y
> > CONFIG_BUG=y
> > CONFIG_ELF_CORE=y
> > CONFIG_PCSPKR_PLATFORM=y
> > CONFIG_BASE_FULL=y
> > CONFIG_FUTEX=y
> > CONFIG_EPOLL=y
> > CONFIG_SIGNALFD=y
> > CONFIG_TIMERFD=y
> > CONFIG_EVENTFD=y
> > CONFIG_SHMEM=y
> > CONFIG_AIO=y
> > CONFIG_HAVE_PERF_COUNTERS=y
> >
> > *
> > *
> > Do let me know if you need the entire .config file or any other details
> >
> > I would appreciate any kind of help with this issue.
> >
> > Thanks in adv
> > rachana
>
> > _______________________________________________
> > ltt-dev mailing list
> > ltt-dev at lists.casi.polymtl.ca
> > http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
>
>
> --
> Mathieu Desnoyers
> Operating System Efficiency Consultant
> EfficiOS Inc.
> http://www.efficios.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.casi.polymtl.ca/pipermail/lttng-dev/attachments/20100316/df31f03d/attachment-0003.htm>