[ltt-dev] Kernel crashes when creating a trace
Mathieu Desnoyers
mathieu.desnoyers at polymtl.ca
Fri Nov 13 19:09:57 EST 2009
* Ashwin Tanugula (ashwin.tanugula at broadcom.com) wrote:
>
>
>
> > Hi Mathieu,
> >
> > I am able to generate and view the trace with the latest lttng patchset and ltt-control package.
> >
> > patch-2.6.31.6-lttng-0.168 and ltt-control-0.74-12112009
> >
> > I didn't even put "return 1;" at the beginning of trace_clock_async_tsc_read, get_trace_clock and put_trace_clock functions.
> >
>
> > >Great ! Thanks for your help with testing on MIPS32. Do you still get warnings when you start tracing ?
>
> Mathieu,
>
> I have tested it again and now it's now working now. Sorry for prematurely sending the email.
>
> I think we are back to where we were.
>
> With "return 1;" at the beginning of trace_clock_async_tsc_read, I can start the trace, but the kernel crashes while stopping the trace
>
OK.. let's continue to test with the "return 1;" in
trace_clock_async_tsc_read and try to fix the teardown crash first. Can
you try the new 0.169 ? I fixes a teardown race with the channel list.
Thanks,
Mathieu
> Controlling trace : trace1
>
> lttctl: Pausing trace
> lttctl: Destroying trace
> CPU 1 Unable to handle kernel paging request at virtual address 00000040, epc == 80233e68, ra == 80245
> 9d4
> Oops[#1]:
> Cpu 1
> $ 0 : 00000000 10008b01 8a886304 00010000
> $ 4 : 8a907d80 80233e18 8c072380 00000003
> $ 8 : 804c2a88 8c1cc668 800c0938 fffffff0
> $12 : 2aab3d60 018b6739 00000000 004048b8
> $16 : 8a907d9c 8c3df9a0 8bd81b88 00000004
> $20 : 8c019ec0 00000000 00404918 00000001
> $24 : 004048a4 800840a4
> $28 : 8ab28000 8ab29e18 7fbb0f08 802459d4
> Hi : 00000012
> Lo : 000180e1
> epc : 80233e68 ltt_chanbuf_free+0x50/0x2cc
> Not tainted
> ra : 802459d4 kref_put+0xdc/0x128
> Status: 10008b03 KERNEL EXL IE
> Cause : 00800008
> BadVA : 00000040
> PrId : 0002a044 (Broadcom BMIPS4380)
> Modules linked in:
> Process lttd (pid: 369, threadinfo=8ab28000, task=8c044078, tls=2aab9440)
> Stack : 00000000 00000000 00000200 8c0a6b8c 8c044078 2aabc930 8a907d80 8ab29f30
> 8a907d9c 80233e18 8bd81b88 8bd80a5c 8c019ec0 8b9a3b8c 00404918 7fbb0f7c
> 7fbb0f08 802459d4 8001d640 8001d604 00000200 8023ebc4 8a907d80 8c072380
> 8023413c 00000000 000001f8 8b81eb44 00000010 8c072380 00000010 80237070
> 00000010 8c072380 8bd81b88 8bd80a5c 800c35ec 800c35a4 8c072380 8b9a3b8c
> ...
> Call Trace:
> [<80233e68>] ltt_chanbuf_free+0x50/0x2cc
> [<802459d4>] kref_put+0xdc/0x128
> [<8023413c>] ltt_chanbuf_release_read+0x58/0x124
> [<80237070>] ltt_release+0x10/0x20
> [<800c35ec>] __fput+0xec/0x21c
> [<800bee4c>] filp_close+0x5c/0xa4
> [<800c0a00>] sys_close+0xc8/0x1a4
> [<8000339c>] stack_done+0x20/0x3c
>
>
> Code: 00021027 02621021 8c550000 <8ea30040> 14600091 00000000 8ea20044 14400085 00000000
> Disabling lock debugging due to kernel taint
> CPU 1 Unable to handle kernel paging request at virtual address 00000040, epc == 80233e68, ra == 80245
> 9d4
> Oops[#2]:
> Cpu 1
> $ 0 : 00000000 00000001 8a886300 00010000
> $ 4 : 8a907e00 80233e18 8c072500 00000003
> $ 8 : 804c2db8 8b85e5f0 00000000 8c046000
> $12 : ffffffff 8ab29938 804092d0 67677562
> $16 : 8a907e1c 8c3df9a0 8bd81cd8 00000000
> $20 : 8c019ec0 00000000 8b81eb74 00000000
> $24 : 00000000 800840a4
> $28 : 8ab28000 8ab29b20 7fbb0f08 802459d4
> Hi : 00000092
> Lo : 809d07c0
> epc : 80233e68 ltt_chanbuf_free+0x50/0x2cc
> Tainted: G D
> ra : 802459d4 kref_put+0xdc/0x128
> Status: 10008b03 KERNEL EXL IE
> Cause : 00800008
> BadVA : 00000040
> PrId : 0002a044 (Broadcom BMIPS4380)
> Modules linked in:
> Process lttd (pid: 369, threadinfo=8ab28000, task=8c044078, tls=2aab9440)
> Stack : 8005891c 8ab0c49c 8bd7b928 800db908 804c2db8 8c072440 8a907e00 0000002a
> 8a907e1c 80233e18 8bd81cd8 8bd80ae0 8c019ec0 804fe900 8b81eb74 00000001
> 7fbb0f08 802459d4 8bd7921c 8bd80ae0 8bd7b928 00000001 8a907e00 8c072500
> 8023413c 804d0000 804d0000 800e1498 00000010 8c072500 00000010 80237070
> 00000010 8c072500 8bd81cd8 8bd80ae0 800c35ec 800c35a4 00030000 8ab29d68
> ...
> Call Trace:
> [<80233e68>] ltt_chanbuf_free+0x50/0x2cc
> [<802459d4>] kref_put+0xdc/0x128
> [<8023413c>] ltt_chanbuf_release_read+0x58/0x124
> [<80237070>] ltt_release+0x10/0x20
> [<800c35ec>] __fput+0xec/0x21c
> [<800bee4c>] filp_close+0x5c/0xa4
> [<800412fc>] put_files_struct+0x1ec/0x228
> [<80042f8c>] do_exit+0x13c/0x764
> [<80018aa0>] die+0x108/0x110
> [<8001d7d0>] do_page_fault+0x31c/0x41c
> [<80001420>] ret_from_exception+0x0/0x24
> [<80233e68>] ltt_chanbuf_free+0x50/0x2cc
> [<802459d4>] kref_put+0xdc/0x128
> [<8023413c>] ltt_chanbuf_release_read+0x58/0x124
> [<80237070>] ltt_release+0x10/0x20
> [<800c35ec>] __fput+0xec/0x21c
> [<800bee4c>] filp_close+0x5c/0xa4
> [<800c0a00>] sys_close+0xc8/0x1a4
> [<8000339c>] stack_done+0x20/0x3c
>
>
>
> If I remove "return 1;" at the beginning of trace_clock_async_tsc_read, it does not even start the trace.
>
>
>
> Connecting /proc/sy------------[ cut here ]------------
> s/debug/ltt/markWARNING: at arch/mips/kernel/trace-clock.c:79 trace_clock_async_tsc_read+0x30/0x17c()
> ers/kernel/timerModules linked in:_itimer_set
> Con
> necting /proc/syCall Trace:
> s/debug/ltt/mark[<80018c9c>] dump_stack+0x8/0x34
> ers/kernel/timer[<8003e6b4>] warn_slowpath_common+0x70/0x98
> _set
> Connecting[<8001c0d8>] trace_clock_async_tsc_read+0x30/0x17c
> /proc/sys/debug[<80086b64>] trace_clock_read_synthetic_tsc+0x94/0xbc
> /ltt/markers/ker[<80086e60>] get_synthetic_tsc+0xe0/0x1a0
> nel/timer_timeou[<8001c2c8>] get_trace_clock+0x34/0x17c
> t
> Connecting /p[<80230354>] ltt_trace_alloc+0x60/0x578
> roc/sys/debug/lt[<8023ce48>] alloc_write+0x130/0x17c
> t/markers/kernel[<800c2168>] vfs_write+0xbc/0x180
> /timer_update_ti[<800c2c70>] sys_write+0x60/0x130
> me
> Connecting /[<8000339c>] stack_done+0x20/0x3c
> proc/sys/debug/l
> tt/markers/kerne---[ end trace 2942edc92828db6c ]---
> l/trap_entry
> Co------------[ cut here ]------------
> WARNING: at arch/mips/kernel/trace-clock.c:79 trace_clock_async_tsc_read+0x30/0x17c()
> Modules linked in:
> Call Trace:
> [<80018c9c>] dump_stack+0x8/0x34
> [<8003e6b4>] warn_slowpath_common+0x70/0x98
> [<8001c0d8>] trace_clock_async_tsc_read+0x30/0x17c
> [<80086b64>] trace_clock_read_synthetic_tsc+0x94/0xbc
> [<8001c12c>] trace_clock_async_tsc_read+0x84/0x17c
> [<80086b64>] trace_clock_read_synthetic_tsc+0x94/0xbc
> [<80086e60>] get_synthetic_tsc+0xe0/0x1a0
> [<8001c2c8>] get_trace_clock+0x34/0x17c
> [<80230354>] ltt_trace_alloc+0x60/0x578
> [<8023ce48>] alloc_write+0x130/0x17c
> [<800c2168>] vfs_write+0xbc/0x180
> [<800c2c70>] sys_write+0x60/0x130
> [<8000339c>] stack_done+0x20/0x3c
>
> ---[ end trace 2942edc92828db6d ]---
>
> Thanks,
> Ashwin
>
> > >Mathieu
>
>
>
--
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
More information about the lttng-dev
mailing list